[PR #24874] [CLOSED] fix: remove duplicate allow_redirects kwarg crashing SafeWebBaseLoader._fetch #131563

New Issue

GiteaMirror · 2026-05-21T17:13:31-05:00

GiteaMirror commented

2026-05-21 17:13:31 -05:00

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/24874
Author: @offbyonebit
Created: 5/18/2026
Status: ❌ Closed

Base: main ← Head: fix/safe-web-loader-allow-redirects-duplicate-kwarg

📝 Commits (1)

e72b771 fix: remove duplicate allow_redirects kwarg in SafeWebBaseLoader._fetch

📊 Changes

1 file changed (+0 additions, -1 deletions)

View changed files

📝 backend/open_webui/retrieval/web/utils.py (+0 -1)

📄 Description

Summary

SafeWebBaseLoader.__init__ stores AIOHTTP_CLIENT_ALLOW_REDIRECTS inside self.requests_kwargs:

# utils.py line 503-506
self.requests_kwargs = {
    **(self.requests_kwargs or {}),
    'allow_redirects': AIOHTTP_CLIENT_ALLOW_REDIRECTS,
}

_fetch then unpacks self.requests_kwargs and passes allow_redirects as an explicit keyword argument:

# utils.py line 521-524
async with session.get(
    url,
    **(self.requests_kwargs | kwargs),   # already contains allow_redirects
    allow_redirects=AIOHTTP_CLIENT_ALLOW_REDIRECTS,  # duplicate → TypeError
) as response:

Python raises:

TypeError: aiohttp.client.ClientSession.get() got multiple values for keyword argument 'allow_redirects'

Because continue_on_failure=True, the exception is silently swallowed for every URL. Web search always returns zero sources ("no sources found"), regardless of what SearXNG returns.

Fix

Remove the redundant explicit keyword argument from session.get(). AIOHTTP_CLIENT_ALLOW_REDIRECTS is already carried by self.requests_kwargs, so the SSRF redirect-blocking protection added in v0.9.5 (ref: GHSA-rh5x-h6pp-cjj6) is fully preserved.

Reproduction

import asyncio, aiohttp

async def test():
    async with aiohttp.ClientSession() as session:
        async with session.get(
            'https://example.com',
            **{'allow_redirects': False},
            allow_redirects=False,   # TypeError here
        ) as r:
            pass

asyncio.run(test())
# TypeError: aiohttp.client.ClientSession.get() got multiple values for keyword argument 'allow_redirects'

Affected version

v0.9.5 — the duplicate was introduced by the SSRF fix that added allow_redirects to requests_kwargs.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/24874 **Author:** [@offbyonebit](https://github.com/offbyonebit) **Created:** 5/18/2026 **Status:** ❌ Closed **Base:** `main` ← **Head:** `fix/safe-web-loader-allow-redirects-duplicate-kwarg` --- ### 📝 Commits (1) - [`e72b771`](https://github.com/open-webui/open-webui/commit/e72b771e83e531003b03610e8a074afbad9db9f0) fix: remove duplicate allow_redirects kwarg in SafeWebBaseLoader._fetch ### 📊 Changes **1 file changed** (+0 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/retrieval/web/utils.py` (+0 -1) </details> ### 📄 Description ## Summary `SafeWebBaseLoader.__init__` stores `AIOHTTP_CLIENT_ALLOW_REDIRECTS` inside `self.requests_kwargs`: ```python # utils.py line 503-506 self.requests_kwargs = { **(self.requests_kwargs or {}), 'allow_redirects': AIOHTTP_CLIENT_ALLOW_REDIRECTS, } ``` `_fetch` then unpacks `self.requests_kwargs` **and** passes `allow_redirects` as an explicit keyword argument: ```python # utils.py line 521-524 async with session.get( url, **(self.requests_kwargs | kwargs), # already contains allow_redirects allow_redirects=AIOHTTP_CLIENT_ALLOW_REDIRECTS, # duplicate → TypeError ) as response: ``` Python raises: ``` TypeError: aiohttp.client.ClientSession.get() got multiple values for keyword argument 'allow_redirects' ``` Because `continue_on_failure=True`, the exception is silently swallowed for **every** URL. Web search always returns zero sources ("no sources found"), regardless of what SearXNG returns. ## Fix Remove the redundant explicit keyword argument from `session.get()`. `AIOHTTP_CLIENT_ALLOW_REDIRECTS` is already carried by `self.requests_kwargs`, so the SSRF redirect-blocking protection added in v0.9.5 (ref: GHSA-rh5x-h6pp-cjj6) is fully preserved. ## Reproduction ```python import asyncio, aiohttp async def test(): async with aiohttp.ClientSession() as session: async with session.get( 'https://example.com', **{'allow_redirects': False}, allow_redirects=False, # TypeError here ) as r: pass asyncio.run(test()) # TypeError: aiohttp.client.ClientSession.get() got multiple values for keyword argument 'allow_redirects' ``` ## Affected version v0.9.5 — the duplicate was introduced by the SSRF fix that added `allow_redirects` to `requests_kwargs`. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-05-21 17:13:31 -05:00

GiteaMirror closed this issue

2026-05-21 17:13:36 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#131563