github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-07-18 10:48:00 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[GH-ISSUE #24875] bug: SafeWebBaseLoader._fetch raises TypeError on every URL, silently breaking all web search (v0.9.5) #123731

New Issue
Closed
opened 2026-05-21 03:12:35 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-05-21 03:12:35 -05:00
Owner
Copy Link

Originally created by @offbyonebit on GitHub (May 18, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/24875

Describe the bug

Web search returns no sources for every query when using the default safe_web loader (SearXNG or any other engine). SearXNG returns results normally but Open WebUI reports zero sources after the fetch step.

Root cause

SafeWebBaseLoader.__init__ stores AIOHTTP_CLIENT_ALLOW_REDIRECTS in self.requests_kwargs:

# retrieval/web/utils.py line 503-506
self.requests_kwargs = {
    **(self.requests_kwargs or {}),
    'allow_redirects': AIOHTTP_CLIENT_ALLOW_REDIRECTS,
}

_fetch then unpacks self.requests_kwargs and also passes allow_redirects as an explicit keyword argument:

# retrieval/web/utils.py line 521-524
async with session.get(
    url,
    **(self.requests_kwargs | kwargs),   # already contains allow_redirects
    allow_redirects=AIOHTTP_CLIENT_ALLOW_REDIRECTS,  # duplicate → TypeError
) as response:

Python raises:

TypeError: aiohttp.client.ClientSession.get() got multiple values for keyword argument 'allow_redirects'

continue_on_failure=True silently swallows this for every URL, so the fetch step always produces zero documents.

Minimal reproduction

import asyncio, aiohttp

async def test():
    async with aiohttp.ClientSession() as session:
        async with session.get(
            'https://example.com',
            **{'allow_redirects': False},
            allow_redirects=False,
        ) as r:
            pass

asyncio.run(test())
# TypeError: aiohttp.client.ClientSession.get() got multiple values for keyword argument 'allow_redirects'

Expected behavior

Web search returns page content from the fetched URLs.

Affected version

v0.9.5 — the duplicate was introduced when allow_redirects was added to requests_kwargs as part of the SSRF redirect-blocking fix (GHSA-rh5x-h6pp-cjj6), but was not removed from the explicit session.get() call.

Fix

Remove the redundant allow_redirects=AIOHTTP_CLIENT_ALLOW_REDIRECTS from the session.get() call in _fetch. The value is already carried by self.requests_kwargs, so the SSRF protection is fully preserved.

Originally created by @offbyonebit on GitHub (May 18, 2026). Original GitHub issue: https://github.com/open-webui/open-webui/issues/24875 ## Describe the bug Web search returns **no sources** for every query when using the default `safe_web` loader (SearXNG or any other engine). SearXNG returns results normally but Open WebUI reports zero sources after the fetch step. ## Root cause `SafeWebBaseLoader.__init__` stores `AIOHTTP_CLIENT_ALLOW_REDIRECTS` in `self.requests_kwargs`: ```python # retrieval/web/utils.py line 503-506 self.requests_kwargs = { **(self.requests_kwargs or {}), 'allow_redirects': AIOHTTP_CLIENT_ALLOW_REDIRECTS, } ``` `_fetch` then unpacks `self.requests_kwargs` **and** also passes `allow_redirects` as an explicit keyword argument: ```python # retrieval/web/utils.py line 521-524 async with session.get( url, **(self.requests_kwargs | kwargs), # already contains allow_redirects allow_redirects=AIOHTTP_CLIENT_ALLOW_REDIRECTS, # duplicate → TypeError ) as response: ``` Python raises: ``` TypeError: aiohttp.client.ClientSession.get() got multiple values for keyword argument 'allow_redirects' ``` `continue_on_failure=True` silently swallows this for every URL, so the fetch step always produces zero documents. ## Minimal reproduction ```python import asyncio, aiohttp async def test(): async with aiohttp.ClientSession() as session: async with session.get( 'https://example.com', **{'allow_redirects': False}, allow_redirects=False, ) as r: pass asyncio.run(test()) # TypeError: aiohttp.client.ClientSession.get() got multiple values for keyword argument 'allow_redirects' ``` ## Expected behavior Web search returns page content from the fetched URLs. ## Affected version v0.9.5 — the duplicate was introduced when `allow_redirects` was added to `requests_kwargs` as part of the SSRF redirect-blocking fix (GHSA-rh5x-h6pp-cjj6), but was not removed from the explicit `session.get()` call. ## Fix Remove the redundant `allow_redirects=AIOHTTP_CLIENT_ALLOW_REDIRECTS` from the `session.get()` call in `_fetch`. The value is already carried by `self.requests_kwargs`, so the SSRF protection is fully preserved.
GiteaMirror commented 2026-05-21 03:12:38 -05:00
Author
Owner
Copy Link

@owui-terminator[bot] commented on GitHub (May 18, 2026):

🔍 Related Issues Found

I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions:

  1. 🟢 #24793 issue: Web search discards all results — duplicate allow_redirects keyword in SafeWebBaseLoader._fetch()
    This is the same underlying bug: web search results are discarded because SafeWebBaseLoader._fetch() passes allow_redirects twice, causing a TypeError on every fetch in v0.9.5.
    by web2bruno · bug

  2. 🟣 #24560 issue: SafeWebBaseLoader._fetch crashes with TypeError on duplicate allow_redirects kwarg, breaking all web search in v0.9.5
    Closed report of the identical root cause and symptom set: silent fetch failures in safe_web due to duplicate allow_redirects, which causes all web search sources to be empty.
    by blaknite

  3. 🟣 #24630 bug: SafeWebBaseLoader._fetch() passes allow_redirects twice → TypeError on every async web search fetch
    Another direct duplicate describing the same aiohttp.ClientSession.get() multiple-values error from SafeWebBaseLoader._fetch(), with web search returning zero content/sources.
    by da-astro · bug

💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead.

This comment was generated automatically. React with 👍 if helpful, 👎 if not.

<!-- gh-comment-id:4478827269 --> @owui-terminator[bot] commented on GitHub (May 18, 2026): <!-- terminator-bot:related-issues-reply --> 🔍 **Related Issues Found** I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions: 1. 🟢 [#24793](https://github.com/open-webui/open-webui/issues/24793) **issue: Web search discards all results — duplicate allow_redirects keyword in SafeWebBaseLoader._fetch()** *This is the same underlying bug: web search results are discarded because `SafeWebBaseLoader._fetch()` passes `allow_redirects` twice, causing a `TypeError` on every fetch in v0.9.5.* *by web2bruno · `bug`* 2. 🟣 [#24560](https://github.com/open-webui/open-webui/issues/24560) **issue: SafeWebBaseLoader._fetch crashes with TypeError on duplicate allow_redirects kwarg, breaking all web search in v0.9.5** *Closed report of the identical root cause and symptom set: silent fetch failures in `safe_web` due to duplicate `allow_redirects`, which causes all web search sources to be empty.* *by blaknite* 3. 🟣 [#24630](https://github.com/open-webui/open-webui/issues/24630) **bug: SafeWebBaseLoader._fetch() passes `allow_redirects` twice → TypeError on every async web search fetch** *Another direct duplicate describing the same `aiohttp.ClientSession.get()` multiple-values error from `SafeWebBaseLoader._fetch()`, with web search returning zero content/sources.* *by da-astro · `bug`* --- 💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead. *This comment was generated automatically.* React with 👍 if helpful, 👎 if not.
GiteaMirror commented 2026-05-21 03:12:39 -05:00
Author
Owner
Copy Link

@offbyonebit commented on GitHub (May 18, 2026):

Confirming that the fix is already present in the dev branch (the allow_redirects duplicate is gone from utils.py there). This issue documents the regression for users on the v0.9.5 release so they know what they're hitting and can either patch the file manually or wait for the next release.

<!-- gh-comment-id:4478912285 --> @offbyonebit commented on GitHub (May 18, 2026): Confirming that the fix is already present in the `dev` branch (the `allow_redirects` duplicate is gone from `utils.py` there). This issue documents the regression for users on the v0.9.5 release so they know what they're hitting and can either patch the file manually or wait for the next release.
GiteaMirror commented 2026-05-21 03:12:40 -05:00
Author
Owner
Copy Link

@Classic298 commented on GitHub (May 18, 2026):

I believe this is fixed in dev and i think this is the third duplicate. Please search for duplicates before opening an issue.

<!-- gh-comment-id:4478929446 --> @Classic298 commented on GitHub (May 18, 2026): I believe this is fixed in dev and i think this is the third duplicate. Please search for duplicates before opening an issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#123731
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?