[GH-ISSUE #24017] issue: Web search breaks persistently after setting Fetch URL Content Length Limit to 80000 #35682

Closed
opened 2026-04-25 09:52:22 -05:00 by GiteaMirror · 1 comment
Owner

Originally created by @mero20j1 on GitHub (Apr 22, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/24017

Check Existing Issues

  • I have searched for any existing and/or related issues.
  • I have searched for any existing and/or related discussions.
  • I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!).
  • I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

0.9.1

Ollama Version (if applicable)

No response

Operating System

Ubuntu

Browser (if applicable)

No response

Confirmation

  • I have read and followed all instructions in README.md.
  • I am using the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.
  • I have provided every relevant configuration, setting, and environment variable used in my setup.
  • I have clearly listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc).
  • I have documented step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation. My steps:
  • Start with the initial platform/version/OS and dependencies used,
  • Specify exact install/launch/configure commands,
  • List URLs visited, user input (incl. example values/emails/passwords if needed),
  • Describe all options and toggles enabled or changed,
  • Include any files or environmental changes,
  • Identify the expected and actual result at each stage,
  • Ensure any reasonably skilled user can follow and hit the same issue.

Expected Behavior

Summary
I found what looks like a persistent configuration bug in web search / retrieval settings.

When I set Fetch URL Content Length Limit to 80000 in the OpenWebUI web search settings, web search starts failing.
The main problem is that the failure persists even after changing the value back in the UI.

At that point, the instance seems stuck in a broken state until I restore from backup.

⸻

Environment

  • OpenWebUI v0.9.1
  • Search engine: SearXNG
  • Web loader: Firecrawl
  • Firecrawl is local and working correctly when tested directly with curl
  • SearXNG is also local and working

Example setup:

⸻

What I changed

In Admin Settings → Web Search / Retrieval, I set:

  • Fetch URL Content Length Limit = 80000

After that, web search started failing.

Even if I later:

  • change the value again in the UI
  • reset the setting
  • restart Docker / container

…the issue remains.

I had to restore from backup to get back to a working state.

⸻

Expected behavior

Changing Fetch URL Content Length Limit should either:

  • work correctly, or
  • fail safely and be reversible by changing the setting back

It should not leave the web search pipeline in a permanently broken state.

⸻

Actual behavior

After setting Fetch URL Content Length Limit = 80000, OpenWebUI starts failing during web search / retrieval.

Observed errors include:

  1. NoneType crash in chat processing

Error processing chat payload: 'NoneType' object has no attribute 'strip'

  1. Firecrawl loader error

500 Server Error: Internal Server Error for url: http://192.168.1.172:3002/v1/scrape

  1. Retrieval later crashes with None

Unsupported input type: NoneType. Expected one of: str, dict, PIL.Image.Image, np.ndarray, torch.Tensor

and in the traceback:

query_embeddings = await embedding_function(queries, ...)
queries = [None]

This suggests that somewhere in the pipeline, OpenWebUI ends up passing None instead of a text query.

⸻

Reproduction steps

  1. Start from a working setup with:
    • SearXNG enabled
    • Firecrawl enabled
    • web search functioning normally
  2. Go to Admin Settings → Retrieval / Web Search
  3. Set Fetch URL Content Length Limit to 80000
  4. Save settings
  5. Run a web search in chat
  6. Observe failures
  7. Change the setting again or reset it
  8. Web search remains broken

⸻

Important observation

This looks persistent.
It is not just a temporary runtime issue.

Even after changing the setting again, the instance stays broken until restoring a previous backup.

So this may be:

  • a config persistence bug
  • a bad value serialization/deserialization bug
  • or a code path that stores an invalid state and later reuses it

⸻

Additional notes

I also observed that OpenWebUI stores retrieval/web settings in the SQLite config JSON (webui.db, table config), under nested keys such as:

  • $.rag.web.search.fetch_url_max_content_length
  • $.rag.web.fetch.max_content_length

It may be worth checking whether setting this value to 80000 causes an inconsistent state in one or both locations.

⸻

Related symptoms seen during debugging

Other errors seen while debugging web search in this same setup:

  • NameError: name 'requests' is not defined in SafeFireCrawlLoader.lazy_load
    (separate issue, fixed locally by adding import requests)
  • SearXNG language handling also seemed fragile in some cases

But the specific issue in this report is:

setting Fetch URL Content Length Limit to 80000 appears to corrupt web search behavior in a persistent way.

⸻

Logs

Example errors:

Error processing chat payload: 'NoneType' object has no attribute 'strip'
Error extracting content from URLs: 500 Server Error: Internal Server Error for url: http://192.168.1.172:3002/v1/scrape
Unsupported input type: NoneType. Expected one of: str, dict, PIL.Image.Image, np.ndarray, torch.Tensor

Actual Behavior

After setting Fetch URL Content Length Limit to 80000, web search becomes unstable and starts failing.

Even after reverting the value in the UI and restarting the container, the issue persists.

Observed errors include:

  • 'NoneType' object has no attribute 'strip'
  • Unsupported input type: NoneType
  • occasional 500 errors from the web loader

At this point, web search remains broken until restoring a previous backup, suggesting a persistent corrupted state in the configuration or retrieval pipeline.

Steps to Reproduce

Steps to Reproduce

  1. Start from a working OpenWebUI setup with web search enabled (e.g. SearXNG + Firecrawl)
  2. Go to Admin Settings → Retrieval / Web Search
  3. Set Fetch URL Content Length Limit to 80000
  4. Save the configuration
  5. Perform a web search in chat
  6. Observe failures
  7. Change the value back to a lower number (or default) and restart the container
  8. Perform another web search → the issue persists

Logs & Screenshots

See desc

Additional Information

No response

Originally created by @mero20j1 on GitHub (Apr 22, 2026). Original GitHub issue: https://github.com/open-webui/open-webui/issues/24017 ### Check Existing Issues - [x] I have searched for any existing and/or related issues. - [x] I have searched for any existing and/or related discussions. - [x] I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!). - [x] I am using the latest version of Open WebUI. ### Installation Method Docker ### Open WebUI Version 0.9.1 ### Ollama Version (if applicable) _No response_ ### Operating System Ubuntu ### Browser (if applicable) _No response_ ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have **provided every relevant configuration, setting, and environment variable used in my setup.** - [x] I have clearly **listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup** (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc). - [x] I have documented **step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation**. My steps: - Start with the initial platform/version/OS and dependencies used, - Specify exact install/launch/configure commands, - List URLs visited, user input (incl. example values/emails/passwords if needed), - Describe all options and toggles enabled or changed, - Include any files or environmental changes, - Identify the expected and actual result at each stage, - Ensure any reasonably skilled user can follow and hit the same issue. ### Expected Behavior Summary I found what looks like a persistent configuration bug in web search / retrieval settings. When I set Fetch URL Content Length Limit to 80000 in the OpenWebUI web search settings, web search starts failing. The main problem is that the failure persists even after changing the value back in the UI. At that point, the instance seems stuck in a broken state until I restore from backup. ⸻ Environment * OpenWebUI v0.9.1 * Search engine: SearXNG * Web loader: Firecrawl * Firecrawl is local and working correctly when tested directly with curl * SearXNG is also local and working Example setup: * SearXNG: http://192.168.1.170:8080/search * Firecrawl: http://192.168.1.172:3002 ⸻ What I changed In Admin Settings → Web Search / Retrieval, I set: * Fetch URL Content Length Limit = 80000 After that, web search started failing. Even if I later: * change the value again in the UI * reset the setting * restart Docker / container …the issue remains. I had to restore from backup to get back to a working state. ⸻ Expected behavior Changing Fetch URL Content Length Limit should either: * work correctly, or * fail safely and be reversible by changing the setting back It should not leave the web search pipeline in a permanently broken state. ⸻ Actual behavior After setting Fetch URL Content Length Limit = 80000, OpenWebUI starts failing during web search / retrieval. Observed errors include: 1. NoneType crash in chat processing Error processing chat payload: 'NoneType' object has no attribute 'strip' 2. Firecrawl loader error 500 Server Error: Internal Server Error for url: http://192.168.1.172:3002/v1/scrape 3. Retrieval later crashes with None Unsupported input type: NoneType. Expected one of: str, dict, PIL.Image.Image, np.ndarray, torch.Tensor and in the traceback: query_embeddings = await embedding_function(queries, ...) queries = [None] This suggests that somewhere in the pipeline, OpenWebUI ends up passing None instead of a text query. ⸻ Reproduction steps 1. Start from a working setup with: * SearXNG enabled * Firecrawl enabled * web search functioning normally 2. Go to Admin Settings → Retrieval / Web Search 3. Set Fetch URL Content Length Limit to 80000 4. Save settings 5. Run a web search in chat 6. Observe failures 7. Change the setting again or reset it 8. Web search remains broken ⸻ Important observation This looks persistent. It is not just a temporary runtime issue. Even after changing the setting again, the instance stays broken until restoring a previous backup. So this may be: * a config persistence bug * a bad value serialization/deserialization bug * or a code path that stores an invalid state and later reuses it ⸻ Additional notes I also observed that OpenWebUI stores retrieval/web settings in the SQLite config JSON (webui.db, table config), under nested keys such as: * $.rag.web.search.fetch_url_max_content_length * $.rag.web.fetch.max_content_length It may be worth checking whether setting this value to 80000 causes an inconsistent state in one or both locations. ⸻ Related symptoms seen during debugging Other errors seen while debugging web search in this same setup: * NameError: name 'requests' is not defined in SafeFireCrawlLoader.lazy_load (separate issue, fixed locally by adding import requests) * SearXNG language handling also seemed fragile in some cases But the specific issue in this report is: setting Fetch URL Content Length Limit to 80000 appears to corrupt web search behavior in a persistent way. ⸻ Logs Example errors: Error processing chat payload: 'NoneType' object has no attribute 'strip' Error extracting content from URLs: 500 Server Error: Internal Server Error for url: http://192.168.1.172:3002/v1/scrape Unsupported input type: NoneType. Expected one of: str, dict, PIL.Image.Image, np.ndarray, torch.Tensor ### Actual Behavior After setting Fetch URL Content Length Limit to 80000, web search becomes unstable and starts failing. Even after reverting the value in the UI and restarting the container, the issue persists. Observed errors include: * 'NoneType' object has no attribute 'strip' * Unsupported input type: NoneType * occasional 500 errors from the web loader At this point, web search remains broken until restoring a previous backup, suggesting a persistent corrupted state in the configuration or retrieval pipeline. ### Steps to Reproduce Steps to Reproduce 1. Start from a working OpenWebUI setup with web search enabled (e.g. SearXNG + Firecrawl) 2. Go to Admin Settings → Retrieval / Web Search 3. Set Fetch URL Content Length Limit to 80000 4. Save the configuration 5. Perform a web search in chat 6. Observe failures 7. Change the value back to a lower number (or default) and restart the container 8. Perform another web search → the issue persists ### Logs & Screenshots See desc ### Additional Information _No response_
GiteaMirror added the bug label 2026-04-25 09:52:22 -05:00
Author
Owner

@tjbck commented on GitHub (Apr 24, 2026):

Addressed in dev.

<!-- gh-comment-id:4311933069 --> @tjbck commented on GitHub (Apr 24, 2026): Addressed in dev.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#35682