SearxNG Search Issues in 0.3.3 #1226

New Issue

2025-11-11T14:40:31-06:00

GiteaMirror commented

2025-11-11 14:40:31 -06:00

Originally created by @ProjectMoon on GitHub (Jun 12, 2024).

Bug Report

Description

Bug Summary:
Searching stalls or sometimes fails when using SearxNG. It will report an error about invalid URLs.

Steps to Reproduce:
Enable web search and configure a searxNG URL like https://search.example.com/search?q=<query>.

Expected Behavior:
Search results are shown and fed to the LLM for context.

Actual Behavior:
The search either stalls (prints "searching" in the logs but does nothing) or errors out (reports error about invalid URL).

The invalid URL error actually comes from the search result URLs themselves, not the actual search query. I added a print statement to verify this, and it was failing to validate one of the top 3 result URLs, even though it is a valid URL.

Environment

Open WebUI Version: 0.3.3
Ollama (if applicable): 0.1.43 (though don't think it is applicable)
Operating System: Gentoo Linux
Browser (if applicable): Firefox 126

Reproduction Details

Confirmation:

I have read and followed all the instructions provided in the README.md.
I am on the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.

Logs and Screenshots

Browser Console Logs:
[Include relevant browser console logs, if applicable]

Docker Container Logs:
(This includes my added print statement)

INFO:     127.0.0.1:44018 - "POST /api/v1/memories/query HTTP/1.1" 200 OK
generate_search_query
INFO:     127.0.0.1:44022 - "POST /api/task/query/completions HTTP/1.1" 400 Bad Request
INFO:root:trying to web search with ('searxng', 'What are some things to do in Haarlem?')
urls are ['https://dutchreview.com/traveling/charming-things-to-do-in-haarlem/', 'https://thirdeyetraveller.com/things-to-do-in-haarlem-netherlands-worth-it/', 'https://www.planetware.com/tourist-attractions-/haarlem-nl-nh-har.htm']
urls are https://dutchreview.com/traveling/charming-things-to-do-in-haarlem/
urls are https://thirdeyetraveller.com/things-to-do-in-haarlem-netherlands-worth-it/
ERROR:apps.rag.main:Oops! The URL you provided is invalid. Please double-check and try again.
Traceback (most recent call last):
  File "/app/backend/apps/rag/main.py", line 837, in store_web_search
    loader = get_web_loader(urls)
             ^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/apps/rag/main.py", line 705, in get_web_loader
    if not validate_url(url):
           ^^^^^^^^^^^^^^^^^
  File "/app/backend/apps/rag/main.py", line 730, in validate_url
    return all(validate_url(u) for u in url)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/backend/apps/rag/main.py", line 730, in <genexpr>
    return all(validate_url(u) for u in url)
               ^^^^^^^^^^^^^^^
  File "/app/backend/apps/rag/main.py", line 725, in validate_url
    raise ValueError(ERROR_MESSAGES.INVALID_URL)

Screenshots (if applicable):
[Attach any relevant screenshots to help illustrate the issue]

Installation Method

Docker

Additional Information

I'm not sure what causes the stall vs validation failure.

Originally created by @ProjectMoon on GitHub (Jun 12, 2024). # Bug Report ## Description **Bug Summary:** Searching stalls or sometimes fails when using SearxNG. It will report an error about invalid URLs. **Steps to Reproduce:** Enable web search and configure a searxNG URL like `https://search.example.com/search?q=<query>`. **Expected Behavior:** Search results are shown and fed to the LLM for context. **Actual Behavior:** The search either stalls (prints "searching" in the logs but does nothing) or errors out (reports error about invalid URL). The invalid URL error actually comes from the search result URLs themselves, not the actual search query. I added a print statement to verify this, and it was failing to validate one of the top 3 result URLs, even though it is a valid URL. ## Environment - **Open WebUI Version:** 0.3.3 - **Ollama (if applicable):** 0.1.43 (though don't think it is applicable) - **Operating System:** Gentoo Linux - **Browser (if applicable):** Firefox 126 ## Reproduction Details **Confirmation:** - [X] I have read and followed all the instructions provided in the README.md. - [X] I am on the latest version of both Open WebUI and Ollama. - [X] I have included the browser console logs. - [X] I have included the Docker container logs. ## Logs and Screenshots **Browser Console Logs:** [Include relevant browser console logs, if applicable] **Docker Container Logs:** (This includes my added `print` statement) ``` INFO: 127.0.0.1:44018 - "POST /api/v1/memories/query HTTP/1.1" 200 OK generate_search_query INFO: 127.0.0.1:44022 - "POST /api/task/query/completions HTTP/1.1" 400 Bad Request INFO:root:trying to web search with ('searxng', 'What are some things to do in Haarlem?') urls are ['https://dutchreview.com/traveling/charming-things-to-do-in-haarlem/', 'https://thirdeyetraveller.com/things-to-do-in-haarlem-netherlands-worth-it/', 'https://www.planetware.com/tourist-attractions-/haarlem-nl-nh-har.htm'] urls are https://dutchreview.com/traveling/charming-things-to-do-in-haarlem/ urls are https://thirdeyetraveller.com/things-to-do-in-haarlem-netherlands-worth-it/ ERROR:apps.rag.main:Oops! The URL you provided is invalid. Please double-check and try again. Traceback (most recent call last): File "/app/backend/apps/rag/main.py", line 837, in store_web_search loader = get_web_loader(urls) ^^^^^^^^^^^^^^^^^^^^ File "/app/backend/apps/rag/main.py", line 705, in get_web_loader if not validate_url(url): ^^^^^^^^^^^^^^^^^ File "/app/backend/apps/rag/main.py", line 730, in validate_url return all(validate_url(u) for u in url) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/backend/apps/rag/main.py", line 730, in <genexpr> return all(validate_url(u) for u in url) ^^^^^^^^^^^^^^^ File "/app/backend/apps/rag/main.py", line 725, in validate_url raise ValueError(ERROR_MESSAGES.INVALID_URL) ``` **Screenshots (if applicable):** [Attach any relevant screenshots to help illustrate the issue] ## Installation Method Docker ## Additional Information I'm not sure what causes the stall vs validation failure.

GiteaMirror closed this issue

2025-11-11 14:40:32 -06:00

GiteaMirror commented

2025-11-11 14:40:32 -06:00

@YetheSamartaka commented on GitHub (Jun 12, 2024):

I have exactly the same issue on Ubuntu 24.04 with searxng after updating from 0.3.1 to 0.3.3. Also the logs are pretty much the same for me.

@YetheSamartaka commented on GitHub (Jun 12, 2024): I have exactly the same issue on Ubuntu 24.04 with searxng after updating from 0.3.1 to 0.3.3. Also the logs are pretty much the same for me.

GiteaMirror commented

2025-11-11 14:40:33 -06:00

@Lakr233 commented on GitHub (Jun 12, 2024):

Add - ENABLE_RAG_LOCAL_WEB_FETCH=True fixed the problem.

@Lakr233 commented on GitHub (Jun 12, 2024): Add `- ENABLE_RAG_LOCAL_WEB_FETCH=True` fixed the problem.

GiteaMirror commented

2025-11-11 14:40:33 -06:00

@justinh-rahb commented on GitHub (Jun 12, 2024):

Add - ENABLE_RAG_LOCAL_WEB_FETCH=True fixed the problem.

Confirmed bug on 0.3.3 here, and your fix resolves it as well.

@justinh-rahb commented on GitHub (Jun 12, 2024): > Add `- ENABLE_RAG_LOCAL_WEB_FETCH=True` fixed the problem. Confirmed bug on 0.3.3 here, and your fix resolves it as well.

GiteaMirror commented

2025-11-11 14:40:33 -06:00

@ProjectMoon commented on GitHub (Jun 12, 2024):

Yes, this works. Looks like the commit is being reverted too?

@ProjectMoon commented on GitHub (Jun 12, 2024): Yes, this works. Looks like the commit is being reverted too?

GiteaMirror commented

2025-11-11 14:40:33 -06:00

@justinh-rahb commented on GitHub (Jun 12, 2024):

Yes, this works. Looks like the commit is being reverted too?

Indeed, would highly recommend reverting that envvar as soon as it's merged to main.

@justinh-rahb commented on GitHub (Jun 12, 2024): > Yes, this works. Looks like the commit is being reverted too? Indeed, would highly recommend reverting that envvar as soon as it's merged to main.

GiteaMirror commented

2025-11-11 14:40:34 -06:00