[GH-ISSUE #16735] issue: Domain Filter List results in no urls when set #72631

New Issue

GiteaMirror · 2026-05-13T04:44:59-05:00

GiteaMirror commented

2026-05-13 04:44:59 -05:00

Originally created by @matthew-kusz on GitHub (Aug 19, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/16735

Check Existing Issues

I have searched the existing issues and discussions.
I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

0.6.22

Ollama Version (if applicable)

No response

Operating System

Ubuntu 22.04

Browser (if applicable)

No response

Confirmation

I have read and followed all instructions in README.md.
I am using the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have provided every relevant configuration, setting, and environment variable used in my setup.
I have clearly listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc).
I have documented step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation. My steps:
Start with the initial platform/version/OS and dependencies used,
Specify exact install/launch/configure commands,
List URLs visited, user input (incl. example values/emails/passwords if needed),
Describe all options and toggles enabled or changed,
Include any files or environmental changes,
Identify the expected and actual result at each stage,
Ensure any reasonably skilled user can follow and hit the same issue.

Expected Behavior

I expect Domain Filter List to filter out the URLs with non-matching domains.

Actual Behavior

Setting Domain Filter List results in no URLs being returned.

Steps to Reproduce

Put any domain in the Domain Filter List field and set the web search engine to duckduckgo. Ask the chatbot a question and it will return 0 urls.

Logs & Screenshots

Resulting output from web search with extra text removed used in the get_filtered_results function:
[{'title': "title", 'href': "url", 'body': "body"}, ... ]

Additional Information

The problem seems to come from this line.. It tries to grab url or link but the results returns a href instead so it ends up becoming an empty string and never validates.

Originally created by @matthew-kusz on GitHub (Aug 19, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/16735 ### Check Existing Issues - [x] I have searched the existing issues and discussions. - [x] I am using the latest version of Open WebUI. ### Installation Method Docker ### Open WebUI Version 0.6.22 ### Ollama Version (if applicable) _No response_ ### Operating System Ubuntu 22.04 ### Browser (if applicable) _No response_ ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [ ] I have included the browser console logs. - [ ] I have included the Docker container logs. - [x] I have **provided every relevant configuration, setting, and environment variable used in my setup.** - [x] I have clearly **listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup** (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc). - [x] I have documented **step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation**. My steps: - Start with the initial platform/version/OS and dependencies used, - Specify exact install/launch/configure commands, - List URLs visited, user input (incl. example values/emails/passwords if needed), - Describe all options and toggles enabled or changed, - Include any files or environmental changes, - Identify the expected and actual result at each stage, - Ensure any reasonably skilled user can follow and hit the same issue. ### Expected Behavior I expect Domain Filter List to filter out the URLs with non-matching domains. ### Actual Behavior Setting Domain Filter List results in no URLs being returned. ### Steps to Reproduce Put any domain in the Domain Filter List field and set the web search engine to duckduckgo. Ask the chatbot a question and it will return 0 urls. ### Logs & Screenshots Resulting output from web search with extra text removed used in the `get_filtered_results` function: [{'title': "title", 'href': "url", 'body': "body"}, ... ] ### Additional Information The problem seems to come from [this line.](https://github.com/open-webui/open-webui/blob/438e5d966f0f64f9ea3feab22724a5bd96a4127b/backend/open_webui/retrieval/web/main.py#L14C28-L14C28). It tries to grab `url` or `link` but the results returns a `href` instead so it ends up becoming an empty string and never validates.

GiteaMirror added the bug label 2026-05-13 04:44:59 -05:00

GiteaMirror closed this issue

2026-05-13 04:45:00 -05:00

GiteaMirror commented

2026-05-13 04:45:03 -05:00

@rgaricano commented on GitHub (Aug 19, 2025):

the filter is done below, in 438e5d966f/backend/open_webui/retrieval/web/main.py (L18)

it filter out all results from domains that doesn't match with the ones setted in domain filter.

@rgaricano commented on GitHub (Aug 19, 2025): the filter is done below, in https://github.com/open-webui/open-webui/blob/438e5d966f0f64f9ea3feab22724a5bd96a4127b/backend/open_webui/retrieval/web/main.py#L18 it filter out all results from domains that doesn't match with the ones setted in domain filter.

GiteaMirror commented

2026-05-13 04:45:03 -05:00

@matthew-kusz commented on GitHub (Aug 19, 2025):

the filter is done below, in

open-webui/backend/open_webui/retrieval/web/main.py

Line 18 in 438e5d9

if any(domain.endswith(filtered_domain) for filtered_domain in filter_list):
it filter out all results from domains that doesn't match with the ones setted in domain filter.

Yes but the lines before that are the issue:

url = result.get("url") or result.get("link", "")
if not validators.url(url):
    continue

Since result doesn't have url or link but href instead for the url, url ends up being an empty string and fails validation so it never gets to the line you mention.

@matthew-kusz commented on GitHub (Aug 19, 2025): > the filter is done below, in > > [open-webui/backend/open_webui/retrieval/web/main.py](https://github.com/open-webui/open-webui/blob/438e5d966f0f64f9ea3feab22724a5bd96a4127b/backend/open_webui/retrieval/web/main.py#L18) > > Line 18 in [438e5d9](/open-webui/open-webui/commit/438e5d966f0f64f9ea3feab22724a5bd96a4127b) > > if any(domain.endswith(filtered_domain) for filtered_domain in filter_list): > it filter out all results from domains that doesn't match with the ones setted in domain filter. Yes but the lines before that are the issue: ``` url = result.get("url") or result.get("link", "") if not validators.url(url): continue ``` Since `result` doesn't have `url` or `link` but `href` instead for the url, `url` ends up being an empty string and fails validation so it never gets to the line you mention.

GiteaMirror commented

2026-05-13 04:45:04 -05:00

@tjbck commented on GitHub (Aug 21, 2025):

02479425a5

@tjbck commented on GitHub (Aug 21, 2025): 02479425a5ab04958540e9f2aa7a62dceead5dbe

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#72631