[GH-ISSUE #12772] feat: Support for Self-Hosted/External Web Search/Loader Engines #16711

New Issue

GiteaMirror · 2026-04-19T22:34:57-05:00

GiteaMirror commented

2026-04-19 22:34:57 -05:00

Originally created by @tth37 on GitHub (Apr 12, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/12772

Check Existing Issues

I have searched the existing issues and discussions.

Problem Description

Currently, open-webui provides built-in web search engines (Bing, DuckDuckGo) and web page loader engines (Tavily, Local Web Fetcher). While useful, this predefined list has limitations:

Lack of Flexibility: Users cannot easily integrate custom or less common search engines/APIs, nor can they customize the behavior (e.g., specific filtering logic, result processing) of the web search/loading process beyond the available settings.
Deployment Constraints: Users may need to run the web search or page loading logic on a seperate machine from the main open-webui backend. This could be due to network restrictions where the open-webui server has limited internet access, and standard http_proxy settings might not work effectively for the underlying search/fetch libraries.

This is similar to the flexibility provided by OpenAPI Tool Servers, which allow users to define external functionalities. Applying a similar concept to web search/loading would be beneficial.

Desired Solution you'd like

I propose adding a new type of engine configuration, perhaps "Self-Hosted" or "External Connection", for both Web Search and Web Loaders.

This would allow advanced users to:

Specify a base URL endpoint in the open-webui settings for their custom engine.
Implement a service at that URL which conforms to a defined API schema expected by open-webui

Based on the apparent structure in the current codebase, the API contract for these external endpoints could potentially look like this (subject to design adjustments):

External Search Engine Endpoint
- POST /search
- Body: { "query": str, "count": int, "filter_list": Optional[list[str]] }
- Returns: list[SearchResult]
External Web Loader Endpoint
- POST /load
- Body: { "url": str, ... }
- Returns: Document

When a user selects their configured "Self-Hosted Engine", Open WebUI would make API calls to the user-provided URL instead of using the built-in logic.

Alternatives Considered

No response

Additional Context

If this feature request is approved, I would be happy to contribute to its implementation 💪. Thanks a lot!

Originally created by @tth37 on GitHub (Apr 12, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/12772 ### Check Existing Issues - [x] I have searched the existing issues and discussions. ### Problem Description Currently, open-webui provides built-in web search engines (Bing, DuckDuckGo) and web page loader engines (Tavily, Local Web Fetcher). While useful, this predefined list has limitations: 1. **Lack of Flexibility**: Users cannot easily integrate custom or less common search engines/APIs, nor can they customize the behavior (e.g., specific filtering logic, result processing) of the web search/loading process beyond the available settings. 2. **Deployment Constraints**: Users may need to run the web search or page loading logic on a seperate machine from the main open-webui backend. This could be due to network restrictions where the open-webui server has limited internet access, and standard `http_proxy` settings might not work effectively for the underlying search/fetch libraries. This is similar to the flexibility provided by OpenAPI Tool Servers, which allow users to define external functionalities. Applying a similar concept to web search/loading would be beneficial. ### Desired Solution you'd like I propose adding a new type of engine configuration, perhaps "Self-Hosted" or "External Connection", for both Web Search and Web Loaders. This would allow advanced users to: 1. Specify a base URL endpoint in the open-webui settings for their custom engine. 2. Implement a service at that URL which conforms to a defined API schema expected by open-webui Based on the apparent structure in the current codebase, the API contract for these external endpoints could potentially look like this (subject to design adjustments): - **External Search Engine Endpoint** - POST /search - Body: `{ "query": str, "count": int, "filter_list": Optional[list[str]] }` - Returns: `list[SearchResult]` - **External Web Loader Endpoint** - POST /load - Body: `{ "url": str, ... }` - Returns: `Document` When a user selects their configured "Self-Hosted Engine", Open WebUI would make API calls to the user-provided URL instead of using the built-in logic. ### Alternatives Considered _No response_ ### Additional Context **If this feature request is approved, I would be happy to contribute to its implementation 💪.** Thanks a lot!

GiteaMirror closed this issue

2026-04-19 22:34:57 -05:00

GiteaMirror commented

2026-04-19 22:34:58 -05:00

@lucky0218 commented on GitHub (Apr 12, 2025):

Really helpful to those have "limited" global internet access.

@lucky0218 commented on GitHub (Apr 12, 2025): Really helpful to those have "limited" global internet access.

GiteaMirror commented

2026-04-19 22:34:58 -05:00

@tjbck commented on GitHub (Apr 12, 2025):

PR Welcome!

@tjbck commented on GitHub (Apr 12, 2025): PR Welcome!