[GH-ISSUE #8579] feat: Optimize BM25 Retriever Creation in Hybrid Search for RAG #85920

New Issue

GiteaMirror · 2026-05-15T10:27:50-05:00

GiteaMirror commented

2026-05-15 10:27:50 -05:00

Originally created by @BobMiles on GitHub (Jan 15, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/8579

Feature Request

When using RAG with hybrid search enabled, the performance is significantly degraded when using larger knowledgebases, especially with multiple collections.
We suspect the issue lies in the way the BM25 Retriever is created for each collection.
Specifically:

query_collection_with_hybrid_search triggers query_doc_with_hybrid_search for each collection and query (4269df041f/backend/open_webui/retrieval/utils.py (L218))
For each query doc, a BM25 Retriever is created (4269df041f/backend/open_webui/retrieval/utils.py (L93))

From our experience in other RAG applications, the BM25 Retriever generation can be costly.

Describe the solution you'd like

We propose the following optimization to address the performance issue:

Cache the BM25 Retriever

Generate the BM25 Retriever once when a knowledge collection is created, updated, or modified.
Cache the retriever using a hash key derived from the knowledge collection’s state (e.g., metadata, content hash).
Reuse the cached retriever for subsequent queries until the collection changes.

On-Demand Caching

As an alternative, create the BM25 Retriever during the first query execution for a collection and cache it for reuse. Discard the retriever when it becomes invalid due to collection updates.

Describe alternatives you've considered

Disabling hybrid retrieval immediately improves RAG performance. However, the retrieval quality without hybrid search is significantly worse and unsuitable for our use case.

Additional context

Test Setup

Knowledge Base: 1000+ pages across multiple collections.
Environment: High-performance Linux machine running the standard public Docker image of open-webui.
Disabling hybrid search confirms that the bottleneck lies in the hybrid retrieval process.

Originally created by @BobMiles on GitHub (Jan 15, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/8579 # Feature Request ### Is your feature request related to a problem? Please describe. When using RAG with hybrid search enabled, the performance is significantly degraded when using larger knowledgebases, especially with multiple collections. We suspect the issue lies in the way the BM25 Retriever is created for each collection. Specifically: - query_collection_with_hybrid_search triggers query_doc_with_hybrid_search for each collection and query (https://github.com/open-webui/open-webui/blob/4269df041fef62208d59babe0faae866d2bfbc3c/backend/open_webui/retrieval/utils.py#L218) - For each query doc, a BM25 Retriever is created (https://github.com/open-webui/open-webui/blob/4269df041fef62208d59babe0faae866d2bfbc3c/backend/open_webui/retrieval/utils.py#L93) From our experience in other RAG applications, the BM25 Retriever generation can be costly. ### Describe the solution you'd like _We propose the following optimization to address the performance issue:_ #### Cache the BM25 Retriever Generate the BM25 Retriever once when a knowledge collection is created, updated, or modified. Cache the retriever using a hash key derived from the knowledge collection’s state (e.g., metadata, content hash). Reuse the cached retriever for subsequent queries until the collection changes. #### On-Demand Caching As an alternative, create the BM25 Retriever during the first query execution for a collection and cache it for reuse. Discard the retriever when it becomes invalid due to collection updates. ### Describe alternatives you've considered Disabling hybrid retrieval immediately improves RAG performance. However, the retrieval quality without hybrid search is significantly worse and unsuitable for our use case. ### Additional context Test Setup - Knowledge Base: 1000+ pages across multiple collections. - Environment: High-performance Linux machine running the standard public Docker image of open-webui. - Disabling hybrid search confirms that the bottleneck lies in the hybrid retrieval process.

GiteaMirror closed this issue

2026-05-15 10:27:50 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#85920