Is your feature request related to a problem? Please describe.
When using RAG with hybrid search enabled, the performance is significantly degraded when using larger knowledgebases, especially with multiple collections.
We suspect the issue lies in the way the BM25 Retriever is created for each collection.
Specifically:
From our experience in other RAG applications, the BM25 Retriever generation can be costly.
Describe the solution you'd like
We propose the following optimization to address the performance issue:
Cache the BM25 Retriever
Generate the BM25 Retriever once when a knowledge collection is created, updated, or modified.
Cache the retriever using a hash key derived from the knowledge collection’s state (e.g., metadata, content hash).
Reuse the cached retriever for subsequent queries until the collection changes.
On-Demand Caching
As an alternative, create the BM25 Retriever during the first query execution for a collection and cache it for reuse. Discard the retriever when it becomes invalid due to collection updates.
Describe alternatives you've considered
Disabling hybrid retrieval immediately improves RAG performance. However, the retrieval quality without hybrid search is significantly worse and unsuitable for our use case.
Additional context
Test Setup
Knowledge Base: 1000+ pages across multiple collections.
Environment: High-performance Linux machine running the standard public Docker image of open-webui.
Disabling hybrid search confirms that the bottleneck lies in the hybrid retrieval process.
Originally created by @BobMiles on GitHub (Jan 15, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/8579
# Feature Request
### Is your feature request related to a problem? Please describe.
When using RAG with hybrid search enabled, the performance is significantly degraded when using larger knowledgebases, especially with multiple collections.
We suspect the issue lies in the way the BM25 Retriever is created for each collection.
Specifically:
- query_collection_with_hybrid_search triggers query_doc_with_hybrid_search for each collection and query (https://github.com/open-webui/open-webui/blob/4269df041fef62208d59babe0faae866d2bfbc3c/backend/open_webui/retrieval/utils.py#L218)
- For each query doc, a BM25 Retriever is created (https://github.com/open-webui/open-webui/blob/4269df041fef62208d59babe0faae866d2bfbc3c/backend/open_webui/retrieval/utils.py#L93)
From our experience in other RAG applications, the BM25 Retriever generation can be costly.
### Describe the solution you'd like
_We propose the following optimization to address the performance issue:_
#### Cache the BM25 Retriever
Generate the BM25 Retriever once when a knowledge collection is created, updated, or modified.
Cache the retriever using a hash key derived from the knowledge collection’s state (e.g., metadata, content hash).
Reuse the cached retriever for subsequent queries until the collection changes.
#### On-Demand Caching
As an alternative, create the BM25 Retriever during the first query execution for a collection and cache it for reuse. Discard the retriever when it becomes invalid due to collection updates.
### Describe alternatives you've considered
Disabling hybrid retrieval immediately improves RAG performance. However, the retrieval quality without hybrid search is significantly worse and unsuitable for our use case.
### Additional context
Test Setup
- Knowledge Base: 1000+ pages across multiple collections.
- Environment: High-performance Linux machine running the standard public Docker image of open-webui.
- Disabling hybrid search confirms that the bottleneck lies in the hybrid retrieval process.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @BobMiles on GitHub (Jan 15, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/8579
Feature Request
Is your feature request related to a problem? Please describe.
When using RAG with hybrid search enabled, the performance is significantly degraded when using larger knowledgebases, especially with multiple collections.
We suspect the issue lies in the way the BM25 Retriever is created for each collection.
Specifically:
From our experience in other RAG applications, the BM25 Retriever generation can be costly.
Describe the solution you'd like
We propose the following optimization to address the performance issue:
Cache the BM25 Retriever
Generate the BM25 Retriever once when a knowledge collection is created, updated, or modified.
Cache the retriever using a hash key derived from the knowledge collection’s state (e.g., metadata, content hash).
Reuse the cached retriever for subsequent queries until the collection changes.
On-Demand Caching
As an alternative, create the BM25 Retriever during the first query execution for a collection and cache it for reuse. Discard the retriever when it becomes invalid due to collection updates.
Describe alternatives you've considered
Disabling hybrid retrieval immediately improves RAG performance. However, the retrieval quality without hybrid search is significantly worse and unsuitable for our use case.
Additional context
Test Setup