mirror of
https://github.com/open-webui/open-webui.git
synced 2026-07-11 10:34:13 -05:00
[PR #19184] [CLOSED] perf: Caching BM25 indexes for faster hybrid search #40753
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/open-webui#40753
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/19184
Author: @runyournode
Created: 11/14/2025
Status: ❌ Closed
Base:
dev← Head:dev-bm25📝 Commits (10+)
09c2917Add a cache for the BM25 retriever.29a3055Add pytest verifying BM25 cache persistencef06e5c2Delete backend/open_webui/test/util/test_bm25_cache.pye0168c8Merge pull request #1 from runyournode/codex/enhance-caching-for-retriever-in-openwebui44fb282small typo editef564ebAdd test for BM25 cache rebuild routeb41c8ffDelete tests/backend/retrieval/test_bm25_cache_update.py53da418Merge pull request #2 from runyournode/codex/add-bm25-cache-update-check-route0d8017bDefer BM25 invalidation until collection updates0ca7bb5Update retrieval.py📊 Changes
4 files changed (+660 additions, -6 deletions)
View changed files
📝
backend/open_webui/main.py(+18 -0)➕
backend/open_webui/retrieval/bm25_cache.py(+422 -0)📝
backend/open_webui/retrieval/utils.py(+13 -1)📝
backend/open_webui/routers/retrieval.py(+207 -5)📄 Description
Pull Request Checklist
Note to first-time contributors: Please open a discussion post in Discussions to discuss your idea/fix with the community before creating a pull request, and describe your changes before submitting a pull request.
This is to ensure large feature PRs are discussed with the community first, before starting work on it. If the community does not want this feature or it is not relevant for Open WebUI as a project, it can be identified in the discussion before working on the feature and submitting the PR.
Before submitting, make sure you've checked the following:
devbranch. Not targeting thedevbranch will lead to immediate closure of the PR.Changelog Entry
Description
Problem
Hybrid search uses BM25 (which is good) but is rebuilding the kb indexes every query (which takes a lot of time - in my case it was 30-50 % of the total answer time). Is is even worse with query generation enabled as every generated query will trigger a rebuild.
Strategy
Build a BM25 index for every kb and cache it on disk.
Indexes are identified with a fast computed key, hashed from {
collection_name+metadatas}. text is not used in the key.When needing to perform a lexical search on an index, we first try to load from cache, or build if not in cache.
As text is not used in the key, we have to be careful not to use an outdated cache. This is why we
invalidate_collection_cachewhenever the vector store is updated (save_docs_to_vector_db).I hope/think this will allow to keep the cache up-to-date when we add or edit a document in the kb. Document deletion may not be supported yet.
I also added some route that can perform a (slow) full cache verification (using the text in the vector store) when called manually.
This may not production-ready yet but my early tests have shown that it seems to work and it make the hybrid search a lot faster.
I opened a thread on discord and was recommended to PR.
disclamer: I did use openai Codex to help me code review and manual testing is still in progress. Any feedback, thought or help is welcomed :-).
Added
Changed
Deprecated
Removed
Fixed
Security
Breaking Changes
Additional Information
Screenshots or Videos
Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.