Note to first-time contributors: Please open a discussion post in Discussions to discuss your idea/fix with the community before creating a pull request, and describe your changes before submitting a pull request.
This is to ensure large feature PRs are discussed with the community first, before starting work on it. If the community does not want this feature or it is not relevant for Open WebUI as a project, it can be identified in the discussion before working on the feature and submitting the PR.
Before submitting, make sure you've checked the following:
Target branch: Verify that the pull request targets the dev branch. Not targeting the dev branch will lead to immediate closure of the PR.
Description: Provide a concise description of the changes made in this pull request down below.
Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
Documentation: If necessary, update relevant documentation Open WebUI Docs like environment variables, the tutorials, or other documentation sources.
Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
Testing: Perform manual tests to verify the implemented fix/feature works as intended AND does not break any other functionality. Take this as an opportunity to make screenshots of the feature/fix and include it in the PR description.
Agentic AI Code: Confirm this Pull Request is not written by any AI Agent or has at least gone through additional human review AND manual testing. If any AI Agent is the co-author of this PR, it may lead to immediate closure of the PR.
Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
Title Prefix: To clearly categorize this pull request, prefix the pull request title using one of the following:
BREAKING CHANGE: Significant changes that may affect compatibility
build: Changes that affect the build system or external dependencies
ci: Changes to our continuous integration processes or workflows
chore: Refactor, cleanup, or other non-functional code changes
docs: Documentation update or addition
feat: Introduces a new feature or enhancement to the codebase
fix: Bug fix or error correction
i18n: Internationalization or localization changes
perf: Performance improvement
refactor: Code restructuring for better maintainability, readability, or scalability
style: Changes that do not affect the meaning of the code (white space, formatting, missing semi-colons, etc.)
test: Adding missing tests or correcting existing tests
WIP: Work in progress, a temporary label for incomplete or ongoing work
Changelog Entry
Description
Problem
Hybrid search uses BM25 (which is good) but is rebuilding the kb indexes every query (which takes a lot of time - in my case it was 30-50 % of the total answer time). Is is even worse with query generation enabled as every generated query will trigger a rebuild.
Strategy
Build a BM25 index for every kb and cache it on disk.
Indexes are identified with a fast computed key, hashed from {collection_name + metadatas}. text is not used in the key.
When needing to perform a lexical search on an index, we first try to load from cache, or build if not in cache.
As text is not used in the key, we have to be careful not to use an outdated cache. This is why we invalidate_collection_cache whenever the vector store is updated (save_docs_to_vector_db).
I hope/think this will allow to keep the cache up-to-date when we add or edit a document in the kb. Document deletion may not be supported yet.
I also added some route that can perform a (slow) full cache verification (using the text in the vector store) when called manually.
This may not production-ready yet but my early tests have shown that it seems to work and it make the hybrid search a lot faster.
I opened a thread on discord and was recommended to PR.
disclamer: I did use openai Codex to help me code review and manual testing is still in progress. Any feedback, thought or help is welcomed :-).
Added
None
Changed
Caching strategy for BM25 indexes
Deprecated
None
Removed
None
Fixed
Trying to address the slowness of hybrid search
Security
None
Breaking Changes
BREAKING CHANGE: (Hopefully) None
Additional Information
None
Screenshots or Videos
None
Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
Note
Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/19184
**Author:** [@runyournode](https://github.com/runyournode)
**Created:** 11/14/2025
**Status:** ❌ Closed
**Base:** `dev` ← **Head:** `dev-bm25`
---
### 📝 Commits (10+)
- [`09c2917`](https://github.com/open-webui/open-webui/commit/09c2917884e25dce42407bcb735e8ceb1866a116) Add a cache for the BM25 retriever.
- [`29a3055`](https://github.com/open-webui/open-webui/commit/29a30555a13528015dfda7b366038005a79d918f) Add pytest verifying BM25 cache persistence
- [`f06e5c2`](https://github.com/open-webui/open-webui/commit/f06e5c236da0b9c56f6c51805992c33b053334c3) Delete backend/open_webui/test/util/test_bm25_cache.py
- [`e0168c8`](https://github.com/open-webui/open-webui/commit/e0168c8f5c87091f4dc61ad00bca992793cfb0de) Merge pull request #1 from runyournode/codex/enhance-caching-for-retriever-in-openwebui
- [`44fb282`](https://github.com/open-webui/open-webui/commit/44fb2820e4f53d1859f86a2ac9b66370df204dec) small typo edit
- [`ef564eb`](https://github.com/open-webui/open-webui/commit/ef564ebab5eb01e711f16ab7cca5a76fb19623d2) Add test for BM25 cache rebuild route
- [`b41c8ff`](https://github.com/open-webui/open-webui/commit/b41c8ff3db3103f1f6c6e9235e91a0fdfe5293e6) Delete tests/backend/retrieval/test_bm25_cache_update.py
- [`53da418`](https://github.com/open-webui/open-webui/commit/53da4188565f2e249cc7472ce21ee926d2d46a05) Merge pull request #2 from runyournode/codex/add-bm25-cache-update-check-route
- [`0d8017b`](https://github.com/open-webui/open-webui/commit/0d8017bf011655f32be5d38bfddb9a6791553e94) Defer BM25 invalidation until collection updates
- [`0ca7bb5`](https://github.com/open-webui/open-webui/commit/0ca7bb5acc25d347ed0590770bed1132cf6b1776) Update retrieval.py
### 📊 Changes
**4 files changed** (+660 additions, -6 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/open_webui/main.py` (+18 -0)
➕ `backend/open_webui/retrieval/bm25_cache.py` (+422 -0)
📝 `backend/open_webui/retrieval/utils.py` (+13 -1)
📝 `backend/open_webui/routers/retrieval.py` (+207 -5)
</details>
### 📄 Description
# Pull Request Checklist
### Note to first-time contributors: Please open a discussion post in [Discussions](https://github.com/open-webui/open-webui/discussions) to discuss your idea/fix with the community before creating a pull request, and describe your changes before submitting a pull request.
This is to ensure large feature PRs are discussed with the community first, before starting work on it. If the community does not want this feature or it is not relevant for Open WebUI as a project, it can be identified in the discussion before working on the feature and submitting the PR.
**Before submitting, make sure you've checked the following:**
- [x] **Target branch:** Verify that the pull request targets the `dev` branch. **Not targeting the `dev` branch will lead to immediate closure of the PR.**
- [x] **Description:** Provide a concise description of the changes made in this pull request down below.
- [x] **Changelog:** Ensure a changelog entry following the format of [Keep a Changelog](https://keepachangelog.com/) is added at the bottom of the PR description.
- [ ] **Documentation:** If necessary, update relevant documentation [Open WebUI Docs](https://github.com/open-webui/docs) like environment variables, the tutorials, or other documentation sources.
- [x] **Dependencies:** Are there any new dependencies? Have you updated the dependency versions in the documentation?
- [ ] **Testing:** Perform manual tests to **verify the implemented fix/feature works as intended AND does not break any other functionality**. Take this as an opportunity to **make screenshots of the feature/fix and include it in the PR description**.
- [ ] **Agentic AI Code:** Confirm this Pull Request is **not written by any AI Agent** or has at least **gone through additional human review AND manual testing**. If any AI Agent is the co-author of this PR, it may lead to immediate closure of the PR.
- [ ] **Code review:** Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
- [ ] **Title Prefix:** To clearly categorize this pull request, prefix the pull request title using one of the following:
- **BREAKING CHANGE**: Significant changes that may affect compatibility
- **build**: Changes that affect the build system or external dependencies
- **ci**: Changes to our continuous integration processes or workflows
- **chore**: Refactor, cleanup, or other non-functional code changes
- **docs**: Documentation update or addition
- **feat**: Introduces a new feature or enhancement to the codebase
- **fix**: Bug fix or error correction
- **i18n**: Internationalization or localization changes
- **perf**: Performance improvement
- **refactor**: Code restructuring for better maintainability, readability, or scalability
- **style**: Changes that do not affect the meaning of the code (white space, formatting, missing semi-colons, etc.)
- **test**: Adding missing tests or correcting existing tests
- **WIP**: Work in progress, a temporary label for incomplete or ongoing work
# Changelog Entry
### Description
**Problem**
Hybrid search uses BM25 (which is good) but is rebuilding the kb indexes every query (which takes a lot of time - in my case it was 30-50 % of the total answer time). Is is even worse with query generation enabled as every generated query will trigger a rebuild.
**Strategy**
Build a BM25 index for every kb and cache it on disk.
Indexes are identified with a fast computed key, hashed from {`collection_name` + `metadatas`}. text is not used in the key.
When needing to perform a lexical search on an index, we first try to load from cache, or build if not in cache.
As text is not used in the key, we have to be careful not to use an outdated cache. This is why we `invalidate_collection_cache` whenever the vector store is updated (`save_docs_to_vector_db`).
I hope/think this will allow to keep the cache up-to-date when we add or edit a document in the kb. **Document deletion may not be supported yet.**
I also added some route that can perform a (slow) full cache verification (using the text in the vector store) when called manually.
This may not production-ready yet but my early tests have shown that it seems to work and it make the hybrid search a lot faster.
I opened a [thread on discord](https://discord.com/channels/1170866489302188073/1438845846589542422/1438845846589542422) and was recommended to PR.
*disclamer*: I did use openai Codex to help me code review and manual testing is still in progress. Any feedback, thought or help is welcomed :-).
### Added
- None
### Changed
- Caching strategy for BM25 indexes
### Deprecated
- None
### Removed
- None
### Fixed
- Trying to address the slowness of hybrid search
### Security
- None
### Breaking Changes
- **BREAKING CHANGE**: (Hopefully) None
---
### Additional Information
- None
### Screenshots or Videos
- None
### Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms.
> [!NOTE]
> Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/19184
Author: @runyournode
Created: 11/14/2025
Status: ❌ Closed
Base:
dev← Head:dev-bm25📝 Commits (10+)
09c2917Add a cache for the BM25 retriever.29a3055Add pytest verifying BM25 cache persistencef06e5c2Delete backend/open_webui/test/util/test_bm25_cache.pye0168c8Merge pull request #1 from runyournode/codex/enhance-caching-for-retriever-in-openwebui44fb282small typo editef564ebAdd test for BM25 cache rebuild routeb41c8ffDelete tests/backend/retrieval/test_bm25_cache_update.py53da418Merge pull request #2 from runyournode/codex/add-bm25-cache-update-check-route0d8017bDefer BM25 invalidation until collection updates0ca7bb5Update retrieval.py📊 Changes
4 files changed (+660 additions, -6 deletions)
View changed files
📝
backend/open_webui/main.py(+18 -0)➕
backend/open_webui/retrieval/bm25_cache.py(+422 -0)📝
backend/open_webui/retrieval/utils.py(+13 -1)📝
backend/open_webui/routers/retrieval.py(+207 -5)📄 Description
Pull Request Checklist
Note to first-time contributors: Please open a discussion post in Discussions to discuss your idea/fix with the community before creating a pull request, and describe your changes before submitting a pull request.
This is to ensure large feature PRs are discussed with the community first, before starting work on it. If the community does not want this feature or it is not relevant for Open WebUI as a project, it can be identified in the discussion before working on the feature and submitting the PR.
Before submitting, make sure you've checked the following:
devbranch. Not targeting thedevbranch will lead to immediate closure of the PR.Changelog Entry
Description
Problem
Hybrid search uses BM25 (which is good) but is rebuilding the kb indexes every query (which takes a lot of time - in my case it was 30-50 % of the total answer time). Is is even worse with query generation enabled as every generated query will trigger a rebuild.
Strategy
Build a BM25 index for every kb and cache it on disk.
Indexes are identified with a fast computed key, hashed from {
collection_name+metadatas}. text is not used in the key.When needing to perform a lexical search on an index, we first try to load from cache, or build if not in cache.
As text is not used in the key, we have to be careful not to use an outdated cache. This is why we
invalidate_collection_cachewhenever the vector store is updated (save_docs_to_vector_db).I hope/think this will allow to keep the cache up-to-date when we add or edit a document in the kb. Document deletion may not be supported yet.
I also added some route that can perform a (slow) full cache verification (using the text in the vector store) when called manually.
This may not production-ready yet but my early tests have shown that it seems to work and it make the hybrid search a lot faster.
I opened a thread on discord and was recommended to PR.
disclamer: I did use openai Codex to help me code review and manual testing is still in progress. Any feedback, thought or help is welcomed :-).
Added
Changed
Deprecated
Removed
Fixed
Security
Breaking Changes
Additional Information
Screenshots or Videos
Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.