Target branch: Please verify that the pull request targets the dev branch.
Description: Provide a concise description of the changes made in this pull request.
Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
Documentation: Have you updated relevant documentation Open WebUI Docs, or other documentation sources?
Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
Testing: Have you written and run sufficient tests for validating the changes?
Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
Prefix: To cleary categorize this pull request, prefix the pull request title, using one of the following
Changelog Entry
Description
I noticed that RAG with hybrid search is slow. Currently, the hybrid search runs a nested 2 level for loop in serial to get all potential relevant parts of the document(s).
Changed
Function query_doc_with_hybrid_search updates:
Replace nested for loop with a ThreadPoolExecutor to get all collection data in parallel and only once for all queries.
Execution of process_query over all potential documents in parallel
Additional Information
I measured the speed up based on log entries time stamps.
Performance Benchmark Results
Cores Allowed
Without PR (s)
With PR (s)
Improvement (%)
2
244.96
246.78
-0.7% (insignificant)
8
82.79
64.93 (avg)
21.6%
32
60.36
37.89
37.2%
Note: 8-core "With PR" time is an average of three runs which had a variance of 0.21%. The 0.7% slowdown on 2 cores is mostly within test variance.
No significant effect on 2 cores (likely test variance).
PR scales well with available cores, optimal for 8+ cores.
The default settings of the ThreadPoolExecutor do not "spam" a low core system.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/12239
**Author:** [@Phlogi](https://github.com/Phlogi)
**Created:** 3/31/2025
**Status:** ✅ Merged
**Merged:** 4/1/2025
**Merged by:** [@tjbck](https://github.com/tjbck)
**Base:** `dev` ← **Head:** `dev-threads-on-hybrid`
---
### 📝 Commits (1)
- [`9c64310`](https://github.com/open-webui/open-webui/commit/9c64310db530d92062dacbfdf4d5409162c8ad5d) Run hybrid_search in parallel
### 📊 Changes
**1 file changed** (+30 additions, -20 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/open_webui/retrieval/utils.py` (+30 -20)
</details>
### 📄 Description
**Before submitting, make sure you've checked the following:**
- This is broken down part of https://github.com/open-webui/open-webui/pull/11814 which was closed
- [x] **Target branch:** Please verify that the pull request targets the `dev` branch.
- [x] **Description:** Provide a concise description of the changes made in this pull request.
- [x] **Changelog:** Ensure a changelog entry following the format of [Keep a Changelog](https://keepachangelog.com/) is added at the bottom of the PR description.
- [x] **Documentation:** Have you updated relevant documentation [Open WebUI Docs](https://github.com/open-webui/docs), or other documentation sources?
- [x] **Dependencies:** Are there any new dependencies? Have you updated the dependency versions in the documentation?
- [x] **Testing:** Have you written and run sufficient tests for validating the changes?
- [x] **Code review:** Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
- [x] **Prefix:** To cleary categorize this pull request, prefix the pull request title, using one of the following
# Changelog Entry
### Description
- I noticed that RAG with hybrid search is slow. Currently, the hybrid search runs a nested 2 level for loop in serial to get all potential relevant parts of the document(s).
### Changed
#### Function query_doc_with_hybrid_search updates:
- Replace nested for loop with a ThreadPoolExecutor to get all collection data in parallel and only once for all queries.
- Execution of process_query over all potential documents in parallel
---
### Additional Information
- I measured the speed up based on log entries time stamps.
## Performance Benchmark Results
| Cores Allowed | Without PR (s) | With PR (s) | Improvement (%) |
|--------------|---------------|------------|----------------|
| 2 | 244.96 | 246.78 | -0.7% (insignificant) |
| 8 | 82.79 | 64.93 (avg) | **21.6%** |
| 32 | 60.36 | 37.89 | **37.2%** |
> *Note:* 8-core "With PR" time is an average of three runs which had a variance of 0.21%. The 0.7% slowdown on 2 cores is mostly within test variance.
### Conclusion
- **Strong speedup on 8+ cores:** +21.6% (8 cores), +37.2% (32 cores).
- **No significant effect on 2 cores** (likely test variance).
- **PR scales well with available cores, optimal for 8+ cores.**
- The **default settings of the ThreadPoolExecutor** do not "spam" a low core system.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/12239
Author: @Phlogi
Created: 3/31/2025
Status: ✅ Merged
Merged: 4/1/2025
Merged by: @tjbck
Base:
dev← Head:dev-threads-on-hybrid📝 Commits (1)
9c64310Run hybrid_search in parallel📊 Changes
1 file changed (+30 additions, -20 deletions)
View changed files
📝
backend/open_webui/retrieval/utils.py(+30 -20)📄 Description
Before submitting, make sure you've checked the following:
This is broken down part of https://github.com/open-webui/open-webui/pull/11814 which was closed
Target branch: Please verify that the pull request targets the
devbranch.Description: Provide a concise description of the changes made in this pull request.
Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
Documentation: Have you updated relevant documentation Open WebUI Docs, or other documentation sources?
Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
Testing: Have you written and run sufficient tests for validating the changes?
Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
Prefix: To cleary categorize this pull request, prefix the pull request title, using one of the following
Changelog Entry
Description
Changed
Function query_doc_with_hybrid_search updates:
Additional Information
Performance Benchmark Results
Conclusion
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.