[PR #12239] [MERGED] perf: parallelize hybrid search #22874

New Issue

GiteaMirror · 2026-04-20T04:27:41-05:00

GiteaMirror commented

2026-04-20 04:27:41 -05:00

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/12239
Author: @Phlogi
Created: 3/31/2025
Status: ✅ Merged
Merged: 4/1/2025
Merged by: @tjbck

Base: dev ← Head: dev-threads-on-hybrid

📝 Commits (1)

9c64310 Run hybrid_search in parallel

📊 Changes

1 file changed (+30 additions, -20 deletions)

View changed files

📝 backend/open_webui/retrieval/utils.py (+30 -20)

📄 Description

Before submitting, make sure you've checked the following:

This is broken down part of https://github.com/open-webui/open-webui/pull/11814 which was closed
Target branch: Please verify that the pull request targets the dev branch.
Description: Provide a concise description of the changes made in this pull request.
Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
Documentation: Have you updated relevant documentation Open WebUI Docs, or other documentation sources?
Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
Testing: Have you written and run sufficient tests for validating the changes?
Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
Prefix: To cleary categorize this pull request, prefix the pull request title, using one of the following

Changelog Entry

Description

I noticed that RAG with hybrid search is slow. Currently, the hybrid search runs a nested 2 level for loop in serial to get all potential relevant parts of the document(s).

Changed

Function query_doc_with_hybrid_search updates:

Replace nested for loop with a ThreadPoolExecutor to get all collection data in parallel and only once for all queries.
Execution of process_query over all potential documents in parallel

Additional Information

I measured the speed up based on log entries time stamps.

Performance Benchmark Results

Cores Allowed	Without PR (s)	With PR (s)	Improvement (%)
2	244.96	246.78	-0.7% (insignificant)
8	82.79	64.93 (avg)	21.6%
32	60.36	37.89	37.2%

Note: 8-core "With PR" time is an average of three runs which had a variance of 0.21%. The 0.7% slowdown on 2 cores is mostly within test variance.

Conclusion

Strong speedup on 8+ cores: +21.6% (8 cores), +37.2% (32 cores).
No significant effect on 2 cores (likely test variance).
PR scales well with available cores, optimal for 8+ cores.
The default settings of the ThreadPoolExecutor do not "spam" a low core system.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/12239 **Author:** [@Phlogi](https://github.com/Phlogi) **Created:** 3/31/2025 **Status:** ✅ Merged **Merged:** 4/1/2025 **Merged by:** [@tjbck](https://github.com/tjbck) **Base:** `dev` ← **Head:** `dev-threads-on-hybrid` --- ### 📝 Commits (1) - [`9c64310`](https://github.com/open-webui/open-webui/commit/9c64310db530d92062dacbfdf4d5409162c8ad5d) Run hybrid_search in parallel ### 📊 Changes **1 file changed** (+30 additions, -20 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/retrieval/utils.py` (+30 -20) </details> ### 📄 Description **Before submitting, make sure you've checked the following:** - This is broken down part of https://github.com/open-webui/open-webui/pull/11814 which was closed - [x] **Target branch:** Please verify that the pull request targets the `dev` branch. - [x] **Description:** Provide a concise description of the changes made in this pull request. - [x] **Changelog:** Ensure a changelog entry following the format of [Keep a Changelog](https://keepachangelog.com/) is added at the bottom of the PR description. - [x] **Documentation:** Have you updated relevant documentation [Open WebUI Docs](https://github.com/open-webui/docs), or other documentation sources? - [x] **Dependencies:** Are there any new dependencies? Have you updated the dependency versions in the documentation? - [x] **Testing:** Have you written and run sufficient tests for validating the changes? - [x] **Code review:** Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards? - [x] **Prefix:** To cleary categorize this pull request, prefix the pull request title, using one of the following # Changelog Entry ### Description - I noticed that RAG with hybrid search is slow. Currently, the hybrid search runs a nested 2 level for loop in serial to get all potential relevant parts of the document(s). ### Changed #### Function query_doc_with_hybrid_search updates: - Replace nested for loop with a ThreadPoolExecutor to get all collection data in parallel and only once for all queries. - Execution of process_query over all potential documents in parallel --- ### Additional Information - I measured the speed up based on log entries time stamps. ## Performance Benchmark Results | Cores Allowed | Without PR (s) | With PR (s) | Improvement (%) | |--------------|---------------|------------|----------------| | 2 | 244.96 | 246.78 | -0.7% (insignificant) | | 8 | 82.79 | 64.93 (avg) | **21.6%** | | 32 | 60.36 | 37.89 | **37.2%** | > *Note:* 8-core "With PR" time is an average of three runs which had a variance of 0.21%. The 0.7% slowdown on 2 cores is mostly within test variance. ### Conclusion - **Strong speedup on 8+ cores:** +21.6% (8 cores), +37.2% (32 cores). - **No significant effect on 2 cores** (likely test variance). - **PR scales well with available cores, optimal for 8+ cores.** - The **default settings of the ThreadPoolExecutor** do not "spam" a low core system. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-04-20 04:27:41 -05:00

GiteaMirror closed this issue

2026-04-20 04:27:42 -05:00

GiteaMirror referenced this issue

2026-04-20 06:45:54 -05:00

[PR #22874] [CLOSED] fix: close dropdown menu after cloning a chat #26902

GiteaMirror referenced this issue

2026-04-25 14:24:14 -05:00

[PR #22874] [CLOSED] fix: close dropdown menu after cloning a chat #42532

GiteaMirror referenced this issue

2026-04-30 02:23:55 -05:00

[PR #22874] [CLOSED] fix: close dropdown menu after cloning a chat #49950