[PR #20049] [MERGED] fix: prevent ExternalReranker from blocking event loop during RAG queries #25444

New Issue

GiteaMirror · 2026-04-20T05:56:24-05:00

GiteaMirror commented

2026-04-20 05:56:24 -05:00

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/20049
Author: @Classic298
Created: 12/19/2025
Status: ✅ Merged
Merged: 12/20/2025
Merged by: @tjbck

Base: dev ← Head: rerank

📝 Commits (2)

4bb9ae8 fix: prevent ExternalReranker from blocking event loop during RAG queries (#120)
cc3793f Merge pull request open-webui#19030 from open-webui/dev (#122)

📊 Changes

5 files changed (+27 additions, -1 deletions)

View changed files

📝 backend/open_webui/config.py (+6 -0)
📝 backend/open_webui/main.py (+3 -0)
📝 backend/open_webui/retrieval/models/external.py (+3 -0)
📝 backend/open_webui/retrieval/utils.py (+1 -1)
📝 backend/open_webui/routers/retrieval.py (+14 -0)

📄 Description

Target branch: Verify that the pull request targets the dev branch. Not targeting the dev branch will lead to immediate closure of the PR.
Description: Provide a concise description of the changes made in this pull request down below.
Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
Documentation: If necessary, update relevant documentation Open WebUI Docs like environment variables, the tutorials, or other documentation sources.
Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
Testing: Perform manual tests to verify the implemented fix/feature works as intended AND does not break any other functionality. Take this as an opportunity to make screenshots of the feature/fix and include it in the PR description.
Agentic AI Code: Confirm this Pull Request is not written by any AI Agent or has at least gone through additional human review AND manual testing. If any AI Agent is the co-author of this PR, it may lead to immediate closure of the PR.
Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
Title Prefix: To clearly categorize this pull request, prefix the pull request title using one of the following:
- fix: Bug fix or error correction

Changelog Entry

Description

Fixes #19900

Fixes #19900 - ExternalReranker's synchronous HTTP requests were blocking the Python event loop, causing the entire Open WebUI application to freeze for 9-36+ seconds during reranking operations. This prevented all users from accessing the application while external reranker API calls were in progress.

This fix wraps the blocking reranking call in asyncio.to_thread() to offload it to a thread pool, and adds a configurable timeout to prevent indefinite hangs.

Added RAG_EXTERNAL_RERANKER_TIMEOUT environment variable (default: 120 seconds) for configurable external reranker request timeout
Timeout parameter to ExternalReranker.init() and get_rf() function signature

Additional Information

Issue Reference: Fixes #19900
Pattern Consistency: This fix follows the exact pattern used in commit 902c6cfb ("perf: 50x performance improvement for external embeddings") which used asyncio.to_thread() for sentence transformers embeddings
Call Chain Verification: Confirmed entire call chain from FastAPI endpoint → chat_completion_files_handler() → get_sources_from_items() → query_doc_with_hybrid_search() → compression_retriever.ainvoke() → acompress_documents() is already async

Contributor License Agreement

By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.

Note

Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/20049 **Author:** [@Classic298](https://github.com/Classic298) **Created:** 12/19/2025 **Status:** ✅ Merged **Merged:** 12/20/2025 **Merged by:** [@tjbck](https://github.com/tjbck) **Base:** `dev` ← **Head:** `rerank` --- ### 📝 Commits (2) - [`4bb9ae8`](https://github.com/open-webui/open-webui/commit/4bb9ae853afc9bc24cb667eafc4d5972ff563074) fix: prevent ExternalReranker from blocking event loop during RAG queries (#120) - [`cc3793f`](https://github.com/open-webui/open-webui/commit/cc3793f668ea6625e5714408a41b635fbfbc62a5) Merge pull request open-webui#19030 from open-webui/dev (#122) ### 📊 Changes **5 files changed** (+27 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/config.py` (+6 -0) 📝 `backend/open_webui/main.py` (+3 -0) 📝 `backend/open_webui/retrieval/models/external.py` (+3 -0) 📝 `backend/open_webui/retrieval/utils.py` (+1 -1) 📝 `backend/open_webui/routers/retrieval.py` (+14 -0) </details> ### 📄 Description - [X] **Target branch:** Verify that the pull request targets the `dev` branch. **Not targeting the `dev` branch will lead to immediate closure of the PR.** - [X] **Description:** Provide a concise description of the changes made in this pull request down below. - [X] **Changelog:** Ensure a changelog entry following the format of [Keep a Changelog](https://keepachangelog.com/) is added at the bottom of the PR description. - [X] **Documentation:** If necessary, update relevant documentation [Open WebUI Docs](https://github.com/open-webui/docs) like environment variables, the tutorials, or other documentation sources. - [X] **Dependencies:** Are there any new dependencies? Have you updated the dependency versions in the documentation? - [X] **Testing:** Perform manual tests to **verify the implemented fix/feature works as intended AND does not break any other functionality**. Take this as an opportunity to **make screenshots of the feature/fix and include it in the PR description**. - [X] **Agentic AI Code:** Confirm this Pull Request is **not written by any AI Agent** or has at least **gone through additional human review AND manual testing**. If any AI Agent is the co-author of this PR, it may lead to immediate closure of the PR. - [X] **Code review:** Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards? - [X] **Title Prefix:** To clearly categorize this pull request, prefix the pull request title using one of the following: - **fix**: Bug fix or error correction # Changelog Entry ### Description Fixes #19900 Fixes #19900 - ExternalReranker's synchronous HTTP requests were blocking the Python event loop, causing the entire Open WebUI application to freeze for 9-36+ seconds during reranking operations. This prevented all users from accessing the application while external reranker API calls were in progress. This fix wraps the blocking reranking call in asyncio.to_thread() to offload it to a thread pool, and adds a configurable timeout to prevent indefinite hangs. - Added RAG_EXTERNAL_RERANKER_TIMEOUT environment variable (default: 120 seconds) for configurable external reranker request timeout - Timeout parameter to ExternalReranker.__init__() and get_rf() function signature --- ### Additional Information Issue Reference: Fixes #19900 Pattern Consistency: This fix follows the exact pattern used in commit 902c6cfb ("perf: 50x performance improvement for external embeddings") which used asyncio.to_thread() for sentence transformers embeddings Call Chain Verification: Confirmed entire call chain from FastAPI endpoint → chat_completion_files_handler() → get_sources_from_items() → query_doc_with_hybrid_search() → compression_retriever.ainvoke() → acompress_documents() is already async ### Contributor License Agreement By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms. > [!NOTE] > Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-04-20 05:56:24 -05:00

GiteaMirror closed this issue

2026-04-20 05:56:24 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#25444