[PR #23706] [MERGED] fix(retrieval): offload sync VECTOR_DB_CLIENT calls in async paths via AsyncVectorDBClient #82198

New Issue

GiteaMirror · 2026-05-13T16:40:28-05:00

GiteaMirror commented

2026-05-13 16:40:28 -05:00

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/23706
Author: @Classic298
Created: 4/14/2026
Status: ✅ Merged
Merged: 4/14/2026
Merged by: @tjbck

Base: dev ← Head: claude/fix/async-vector-db-client

📝 Commits (4)

7d865db fix(retrieval): offload sync VECTOR_DB_CLIENT calls in async paths via AsyncVectorDBClient
b563d4a fix(retrieval): restore explicit AsyncVectorDBClient signatures matching VectorDBBase
3ebc2b0 fix(retrieval): parallelize hybrid-search collection prefetch; document async facade contracts
4a9bd01 fix(retrieval): guard /delete against null file.hash and let HTTPException reach the client

📊 Changes

7 files changed (+232 additions, -50 deletions)

View changed files

📝 backend/open_webui/retrieval/utils.py (+25 -10)
➕ backend/open_webui/retrieval/vector/async_client.py (+135 -0)
📝 backend/open_webui/routers/files.py (+6 -6)
📝 backend/open_webui/routers/knowledge.py (+14 -14)
📝 backend/open_webui/routers/memories.py (+8 -8)
📝 backend/open_webui/routers/retrieval.py (+40 -8)
📝 backend/open_webui/tools/builtin.py (+4 -4)

📄 Description

The vector DB backends (Chroma, pgvector, Qdrant, Milvus, Pinecone, Weaviate, …) are uniformly synchronous and their methods perform blocking network or disk I/O. Multiple async route handlers and helpers were calling them directly on the event loop — file processing, memories, knowledge bases, hybrid search bookkeeping — so a single upsert/delete/search would freeze every other in-flight request for the duration of the call.

Introduce AsyncVectorDBClient, a thin async facade that wraps the existing sync client and dispatches each method through asyncio.to_thread. It mirrors VectorDBBase exactly and forwards *args/**kwargs so backend-specific extra parameters keep working.

Update every async-context call site (routers/retrieval, routers/files, routers/memories, routers/knowledge, retrieval/utils, tools/builtin) to await ASYNC_VECTOR_DB_CLIENT instead of calling the sync client directly. Two helpers that were sync-only also acquire async siblings or are awaited via asyncio.to_thread at their async call site (remove_knowledge_base_metadata_embedding, get_all_items_from_collections, query_doc).

The original sync VECTOR_DB_CLIENT is unchanged, so callers that already run inside run_in_threadpool (e.g. save_docs_to_vector_db and the sync query_doc/get_doc helpers) are unaffected.

Contributor License Agreement

By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.

Note

Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/23706 **Author:** [@Classic298](https://github.com/Classic298) **Created:** 4/14/2026 **Status:** ✅ Merged **Merged:** 4/14/2026 **Merged by:** [@tjbck](https://github.com/tjbck) **Base:** `dev` ← **Head:** `claude/fix/async-vector-db-client` --- ### 📝 Commits (4) - [`7d865db`](https://github.com/open-webui/open-webui/commit/7d865db613fbe61ac65652eb06590ae5af5e640b) fix(retrieval): offload sync VECTOR_DB_CLIENT calls in async paths via AsyncVectorDBClient - [`b563d4a`](https://github.com/open-webui/open-webui/commit/b563d4a3cfbe78d4e4a5660e107bea7b62bf9993) fix(retrieval): restore explicit AsyncVectorDBClient signatures matching VectorDBBase - [`3ebc2b0`](https://github.com/open-webui/open-webui/commit/3ebc2b0c2f03d746bca251876437e339fda4ab5b) fix(retrieval): parallelize hybrid-search collection prefetch; document async facade contracts - [`4a9bd01`](https://github.com/open-webui/open-webui/commit/4a9bd0172e60ee7b4ca842aaf9d0d091bf2bd721) fix(retrieval): guard /delete against null file.hash and let HTTPException reach the client ### 📊 Changes **7 files changed** (+232 additions, -50 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/retrieval/utils.py` (+25 -10) ➕ `backend/open_webui/retrieval/vector/async_client.py` (+135 -0) 📝 `backend/open_webui/routers/files.py` (+6 -6) 📝 `backend/open_webui/routers/knowledge.py` (+14 -14) 📝 `backend/open_webui/routers/memories.py` (+8 -8) 📝 `backend/open_webui/routers/retrieval.py` (+40 -8) 📝 `backend/open_webui/tools/builtin.py` (+4 -4) </details> ### 📄 Description The vector DB backends (Chroma, pgvector, Qdrant, Milvus, Pinecone, Weaviate, …) are uniformly synchronous and their methods perform blocking network or disk I/O. Multiple async route handlers and helpers were calling them directly on the event loop — file processing, memories, knowledge bases, hybrid search bookkeeping — so a single upsert/delete/search would freeze every other in-flight request for the duration of the call. Introduce `AsyncVectorDBClient`, a thin async facade that wraps the existing sync client and dispatches each method through `asyncio.to_thread`. It mirrors `VectorDBBase` exactly and forwards *args/**kwargs so backend-specific extra parameters keep working. Update every async-context call site (routers/retrieval, routers/files, routers/memories, routers/knowledge, retrieval/utils, tools/builtin) to await `ASYNC_VECTOR_DB_CLIENT` instead of calling the sync client directly. Two helpers that were sync-only also acquire async siblings or are awaited via `asyncio.to_thread` at their async call site (`remove_knowledge_base_metadata_embedding`, `get_all_items_from_collections`, `query_doc`). The original sync `VECTOR_DB_CLIENT` is unchanged, so callers that already run inside `run_in_threadpool` (e.g. `save_docs_to_vector_db` and the sync `query_doc`/`get_doc` helpers) are unaffected. ### Contributor License Agreement  - [X] By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms. > [!NOTE] > Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-05-13 16:40:28 -05:00

GiteaMirror closed this issue

2026-05-13 16:40:30 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#82198