mirror of
https://github.com/open-webui/open-webui.git
synced 2026-08-27 06:46:30 -05:00
Classic298andGitHub
466e05801b
RAG vector search runs in a thread pool, but then calls `future.result()` on the event loop thread, so the whole worker freezes until every collection answers. Every other user's token stream stops for that long. It's the default retrieval path. Now `asyncio.gather` over `asyncio.to_thread`, matching what `routers/retrieval.py:2779` already does for the same call. Measured with 3 queries across 4 collections, 60 ms search, and a second request wanting a turn every 5 ms: | | before | after | |---|---|---| | RAG call | 62.0 ms | 61.2 ms | | other request's turns | 0 | 7 | | its worst stall | 62.5 ms | 16.0 ms | Same results, same order, same `(result, error)` contract. Cancellation now lands mid-search instead of after every thread finishes. Threads move from an unbounded per-call pool to the loop's bounded shared one.