Classic298 a3ea7bf043 fix(retrieval): offload Loader.load to a worker thread so file uploads stop blocking the event loop (#23705) ...

Loader.load() dispatches to the underlying langchain document loaders
(PyMuPDF, Unstructured, python-docx, Tika, …) which are all
synchronous and CPU/IO-bound. process_file() awaited it directly on
the event loop, so parsing a non-trivial PDF/DOCX would freeze the
entire FastAPI app for the duration of the parse — which is what users
experience as "the server hangs whenever I upload a file."

Add an `aload()` async wrapper on Loader that runs the sync load on a
worker thread via asyncio.to_thread, and update process_file() to
await it. The sync API is preserved so existing callers that already
run inside run_in_threadpool (e.g. save_docs_to_vector_db) are
unaffected.

https://claude.ai/code/session_01JSr4NZSskEUQvoJnavVXh8

Co-authored-by: Claude <noreply@anthropic.com>

2026-04-14 10:55:46 -05:00

..

datalab_marker.py

refac

2026-03-17 17:58:01 -05:00

external_document.py

refac

2026-03-17 17:58:01 -05:00

external_web.py

refac

2026-03-17 17:58:01 -05:00

main.py

fix(retrieval): offload Loader.load to a worker thread so file uploads stop blocking the event loop (#23705)

2026-04-14 10:55:46 -05:00

mineru.py

refac

2026-03-17 17:58:01 -05:00

mistral.py

refac

2026-03-17 17:58:01 -05:00

tavily.py

refac

2026-03-17 17:58:01 -05:00

youtube.py

refac

2026-03-17 17:58:01 -05:00