I have searched for all existing open AND closed issues and discussions for similar requests. I have found none that is comparable to my request.
Verify Feature Scope
I have read through and understood the scope definition for feature requests in the Issues section. I believe my feature request meets the definition and belongs in the Issues section instead of the Discussions.
Problem Description
Currently when one user uploads multipel files for example selecting multiple ones or Dir Upload / Sync Dir in Knowledge the files all get processed in series.
AFAIK this is how its currently going when uploading multiple files:
One single file gets uploaded by Browser and stored in the file directory (this blocks the process and waits for browser to finish)
That single file THEN gets text extracted (this blocks the process and waits for the extractor like tika to finish)
That single file THEN gets text splitted into chunks (this blocks the process till splitting is finished)
If async embedding is enabled and the batchsize is low enough for multiple batches to be created OWUI will async send multiple embedding jobs (to the loadbalancer or single endpoints) If multiple endpoints are available behind loadbalancer, Async will process faster. If only one GPU is available, upping Batch size is better for less OWUI to embedding endpoint (for example Ollama) overhead. This is the only thing that is async, but this step again blocks the entire process of uploading multiple files.
Vectors of that sigle file THEN are stored in the vector DB, and sql stuff is done to finish the successfull upload of that single file, the single file is now uploaded and available in OWUI for the user.
THEN everything repeats for the next file by requesting the next file form the browser.
This serial approach of handling file upload is blocking and extremely slowing down multiple file uploads.
The result is even with large batch size and or async dependent on the parasllelism of the embeddijng API Endpoint, the GPU(s) will be idle most of the time.
Uploading tons of smaller files takes ages.
Only uploading really big ones is somewhat fast and leverages fast GPUs / async GPU embedding on multiple ones.
Desired Solution you'd like
Make file upload async for (if possible) every step with their own queues, also with ENV variables on how many files can be parallel uploaded per user.
Even if in theory only one Tika, one Textsplitter, one non async Embedding GPU and one connection to vector db is available in total - they could all work at the same time processing.
When for example uploading 5 files and lets assume we have one single GPU and assume (unrealistically) that each step takes the same ammount of time:
Browser is uploading the last file E which is larger, smallest ones probably are finished first (just like you used to when uploading to a solution like google drive) - browser is still uploading file E which is larger.
nothin in qeue for tika / text extractor
tika (or other extractor) is working onm file D which was uploaded before E
nothing in qeue for textspplitter
Textsplitter is making chunks of File C which finished uploading before D
nothing in qeue for embeding
File B is currently being Embedded by a GPU
nothing in qeue for completing everything
File A is finished and Vectors are stored in the vector DB, and sql stuff is done to finish the successfull upload of that single file, the file available in OWUI for the user.GPU Utilization and all CPU utilization will be MUCH higher and the upload especially of many files will be MUCH faster.
Now lets assume multiple bigger files and more realistic qeues as every step takes a different ammount of time with multiple GPUs available:
Browser is uploading currently the last 2 files which are quite big.
6 Files waiting in qeue for tika / text extractor
tika (or other extractor) is working on a file, after its finished it directly takes the next in qeue
nothing in qeue for tika / text splitter
Textsplitter is currently idle
4 files waiting in qeue for embedding
1 fiel which his split in 3 batches and another that is split in 2 batches are parallely being embedded by 5 GPUs (behind loadbalancer) at the same time. As soon as some batches are finished, embedding is taking the next batches from the other files.
one file in qeue for completion steps
Some Files are already finished and Vectors are stored in the vector DB, and sql stuff is done to finish the successfull upload of that files, the files are now available in OWUI for the user.This way GPU utilization stays very high.
Obviosly this will need qeues for every step and probably quite a lot of caching that stuff in the qeues.
Besides a setting on how many parallel uploads are possible, it might be good to have individual settings for the max qeue depth of each step.
Alternatives Considered
No response
Additional Context
No response
Originally created by @TomTheWise on GitHub (Feb 15, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/21433
### Check Existing Issues
- [x] I have searched for all existing **open AND closed** issues and discussions for similar requests. I have found none that is comparable to my request.
### Verify Feature Scope
- [x] I have read through and understood the scope definition for feature requests in the Issues section. I believe my feature request meets the definition and belongs in the Issues section instead of the Discussions.
### Problem Description
Currently when one user uploads multipel files for example selecting multiple ones or Dir Upload / Sync Dir in Knowledge the files all get processed in series.
AFAIK this is how its currently going when uploading multiple files:
1. One single file gets uploaded by Browser and stored in the file directory (this blocks the process and waits for browser to finish)
2. That single file THEN gets text extracted (this blocks the process and waits for the extractor like tika to finish)
3. That single file THEN gets text splitted into chunks (this blocks the process till splitting is finished)
4. If async embedding is enabled and the batchsize is low enough for multiple batches to be created OWUI will async send multiple embedding jobs (to the loadbalancer or single endpoints) If multiple endpoints are available behind loadbalancer, Async will process faster. If only one GPU is available, upping Batch size is better for less OWUI to embedding endpoint (for example Ollama) overhead. This is the only thing that is async, but this step again blocks the entire process of uploading multiple files.
5. Vectors of that sigle file THEN are stored in the vector DB, and sql stuff is done to finish the successfull upload of that single file, the single file is now uploaded and available in OWUI for the user.
6. THEN everything repeats for the next file by requesting the next file form the browser.
This serial approach of handling file upload is blocking and extremely slowing down multiple file uploads.
The result is even with large batch size and or async dependent on the parasllelism of the embeddijng API Endpoint, the GPU(s) will be idle most of the time.
Uploading tons of smaller files takes ages.
Only uploading really big ones is somewhat fast and leverages fast GPUs / async GPU embedding on multiple ones.
### Desired Solution you'd like
Make file upload async for (if possible) every step with their own queues, also with ENV variables on how many files can be parallel uploaded per user.
Even if in theory only one Tika, one Textsplitter, one non async Embedding GPU and one connection to vector db is available in total - they could all work at the same time processing.
When for example uploading 5 files and lets assume we have one single GPU and assume (unrealistically) that each step takes the same ammount of time:
1. Browser is uploading the last file E which is larger, smallest ones probably are finished first (just like you used to when uploading to a solution like google drive) - browser is still uploading file E which is larger.
2. nothin in qeue for tika / text extractor
3. tika (or other extractor) is working onm file D which was uploaded before E
4. nothing in qeue for textspplitter
5. Textsplitter is making chunks of File C which finished uploading before D
6. nothing in qeue for embeding
7. File B is currently being Embedded by a GPU
8. nothing in qeue for completing everything
9. File A is finished and Vectors are stored in the vector DB, and sql stuff is done to finish the successfull upload of that single file, the file available in OWUI for the user.GPU Utilization and all CPU utilization will be MUCH higher and the upload especially of many files will be MUCH faster.
Now lets assume multiple bigger files and more realistic qeues as every step takes a different ammount of time with multiple GPUs available:
1. Browser is uploading currently the last 2 files which are quite big.
2. 6 Files waiting in qeue for tika / text extractor
3. tika (or other extractor) is working on a file, after its finished it directly takes the next in qeue
4. nothing in qeue for tika / text splitter
5. Textsplitter is currently idle
6. 4 files waiting in qeue for embedding
7. 1 fiel which his split in 3 batches and another that is split in 2 batches are parallely being embedded by 5 GPUs (behind loadbalancer) at the same time. As soon as some batches are finished, embedding is taking the next batches from the other files.
8. one file in qeue for completion steps
9. Some Files are already finished and Vectors are stored in the vector DB, and sql stuff is done to finish the successfull upload of that files, the files are now available in OWUI for the user.This way GPU utilization stays very high.
Obviosly this will need qeues for every step and probably quite a lot of caching that stuff in the qeues.
Besides a setting on how many parallel uploads are possible, it might be good to have individual settings for the max qeue depth of each step.
### Alternatives Considered
_No response_
### Additional Context
_No response_
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @TomTheWise on GitHub (Feb 15, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/21433
Check Existing Issues
Verify Feature Scope
Problem Description
Currently when one user uploads multipel files for example selecting multiple ones or Dir Upload / Sync Dir in Knowledge the files all get processed in series.
AFAIK this is how its currently going when uploading multiple files:
This serial approach of handling file upload is blocking and extremely slowing down multiple file uploads.
The result is even with large batch size and or async dependent on the parasllelism of the embeddijng API Endpoint, the GPU(s) will be idle most of the time.
Uploading tons of smaller files takes ages.
Only uploading really big ones is somewhat fast and leverages fast GPUs / async GPU embedding on multiple ones.
Desired Solution you'd like
Make file upload async for (if possible) every step with their own queues, also with ENV variables on how many files can be parallel uploaded per user.
Even if in theory only one Tika, one Textsplitter, one non async Embedding GPU and one connection to vector db is available in total - they could all work at the same time processing.
When for example uploading 5 files and lets assume we have one single GPU and assume (unrealistically) that each step takes the same ammount of time:
Now lets assume multiple bigger files and more realistic qeues as every step takes a different ammount of time with multiple GPUs available:
Obviosly this will need qeues for every step and probably quite a lot of caching that stuff in the qeues.
Besides a setting on how many parallel uploads are possible, it might be good to have individual settings for the max qeue depth of each step.
Alternatives Considered
No response
Additional Context
No response