[GH-ISSUE #15023] refac: make file upload process async (w/ polling) #17442

New Issue

GiteaMirror · 2026-04-19T23:12:10-05:00

GiteaMirror commented

2026-04-19 23:12:10 -05:00

Originally created by @tjbck on GitHub (Jun 16, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/15023

Originally assigned to: @tjbck on GitHub.

Originally created by @tjbck on GitHub (Jun 16, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/15023 Originally assigned to: @tjbck on GitHub.

GiteaMirror closed this issue

2026-04-19 23:12:11 -05:00

GiteaMirror commented

2026-04-19 23:12:12 -05:00

@kns-12 commented on GitHub (Jul 16, 2025):

@tjbck is there currently any planning on when this feature is going to be added?
And maybe a short term solution via environment variable which increases the Gateway Timeout?

@kns-12 commented on GitHub (Jul 16, 2025): @tjbck is there currently any planning on when this feature is going to be added? And maybe a short term solution via environment variable which increases the Gateway Timeout?

GiteaMirror commented

2026-04-19 23:12:13 -05:00

@rgaricano commented on GitHub (Jul 16, 2025):

@tjbck is there currently any planning on when this feature is going to be added? And maybe a short term solution via environment variable which increases the Gateway Timeout?

https://docs.openwebui.com/getting-started/env-configuration#aiohttp-client

& setting in reverse proxy if you use it.

@rgaricano commented on GitHub (Jul 16, 2025): > [@tjbck](https://github.com/tjbck) is there currently any planning on when this feature is going to be added? And maybe a short term solution via environment variable which increases the Gateway Timeout? https://docs.openwebui.com/getting-started/env-configuration#aiohttp-client & setting in reverse proxy if you use it.

GiteaMirror commented

2026-04-19 23:12:13 -05:00

@kns-12 commented on GitHub (Jul 16, 2025):

@tjbck is there currently any planning on when this feature is going to be added? And maybe a short term solution via environment variable which increases the Gateway Timeout?

https://docs.openwebui.com/getting-started/env-configuration#aiohttp-client

& setting in reverse proxy if you use it.

@rgaricano is that related to the 30 seconds Gateway Timeout for document Uploading/Embedding?
According to the docs, the default is already set to 300 seconds.

How would a reverse proxy tackle the internal timeout of Open WebUI?

@kns-12 commented on GitHub (Jul 16, 2025): > > [@tjbck](https://github.com/tjbck) is there currently any planning on when this feature is going to be added? And maybe a short term solution via environment variable which increases the Gateway Timeout? > > https://docs.openwebui.com/getting-started/env-configuration#aiohttp-client > > & setting in reverse proxy if you use it. @rgaricano is that related to the 30 seconds Gateway Timeout for document Uploading/Embedding? According to the docs, the default is already set to 300 seconds. How would a reverse proxy tackle the internal timeout of Open WebUI?

GiteaMirror commented

2026-04-19 23:12:14 -05:00

@rgaricano commented on GitHub (Jul 16, 2025):

maybe? https://github.com/open-webui/open-webui/discussions/11345#discussioncomment-12423102

I don't know if you are using proxy to access, just a suggestion for check.
(any console log? )

@rgaricano commented on GitHub (Jul 16, 2025): maybe? https://github.com/open-webui/open-webui/discussions/11345#discussioncomment-12423102 I don't know if you are using proxy to access, just a suggestion for check. (any console log? )

GiteaMirror commented

2026-04-19 23:12:15 -05:00

@kns-12 commented on GitHub (Jul 16, 2025):

maybe? #11345 (comment)

I don't know if you are using proxy to access, just a suggestion for check. (any console log? )

Nope, I am not using any proxy to access it.
I described my situation here: https://github.com/open-webui/open-webui/issues/15758 but it has been closed in favour of this current issue.
I am just getting a Gateway Timeout after exactly 30 seconds each time I try to upload a large file.
Concerning the Open WebUI logs, it just keeps embedding the document, after removing it from the chat due to the Gateway Timeout.

@kns-12 commented on GitHub (Jul 16, 2025): > maybe? [#11345 (comment)](https://github.com/open-webui/open-webui/discussions/11345#discussioncomment-12423102) > > I don't know if you are using proxy to access, just a suggestion for check. (any console log? ) Nope, I am not using any proxy to access it. I described my situation here: https://github.com/open-webui/open-webui/issues/15758 but it has been closed in favour of this current issue. I am just getting a Gateway Timeout after exactly 30 seconds each time I try to upload a large file. Concerning the Open WebUI logs, it just keeps embedding the document, after removing it from the chat due to the Gateway Timeout.

GiteaMirror commented

2026-04-19 23:12:16 -05:00

@rgaricano commented on GitHub (Jul 16, 2025):

follow on your opened thread

@rgaricano commented on GitHub (Jul 16, 2025): follow on your opened thread

GiteaMirror commented

2026-04-19 23:12:17 -05:00

@avmturo commented on GitHub (Aug 1, 2025):

maybe? #11345 (comment)

I don't know if you are using proxy to access, just a suggestion for check. (any console log? )

Nope, I am not using any proxy to access it.
I described my situation here: https://github.com/open-webui/open-webui/issues/15758 but it has been closed in favour of this current issue.
I am just getting a Gateway Timeout after exactly 30 seconds each time I try to upload a large file.
Concerning the Open WebUI logs, it just keeps embedding the document, after removing it from the chat due to the Gateway Timeout.

Marked as duplicate but my understanding is there is actually a couple of separate issues here. To yours specifically, I don't not think the timeout is coming from the OWUI server or config. I have the exact same issues, 504 gateway timeout in the file uploads, when the embedding takes more than 1minute, the POST to files from the frontend times out. For me this timeout comes from my ALB idle timeout. If it is 30 seconds for you it is coming from some other gateway/api/proxy that you have.

This issue thread should fix this by adding the polling, to keep that timeout from idling and is a general improvement. But you could fix it by increasing wherever this timeout is coming from.

I'm on mobile now but will take a proper look when I'm home this weekend.

@avmturo commented on GitHub (Aug 1, 2025): > > maybe? [#11345 (comment)](https://github.com/open-webui/open-webui/discussions/11345#discussioncomment-12423102) > > > > I don't know if you are using proxy to access, just a suggestion for check. (any console log? ) > > Nope, I am not using any proxy to access it. > I described my situation here: https://github.com/open-webui/open-webui/issues/15758 but it has been closed in favour of this current issue. > I am just getting a Gateway Timeout after exactly 30 seconds each time I try to upload a large file. > Concerning the Open WebUI logs, it just keeps embedding the document, after removing it from the chat due to the Gateway Timeout. Marked as duplicate but my understanding is there is actually a couple of separate issues here. To yours specifically, I don't not think the timeout is coming from the OWUI server or config. I have the exact same issues, 504 gateway timeout in the file uploads, when the embedding takes more than 1minute, the POST to files from the frontend times out. For me this timeout comes from my ALB idle timeout. If it is 30 seconds for you it is coming from some other gateway/api/proxy that you have. This issue thread should fix this by adding the polling, to keep that timeout from idling and is a general improvement. But you could fix it by increasing wherever this timeout is coming from. I'm on mobile now but will take a proper look when I'm home this weekend.

GiteaMirror commented

2026-04-19 23:12:18 -05:00

@avmturo commented on GitHub (Aug 1, 2025):

maybe? #11345 (comment)

I don't know if you are using proxy to access, just a suggestion for check. (any console log? )

Nope, I am not using any proxy to access it.
I described my situation here: https://github.com/open-webui/open-webui/issues/15758 but it has been closed in favour of this current issue.
I am just getting a Gateway Timeout after exactly 30 seconds each time I try to upload a large file.
Concerning the Open WebUI logs, it just keeps embedding the document, after removing it from the chat due to the Gateway Timeout.

Marked as duplicate but my understanding is there is actually a couple of separate issues here. To yours specifically, I don't not think the timeout is coming from the OWUI server or config. I have the exact same issues, 504 gateway timeout in the file uploads, when the embedding takes more than 1minute, the POST to files from the frontend times out. For me this timeout comes from my ALB idle timeout. If it is 30 seconds for you it is coming from some other gateway/api/proxy that you have.

The proper issue I think is about why the POST request hangs, while the embedding is happening. The issue does not persist when the embedding is bypassed.

I'm on mobile now but will take a proper look when I'm home this weekend.

@kns-12 - Quick Look shows GCP and Heroku have some default timeouts of 30 seconds - here is my understanding of what's going on for you:

Frontend Posts to files endpoint (begin timer)
Server file endpoint calls process_file
File processing take more then 30seconds
Your request from the frontend timeout (unrelated to the owui server), due to some config from the above mentioned

@avmturo commented on GitHub (Aug 1, 2025): > > > maybe? [#11345 (comment)](https://github.com/open-webui/open-webui/discussions/11345#discussioncomment-12423102) > > > > > > I don't know if you are using proxy to access, just a suggestion for check. (any console log? ) > > > > Nope, I am not using any proxy to access it. > > I described my situation here: https://github.com/open-webui/open-webui/issues/15758 but it has been closed in favour of this current issue. > > I am just getting a Gateway Timeout after exactly 30 seconds each time I try to upload a large file. > > Concerning the Open WebUI logs, it just keeps embedding the document, after removing it from the chat due to the Gateway Timeout. > > Marked as duplicate but my understanding is there is actually a couple of separate issues here. To yours specifically, I don't not think the timeout is coming from the OWUI server or config. I have the exact same issues, 504 gateway timeout in the file uploads, when the embedding takes more than 1minute, the POST to files from the frontend times out. For me this timeout comes from my ALB idle timeout. If it is 30 seconds for you it is coming from some other gateway/api/proxy that you have. > > > The proper issue I think is about why the POST request hangs, while the embedding is happening. The issue does not persist when the embedding is bypassed. > > I'm on mobile now but will take a proper look when I'm home this weekend. @kns-12 - Quick Look shows GCP and Heroku have some default timeouts of 30 seconds - here is my understanding of what's going on for you: 1. Frontend Posts to files endpoint (begin timer) 2. Server file endpoint calls process_file 3. File processing take more then 30seconds 4. Your request from the frontend timeout (unrelated to the owui server), due to some config from the above mentioned

GiteaMirror commented

2026-04-19 23:12:19 -05:00

@kns-12 commented on GitHub (Aug 5, 2025):

maybe? #11345 (comment)
I don't know if you are using proxy to access, just a suggestion for check. (any console log? )

Nope, I am not using any proxy to access it.
I described my situation here: #15758 but it has been closed in favour of this current issue.
I am just getting a Gateway Timeout after exactly 30 seconds each time I try to upload a large file.
Concerning the Open WebUI logs, it just keeps embedding the document, after removing it from the chat due to the Gateway Timeout.

Marked as duplicate but my understanding is there is actually a couple of separate issues here. To yours specifically, I don't not think the timeout is coming from the OWUI server or config. I have the exact same issues, 504 gateway timeout in the file uploads, when the embedding takes more than 1minute, the POST to files from the frontend times out. For me this timeout comes from my ALB idle timeout. If it is 30 seconds for you it is coming from some other gateway/api/proxy that you have.
The proper issue I think is about why the POST request hangs, while the embedding is happening. The issue does not persist when the embedding is bypassed.
I'm on mobile now but will take a proper look when I'm home this weekend.

@kns-12 - Quick Look shows GCP and Heroku have some default timeouts of 30 seconds - here is my understanding of what's going on for you:

Frontend Posts to files endpoint (begin timer)

Server file endpoint calls process_file

File processing take more then 30seconds

Your request from the frontend timeout (unrelated to the owui server), due to some config from the above mentioned

Hi,
it seems like the timeout happens internally in open-webui during the file upload/embedding.
The embedding still continues and finishes after the file gets removed from the chat in my case.
Also the upload to s3 as well as my own vector storage works.
I am not using GCP or Heroku and the remote server isnt timeouting either.
Having the async upload like described in this issue would resolve that most likely.
There is no possibility to extend the timeout in open-webui afaik

@kns-12 commented on GitHub (Aug 5, 2025): > > > > maybe? [#11345 (comment)](https://github.com/open-webui/open-webui/discussions/11345#discussioncomment-12423102) > > > > I don't know if you are using proxy to access, just a suggestion for check. (any console log? ) > > > > > > > > > Nope, I am not using any proxy to access it. > > > I described my situation here: [#15758](https://github.com/open-webui/open-webui/issues/15758) but it has been closed in favour of this current issue. > > > I am just getting a Gateway Timeout after exactly 30 seconds each time I try to upload a large file. > > > Concerning the Open WebUI logs, it just keeps embedding the document, after removing it from the chat due to the Gateway Timeout. > > > > > > Marked as duplicate but my understanding is there is actually a couple of separate issues here. To yours specifically, I don't not think the timeout is coming from the OWUI server or config. I have the exact same issues, 504 gateway timeout in the file uploads, when the embedding takes more than 1minute, the POST to files from the frontend times out. For me this timeout comes from my ALB idle timeout. If it is 30 seconds for you it is coming from some other gateway/api/proxy that you have. > > The proper issue I think is about why the POST request hangs, while the embedding is happening. The issue does not persist when the embedding is bypassed. > > I'm on mobile now but will take a proper look when I'm home this weekend. > > [@kns-12](https://github.com/kns-12) - Quick Look shows GCP and Heroku have some default timeouts of 30 seconds - here is my understanding of what's going on for you: > > 1. Frontend Posts to files endpoint (begin timer) > 2. Server file endpoint calls process_file > 3. File processing take more then 30seconds > 4. Your request from the frontend timeout (unrelated to the owui server), due to some config from the above mentioned Hi, it seems like the timeout happens internally in open-webui during the file upload/embedding. The embedding still continues and finishes after the file gets removed from the chat in my case. Also the upload to s3 as well as my own vector storage works. I am not using GCP or Heroku and the remote server isnt timeouting either. Having the async upload like described in this issue would resolve that most likely. There is no possibility to extend the timeout in open-webui afaik

GiteaMirror commented

2026-04-19 23:12:21 -05:00

@rgaricano commented on GitHub (Aug 5, 2025):

openwebui env vars:
AIOHTTP_CLIENT_TIMEOUT (default 300)
AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER_DATA (default 10)
AIOHTTP_CLIENT_TIMEOUT_OPENAI_MODEL_LIST (default 10)

also I saw that is hardcoded in
b8da4a8cd8/backend/open_webui/main.py (L1695)

(maybe increase it or set as total=None or 0 for unlimited &| adding also sock_connect & sock_read times params? )

@rgaricano commented on GitHub (Aug 5, 2025): openwebui env vars: AIOHTTP_CLIENT_TIMEOUT (default 300) AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER_DATA (default 10) AIOHTTP_CLIENT_TIMEOUT_OPENAI_MODEL_LIST (default 10) also I saw that is hardcoded in https://github.com/open-webui/open-webui/blob/b8da4a8cd8257d4846f3608e299618a0b4f185ed/backend/open_webui/main.py#L1695 (maybe increase it or set as total=None or 0 for unlimited &| adding also sock_connect & sock_read times params? )

GiteaMirror commented

2026-04-19 23:12:21 -05:00

@kns-12 commented on GitHub (Aug 5, 2025):

openwebui env vars: AIOHTTP_CLIENT_TIMEOUT (default 300) AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER_DATA (default 10) AIOHTTP_CLIENT_TIMEOUT_OPENAI_MODEL_LIST (default 10)

also I saw that is hardcoded in

open-webui/backend/open_webui/main.py

Line 1695 in b8da4a8

timeout = aiohttp.ClientTimeout(total=1)
(maybe increase it or set as total=None or 0 for unlimited &| adding also sock_connect & sock_read times params? )

Setting the mentioned ENVs doesn't have any effect.
Think I will just wait for the refactoring. As a workaround splitting large files does work.

@kns-12 commented on GitHub (Aug 5, 2025): > openwebui env vars: AIOHTTP_CLIENT_TIMEOUT (default 300) AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER_DATA (default 10) AIOHTTP_CLIENT_TIMEOUT_OPENAI_MODEL_LIST (default 10) > > also I saw that is hardcoded in > > [open-webui/backend/open_webui/main.py](https://github.com/open-webui/open-webui/blob/b8da4a8cd8257d4846f3608e299618a0b4f185ed/backend/open_webui/main.py#L1695) > > Line 1695 in [b8da4a8](/open-webui/open-webui/commit/b8da4a8cd8257d4846f3608e299618a0b4f185ed) > > timeout = aiohttp.ClientTimeout(total=1) > (maybe increase it or set as total=None or 0 for unlimited &| adding also sock_connect & sock_read times params? ) Setting the mentioned ENVs doesn't have any effect. Think I will just wait for the refactoring. As a workaround splitting large files does work.

GiteaMirror commented

2026-04-19 23:12:22 -05:00

@decent-engineer-decent-datascientist commented on GitHub (Aug 11, 2025):

Hey friends, signing up to this thread.

@decent-engineer-decent-datascientist commented on GitHub (Aug 11, 2025): Hey friends, signing up to this thread.

GiteaMirror commented

2026-04-19 23:12:24 -05:00

@robbiekouwenberg commented on GitHub (Aug 13, 2025):

+1 for refactor. Also less hardcoded vars in general would be nice

@robbiekouwenberg commented on GitHub (Aug 13, 2025): +1 for refactor. Also less hardcoded vars in general would be nice

GiteaMirror commented

2026-04-19 23:12:25 -05:00

@tjbck commented on GitHub (Aug 19, 2025):

Addressed with 5e1f4fa0ff in dev.

@tjbck commented on GitHub (Aug 19, 2025): Addressed with 5e1f4fa0ff75cec8a4569797ff3bdd118eb0e601 in dev.

GiteaMirror referenced this issue

2026-04-20 00:29:29 -05:00

[GH-ISSUE #17442] feat: Add a youtube video searcher, like what is the latest video from this channel, shall be easy with Youtube API ig #18288

GiteaMirror referenced this issue

2026-04-25 07:42:08 -05:00

[GH-ISSUE #17442] feat: Add a youtube video searcher, like what is the latest video from this channel, shall be easy with Youtube API ig #33817

GiteaMirror referenced this issue