[GH-ISSUE #24579] issue: /api/chat/completions hangs indefinitely behind IIS/ARR proxy after upgrading from 0.9.2 to 0.9.3 #107341

New Issue

GiteaMirror · 2026-05-18T06:08:14-05:00

GiteaMirror commented

2026-05-18 06:08:14 -05:00

Originally created by @Polak149 on GitHub (May 11, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/24579

Check Existing Issues

I have searched for any existing and/or related issues.
I have searched for any existing and/or related discussions.
I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!).
I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

0.9.5

Ollama Version (if applicable)

0.23.2

Operating System

Ubuntu 22.04

Browser (if applicable)

No response

Confirmation

I have read and followed all instructions in README.md.
I am using the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have provided every relevant configuration, setting, and environment variable used in my setup.
I have clearly listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc).
I have documented step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation. My steps:
Start with the initial platform/version/OS and dependencies used,
Specify exact install/launch/configure commands,
List URLs visited, user input (incl. example values/emails/passwords if needed),
Describe all options and toggles enabled or changed,
Include any files or environmental changes,
Identify the expected and actual result at each stage,
Ensure any reasonably skilled user can follow and hit the same issue.

Expected Behavior

When sending a message in the chat UI, the /api/chat/completions request should stream the response correctly and finish normally.

The generated assistant response should appear in the UI without requiring a browser refresh.

This behavior worked correctly in Open WebUI 0.9.2 with the same IIS/ARR reverse proxy configuration.

Actual Behavior

After upgrading Open WebUI from 0.9.2 to 0.9.3, chat generation no longer works correctly when Open WebUI is accessed through IIS with Application Request Routing reverse proxy.

The message is sent correctly and the LLM generates a response. However, the browser request to:

/api/chat/completions

remains in pending state indefinitely.

The assistant response is not streamed or rendered live in the UI.

After refreshing the browser page, the generated response becomes visible in the conversation history. This indicates that the backend/model generation completed successfully and the response was persisted, but the streaming response did not complete properly through IIS/ARR.

IIS sometimes returns:

502 - Web server received an invalid response while acting as a gateway or proxy server.

The issue does not occur when accessing Open WebUI directly on the internal address/port. It only occurs through IIS/ARR.

Steps to Reproduce

Deploy Open WebUI 0.9.3 in Docker.
Expose Open WebUI internally on port 9000.
Configure IIS 10 with Application Request Routing and URL Rewrite as a reverse proxy to:
http://10.10.0.17:9000
Open Open WebUI through the public HTTPS domain handled by IIS.
Send any message in the chat UI.
Open browser DevTools → Network.
Observe that the /api/chat/completions request remains in pending state indefinitely.
Refresh the browser page.
The generated assistant response is now visible in the conversation history.

Comparison test:

Open WebUI 0.9.2 + same IIS/ARR config = works correctly
Open WebUI 0.9.3 + same IIS/ARR config = /api/chat/completions hangs

Direct access test:

http://10.10.0.17:9000

Result:

Works correctly

Through IIS/ARR reverse proxy:

https://public-domain

Result:

/api/chat/completions remains pending indefinitely
Response appears only after page refresh

Logs & Screenshots

Open WebUI container logs do not show any obvious error.

Browser DevTools Network tab shows:

tools/          200
completions     pending
version.json    200

The request to /api/chat/completions remains pending indefinitely.

IIS sometimes returns:
502 - Web server received an invalid response while acting as a gateway or proxy server.

Current IIS web.config:

<configuration>
  <system.web>
    <httpRuntime requestPathInvalidCharacters=""
                 maxUrlLength="100000"
                 maxQueryStringLength="100000"
                 maxRequestLength="512000" />
  </system.web>

  <system.webServer>
    <security>
      <requestFiltering>
        <requestLimits maxAllowedContentLength="2147483648" />
      </requestFiltering>
    </security>

    <webSocket enabled="true" />

    <rewrite>
      <rules>
        <rule name="WS proxy any" stopProcessing="true">
          <match url="(.*)" />
          <conditions logicalGrouping="MatchAll">
            <add input="{HTTP_UPGRADE}" pattern="WebSocket" ignoreCase="true" />
          </conditions>
          <action type="Rewrite" url="http://10.10.0.17:9000/{R:1}" />
        </rule>

        <rule name="HTTP proxy catch-all" stopProcessing="true">
          <match url="(.*)" />
          <action type="Rewrite" url="http://10.10.0.17:9000/{R:1}" />
        </rule>
      </rules>
    </rewrite>

    <urlCompression doDynamicCompression="false" />
  </system.webServer>
</configuration>

ARR proxy settings tested:

Enable proxy: ON
HTTP version: Pass through / also tested HTTP/1.1
Keep alive: ON
Timeout: tested 120 and higher values
Reverse rewrite host in response headers: OFF
Enable disk cache: OFF
Enable request consolidation: OFF
Response buffer threshold (KB): 0
Response buffer (KB): 65536

I also tested an explicit configuration with forwarded headers and disabled buffering/compression:

<serverVariables>
  <set name="HTTP_X_FORWARDED_PROTO" value="https" />
  <set name="HTTP_X_FORWARDED_SCHEME" value="https" />
  <set name="HTTP_X_FORWARDED_HOST" value="{HTTP_HOST}" />
  <set name="HTTP_X_FORWARDED_FOR" value="{REMOTE_ADDR}" />
  <set name="HTTP_ACCEPT_ENCODING" value="" />
  <set name="RESPONSE_X_ACCEL_BUFFERING" value="no" />
  <set name="RESPONSE_CACHE_CONTROL" value="no-cache" />
</serverVariables>

This did not resolve the issue.

Additional Information

Environment:

Open WebUI version: 0.9.3
Previous working version: 0.9.2
Deployment: Docker
Reverse proxy: IIS 10 + Application Request Routing + URL Rewrite
Internal Open WebUI URL: http://10.10.0.17:9000
Public access: HTTPS domain through IIS reverse proxy
Browser: Chrome / Edge
Model example: kimi-k2.6:cloud

Important observation:

The backend appears to complete generation correctly, because the generated response is visible after refreshing the page. The issue seems related to the streaming response lifecycle of /api/chat/completions when passing through IIS/ARR.

Possible regression areas:

Streaming response format
Chunked transfer encoding
SSE/fetch streaming behavior
Response headers
Connection close/finalization
Keep-alive behavior
Compression/buffering interaction

Question:

Was anything changed in Open WebUI 0.9.3 regarding /api/chat/completions streaming, response headers, chunked transfer encoding, or connection finalization that could cause incompatibility with IIS/ARR reverse proxy?

The regression is reproducible because the exact same IIS/ARR setup works correctly with 0.9.2.

Update 0.9.4 and 0.9.5 did not fix the issue

Originally created by @Polak149 on GitHub (May 11, 2026). Original GitHub issue: https://github.com/open-webui/open-webui/issues/24579 ### Check Existing Issues - [x] I have searched for any existing and/or related issues. - [x] I have searched for any existing and/or related discussions. - [x] I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!). - [x] I am using the latest version of Open WebUI. ### Installation Method Docker ### Open WebUI Version 0.9.5 ### Ollama Version (if applicable) 0.23.2 ### Operating System Ubuntu 22.04 ### Browser (if applicable) _No response_ ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have **provided every relevant configuration, setting, and environment variable used in my setup.** - [x] I have clearly **listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup** (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc). - [x] I have documented **step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation**. My steps: - Start with the initial platform/version/OS and dependencies used, - Specify exact install/launch/configure commands, - List URLs visited, user input (incl. example values/emails/passwords if needed), - Describe all options and toggles enabled or changed, - Include any files or environmental changes, - Identify the expected and actual result at each stage, - Ensure any reasonably skilled user can follow and hit the same issue. ### Expected Behavior When sending a message in the chat UI, the /api/chat/completions request should stream the response correctly and finish normally. The generated assistant response should appear in the UI without requiring a browser refresh. This behavior worked correctly in Open WebUI 0.9.2 with the same IIS/ARR reverse proxy configuration. ### Actual Behavior After upgrading Open WebUI from 0.9.2 to 0.9.3, chat generation no longer works correctly when Open WebUI is accessed through IIS with Application Request Routing reverse proxy. The message is sent correctly and the LLM generates a response. However, the browser request to: `/api/chat/completions` remains in pending state indefinitely. The assistant response is not streamed or rendered live in the UI. After refreshing the browser page, the generated response becomes visible in the conversation history. This indicates that the backend/model generation completed successfully and the response was persisted, but the streaming response did not complete properly through IIS/ARR. IIS sometimes returns: `502 - Web server received an invalid response while acting as a gateway or proxy server.` The issue does not occur when accessing Open WebUI directly on the internal address/port. It only occurs through IIS/ARR. ### Steps to Reproduce 1. Deploy Open WebUI 0.9.3 in Docker. 2. Expose Open WebUI internally on port 9000. 3. Configure IIS 10 with Application Request Routing and URL Rewrite as a reverse proxy to: `http://10.10.0.17:9000` 4. Open Open WebUI through the public HTTPS domain handled by IIS. 5. Send any message in the chat UI. 6. Open browser DevTools → Network. 7. Observe that the /api/chat/completions request remains in pending state indefinitely. 8. Refresh the browser page. 9. The generated assistant response is now visible in the conversation history. Comparison test: ``` Open WebUI 0.9.2 + same IIS/ARR config = works correctly Open WebUI 0.9.3 + same IIS/ARR config = /api/chat/completions hangs ``` Direct access test: `http://10.10.0.17:9000` Result: `Works correctly` Through IIS/ARR reverse proxy: `https://public-domain` Result: ``` /api/chat/completions remains pending indefinitely Response appears only after page refresh ``` ### Logs & Screenshots Open WebUI container logs do not show any obvious error. Browser DevTools Network tab shows: ``` tools/ 200 completions pending version.json 200 ``` The request to /api/chat/completions remains pending indefinitely. IIS sometimes returns: `502 - Web server received an invalid response while acting as a gateway or proxy server.` Current IIS web.config: ``` <configuration> <system.web> <httpRuntime requestPathInvalidCharacters="" maxUrlLength="100000" maxQueryStringLength="100000" maxRequestLength="512000" /> </system.web> <system.webServer> <security> <requestFiltering> <requestLimits maxAllowedContentLength="2147483648" /> </requestFiltering> </security> <webSocket enabled="true" /> <rewrite> <rules> <rule name="WS proxy any" stopProcessing="true"> <match url="(.*)" /> <conditions logicalGrouping="MatchAll"> <add input="{HTTP_UPGRADE}" pattern="WebSocket" ignoreCase="true" /> </conditions> <action type="Rewrite" url="http://10.10.0.17:9000/{R:1}" /> </rule> <rule name="HTTP proxy catch-all" stopProcessing="true"> <match url="(.*)" /> <action type="Rewrite" url="http://10.10.0.17:9000/{R:1}" /> </rule> </rules> </rewrite> <urlCompression doDynamicCompression="false" /> </system.webServer> </configuration> ``` ARR proxy settings tested: ``` Enable proxy: ON HTTP version: Pass through / also tested HTTP/1.1 Keep alive: ON Timeout: tested 120 and higher values Reverse rewrite host in response headers: OFF Enable disk cache: OFF Enable request consolidation: OFF Response buffer threshold (KB): 0 Response buffer (KB): 65536 ``` I also tested an explicit configuration with forwarded headers and disabled buffering/compression: ``` <serverVariables> <set name="HTTP_X_FORWARDED_PROTO" value="https" /> <set name="HTTP_X_FORWARDED_SCHEME" value="https" /> <set name="HTTP_X_FORWARDED_HOST" value="{HTTP_HOST}" /> <set name="HTTP_X_FORWARDED_FOR" value="{REMOTE_ADDR}" /> <set name="HTTP_ACCEPT_ENCODING" value="" /> <set name="RESPONSE_X_ACCEL_BUFFERING" value="no" /> <set name="RESPONSE_CACHE_CONTROL" value="no-cache" /> </serverVariables> ``` This did not resolve the issue. ### Additional Information Environment: ``` Open WebUI version: 0.9.3 Previous working version: 0.9.2 Deployment: Docker Reverse proxy: IIS 10 + Application Request Routing + URL Rewrite Internal Open WebUI URL: http://10.10.0.17:9000 Public access: HTTPS domain through IIS reverse proxy Browser: Chrome / Edge Model example: kimi-k2.6:cloud ``` Important observation: The backend appears to complete generation correctly, because the generated response is visible after refreshing the page. The issue seems related to the streaming response lifecycle of /api/chat/completions when passing through IIS/ARR. Possible regression areas: ``` Streaming response format Chunked transfer encoding SSE/fetch streaming behavior Response headers Connection close/finalization Keep-alive behavior Compression/buffering interaction ``` Question: Was anything changed in Open WebUI 0.9.3 regarding /api/chat/completions streaming, response headers, chunked transfer encoding, or connection finalization that could cause incompatibility with IIS/ARR reverse proxy? The regression is reproducible because the exact same IIS/ARR setup works correctly with 0.9.2. Update 0.9.4 and 0.9.5 did not fix the issue

GiteaMirror added the bug label 2026-05-18 06:08:14 -05:00

GiteaMirror closed this issue

2026-05-18 06:08:17 -05:00

GiteaMirror commented

2026-05-18 06:08:19 -05:00

@owui-terminator[bot] commented on GitHub (May 11, 2026):

🔍 Related Issues Found

I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions:

🟢 #24553 issue: /api/chat/completions runs into error
This is the closest current open report on /api/chat/completions misbehaving in 0.9.5. While the exact symptom differs, it involves the same endpoint and version range, making it a likely related regression.
by tweinberger-lei · bug
🟣 #23920 fix: Stale Content-Encoding header forwarded from upstream APIs causes ZlibError in desktop clients
This closed issue documents a streaming/proxy header regression in the OpenAI-compatible API path, where stale response headers broke SSE clients. Your issue also appears to be a streaming/proxy compatibility problem after an upgrade, so it is relevant as a possible neighboring regression area.
by pvyswiss
🟣 #22141 issue: Tokens dropped with streaming enabled
This issue is about streaming token delivery problems in Open WebUI's chat completions path. Although it is about dropped tokens rather than an indefinite hang, it points to the same streaming pipeline and buffering logic that could be implicated in your regression.
by anselor · bug

💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead.

This comment was generated automatically. React with 👍 if helpful, 👎 if not.

@owui-terminator[bot] commented on GitHub (May 11, 2026):  🔍 **Related Issues Found** I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions: 1. 🟢 [#24553](https://github.com/open-webui/open-webui/issues/24553) **issue: /api/chat/completions runs into error** *This is the closest current open report on `/api/chat/completions` misbehaving in 0.9.5. While the exact symptom differs, it involves the same endpoint and version range, making it a likely related regression.* *by tweinberger-lei · `bug`* 2. 🟣 [#23920](https://github.com/open-webui/open-webui/issues/23920) **fix: Stale Content-Encoding header forwarded from upstream APIs causes ZlibError in desktop clients** *This closed issue documents a streaming/proxy header regression in the OpenAI-compatible API path, where stale response headers broke SSE clients. Your issue also appears to be a streaming/proxy compatibility problem after an upgrade, so it is relevant as a possible neighboring regression area.* *by pvyswiss* 3. 🟣 [#22141](https://github.com/open-webui/open-webui/issues/22141) **issue: Tokens dropped with streaming enabled** *This issue is about streaming token delivery problems in Open WebUI's chat completions path. Although it is about dropped tokens rather than an indefinite hang, it points to the same streaming pipeline and buffering logic that could be implicated in your regression.* *by anselor · `bug`* --- 💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead. *This comment was generated automatically.* React with 👍 if helpful, 👎 if not.

GiteaMirror commented

2026-05-18 06:08:19 -05:00

@Classic298 commented on GitHub (May 11, 2026):

I don't think anything in regards to websockets or how the response is handled has changed since 0.9.0... but i'll take a look

@Classic298 commented on GitHub (May 11, 2026): I don't think anything in regards to websockets or how the response is handled has changed since 0.9.0... but i'll take a look

GiteaMirror commented

2026-05-18 06:08:20 -05:00

@Classic298 commented on GitHub (May 11, 2026):

I went through every change to /api/chat/completions and its surrounding code paths between v0.9.2 and v0.9.5 and can't find a single regression that would change how this endpoint behaves through a reverse proxy.

A few things worth knowing about the endpoint in the UI flow:

It is not a streaming endpoint. When session_id + chat_id are present (i.e. you have a working socket), it kicks off a background task and returns a small JSON: { "status": true, "task_ids": [...], "chat_id": "..." }. The actual tokens stream via Socket.IO, not the HTTP body. That code path is identical in 0.9.2 and 0.9.5.
If session_id is missing (i.e. the WebSocket never connected), the backend falls back to a real SSE StreamingResponse over /api/chat/completions. ARR is known to buffer/mangle chunked SSE responses. This fallback existed identically in 0.9.2 too.
_clean_proxy_headers (strips Content-Encoding/Content-Length/Transfer-Encoding from upstream), CompressMiddleware, the AuthToken/CommitSession middlewares, and the Socket.IO setup are all unchanged between 0.9.2 and 0.9.5. No dependency bumps either.

My best guess is that this is a WebSocket-establishment issue through your ARR config (not a backend code change). If the WS handshake fails, the frontend posts the chat completion with session_id: undefined and the endpoint goes down the SSE-streaming legacy branch, which ARR will gladly buffer to death.

To confirm, can you check:

The full URL of the "pending" request in DevTools — is it /api/chat/completions or /ws/socket.io/?transport=…?
Does /ws/socket.io/?...&transport=websocket actually return 101 Switching Protocols?
What's the response Content-Type header on the pending /api/chat/completions request? application/json vs text/event-stream would tell us which branch you're on.
As a quick test, set ENABLE_WEBSOCKET=false to force long-polling — if the hang goes away, it's the WS handshake; if it stays, it's response delivery.

@Classic298 commented on GitHub (May 11, 2026): I went through every change to `/api/chat/completions` and its surrounding code paths between v0.9.2 and v0.9.5 and can't find a single regression that would change how this endpoint behaves through a reverse proxy. A few things worth knowing about the endpoint in the UI flow: - It is **not** a streaming endpoint. When `session_id` + `chat_id` are present (i.e. you have a working socket), it kicks off a background task and returns a small JSON: `{ "status": true, "task_ids": [...], "chat_id": "..." }`. The actual tokens stream via Socket.IO, not the HTTP body. That code path is identical in 0.9.2 and 0.9.5. - If `session_id` is missing (i.e. the WebSocket never connected), the backend falls back to a real SSE `StreamingResponse` over `/api/chat/completions`. ARR is known to buffer/mangle chunked SSE responses. This fallback existed identically in 0.9.2 too. - `_clean_proxy_headers` (strips `Content-Encoding`/`Content-Length`/`Transfer-Encoding` from upstream), `CompressMiddleware`, the AuthToken/CommitSession middlewares, and the Socket.IO setup are all unchanged between 0.9.2 and 0.9.5. No dependency bumps either. My best guess is that this is a WebSocket-establishment issue through your ARR config (not a backend code change). If the WS handshake fails, the frontend posts the chat completion with `session_id: undefined` and the endpoint goes down the SSE-streaming legacy branch, which ARR will gladly buffer to death. To confirm, can you check: 1. The full URL of the "pending" request in DevTools — is it `/api/chat/completions` or `/ws/socket.io/?transport=…`? 2. Does `/ws/socket.io/?...&transport=websocket` actually return `101 Switching Protocols`? 3. What's the response `Content-Type` header on the pending `/api/chat/completions` request? `application/json` vs `text/event-stream` would tell us which branch you're on. 4. As a quick test, set `ENABLE_WEBSOCKET=false` to force long-polling — if the hang goes away, it's the WS handshake; if it stays, it's response delivery.

GiteaMirror commented

2026-05-18 06:08:22 -05:00

@Polak149 commented on GitHub (May 11, 2026):

Thank you for the detailed analysis. Your explanation makes sense and helped me identify the real cause.

It looks like this was not a regression between 0.9.2 and 0.9.3 after all.

Some time ago I added the following Docker environment variable:

ENABLE_WEBSOCKET=true

At that time I only rebooted the server / restarted the environment and assumed that the setting had been applied, because everything still seemed to work.

However, I now realized that the existing Docker container was not recreated, so the new environment variable was not actually applied after the reboot. It only took effect later when I rebuilt/recreated the Docker Compose stack during the Open WebUI update. That made it look like the issue was caused by the Open WebUI version upgrade, but in reality the upgrade was simply the first moment when ENABLE_WEBSOCKET=true became active.

After your explanation, this matches the behavior perfectly:

with working WebSocket, /api/chat/completions should only return a small JSON response with task_ids and chat_id;
if the WebSocket connection is not established correctly, the frontend falls back to the SSE StreamingResponse branch;
IIS/ARR appears to have problems with that fallback streaming response and keeps /api/chat/completions pending.

So the most likely root cause is my IIS/ARR WebSocket proxy configuration, not Open WebUI itself.

I tested disabling WebSocket support again by setting:

ENABLE_WEBSOCKET=false

and recreating the Docker containers properly. After that, the chat works again through IIS/ARR.

So I think this issue can be closed as a configuration/proxy issue on my side rather than a bug in Open WebUI.

For reference, the important lesson here is that changing ENABLE_WEBSOCKET in Docker requires recreating the container, not just rebooting/restarting the host. In my case, the delayed application of that environment variable made the problem appear to be related to the Open WebUI update.

Thank you again for pointing me in the right direction.

@Polak149 commented on GitHub (May 11, 2026): Thank you for the detailed analysis. Your explanation makes sense and helped me identify the real cause. It looks like this was not a regression between 0.9.2 and 0.9.3 after all. Some time ago I added the following Docker environment variable: ENABLE_WEBSOCKET=true At that time I only rebooted the server / restarted the environment and assumed that the setting had been applied, because everything still seemed to work. However, I now realized that the existing Docker container was not recreated, so the new environment variable was not actually applied after the reboot. It only took effect later when I rebuilt/recreated the Docker Compose stack during the Open WebUI update. That made it look like the issue was caused by the Open WebUI version upgrade, but in reality the upgrade was simply the first moment when ENABLE_WEBSOCKET=true became active. After your explanation, this matches the behavior perfectly: - with working WebSocket, /api/chat/completions should only return a small JSON response with task_ids and chat_id; - if the WebSocket connection is not established correctly, the frontend falls back to the SSE StreamingResponse branch; - IIS/ARR appears to have problems with that fallback streaming response and keeps /api/chat/completions pending. So the most likely root cause is my IIS/ARR WebSocket proxy configuration, not Open WebUI itself. I tested disabling WebSocket support again by setting: ENABLE_WEBSOCKET=false and recreating the Docker containers properly. After that, the chat works again through IIS/ARR. So I think this issue can be closed as a configuration/proxy issue on my side rather than a bug in Open WebUI. For reference, the important lesson here is that changing ENABLE_WEBSOCKET in Docker requires recreating the container, not just rebooting/restarting the host. In my case, the delayed application of that environment variable made the problem appear to be related to the Open WebUI update. Thank you again for pointing me in the right direction.

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#107341