When using direct connection mode with multiple backend workers (uvicorn workers > 1 or multiple k8s pods), streaming chat requests intermittently timeout and never complete.
Root Cause: The generate_direct_chat_completion function dynamically registers a Socket.IO event handler using sio.on(channel, handler) for each streaming request. This handler only exists in the local worker's memory. When the browser's WebSocket connection lives on a different worker, streaming chunks arrive at that worker but no handler exists there - causing messages to be silently dropped and the original request to wait forever.
Solution
Replace per-request dynamic handler registration with a global static handler and Redis pub/sub routing:
Global Queue Registry: Add DIRECT_CHAT_QUEUES dictionary to track channel-to-queue mappings per worker
Static Global Handler: Add a sio.on("direct-chat-stream") handler that exists in all workers and routes messages to local queues or publishes via Redis
Redis Pub/Sub Listener: Each worker subscribes to direct chat channel patterns and delivers messages to local queues when present
Frontend Update: Changed socket.emit to use the global event name with channel metadata instead of dynamic channel names
Flow After Fix
HTTP request arrives at Worker A, registers queue in DIRECT_CHAT_QUEUES
Browser emits streaming chunks to "direct-chat-stream" global event
Worker B (holding WebSocket) receives event via global handler
Worker B publishes to Redis since queue is not local
Worker A receives Redis message, finds local queue, delivers data
Streaming completes successfully
Files Changed
backend/open_webui/socket/main.py - Added global handler and Redis pub/sub listener
backend/open_webui/utils/chat.py - Use queue registry instead of dynamic sio.on()
backend/open_webui/main.py - Start Redis listener on startup
src/routes/+layout.svelte - Emit to global event with channel metadata
Testing
Deploy with WEBSOCKET_MANAGER=redis and multiple workers
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
Note
Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/20259
**Author:** [@Classic298](https://github.com/Classic298)
**Created:** 12/30/2025
**Status:** ❌ Closed
**Base:** `dev` ← **Head:** `redis-queue`
---
### 📝 Commits (10+)
- [`710003b`](https://github.com/open-webui/open-webui/commit/710003b2f8d3a4796cd71f5c1c9f5bcfa0103e60) init
- [`8fa83a0`](https://github.com/open-webui/open-webui/commit/8fa83a0f1f877c49d3bd74cb02454e3673a75d04) Update main.py
- [`5a61241`](https://github.com/open-webui/open-webui/commit/5a61241900999c1e957c62f49a33b95422a9615f) Update chat.py
- [`a0849d0`](https://github.com/open-webui/open-webui/commit/a0849d00f98ccc3ba82ff4b2851f4fd8af9da399) Update +layout.svelte
- [`2cfd08b`](https://github.com/open-webui/open-webui/commit/2cfd08b2c2b5339315fca787b3bfdc118274067e) Update main.py
- [`44450e8`](https://github.com/open-webui/open-webui/commit/44450e8a6dc75754daf013bb6c91bf8c3039a168) Update main.py
- [`3950e22`](https://github.com/open-webui/open-webui/commit/3950e2222fa5ec9a6b5888cf48fcd72c9d943f22) Update main.py
- [`296686d`](https://github.com/open-webui/open-webui/commit/296686da4081cb52704b45b05080a4553154c2a8) Update main.py
- [`5107f34`](https://github.com/open-webui/open-webui/commit/5107f34daa75869728bb65fdddbe3921ca5b40e0) Update main.py
- [`7db6d6c`](https://github.com/open-webui/open-webui/commit/7db6d6cbfc71fc99e1338be16b23792e6260032b) Update chat.py
### 📊 Changes
**3 files changed** (+98 additions, -7 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/open_webui/main.py` (+14 -0)
📝 `backend/open_webui/socket/main.py` (+69 -0)
📝 `backend/open_webui/utils/chat.py` (+15 -7)
</details>
### 📄 Description
## Problem
When using direct connection mode with multiple backend workers (uvicorn workers > 1 or multiple k8s pods), streaming chat requests intermittently timeout and never complete.
**Root Cause:** The generate_direct_chat_completion function dynamically registers a Socket.IO event handler using sio.on(channel, handler) for each streaming request. This handler only exists in the local worker's memory. When the browser's WebSocket connection lives on a different worker, streaming chunks arrive at that worker but no handler exists there - causing messages to be silently dropped and the original request to wait forever.
## Solution
Replace per-request dynamic handler registration with a global static handler and Redis pub/sub routing:
1. **Global Queue Registry**: Add DIRECT_CHAT_QUEUES dictionary to track channel-to-queue mappings per worker
2. **Static Global Handler**: Add a sio.on("direct-chat-stream") handler that exists in all workers and routes messages to local queues or publishes via Redis
3. **Redis Pub/Sub Listener**: Each worker subscribes to direct chat channel patterns and delivers messages to local queues when present
4. **Frontend Update**: Changed socket.emit to use the global event name with channel metadata instead of dynamic channel names
## Flow After Fix
1. HTTP request arrives at Worker A, registers queue in DIRECT_CHAT_QUEUES
2. Browser emits streaming chunks to "direct-chat-stream" global event
3. Worker B (holding WebSocket) receives event via global handler
4. Worker B publishes to Redis since queue is not local
5. Worker A receives Redis message, finds local queue, delivers data
6. Streaming completes successfully
## Files Changed
- backend/open_webui/socket/main.py - Added global handler and Redis pub/sub listener
- backend/open_webui/utils/chat.py - Use queue registry instead of dynamic sio.on()
- backend/open_webui/main.py - Start Redis listener on startup
- src/routes/+layout.svelte - Emit to global event with channel metadata
## Testing
- Deploy with WEBSOCKET_MANAGER=redis and multiple workers
- Enable direct connection mode in user settings
- Send concurrent streaming chat requests
- Verify all requests complete without timeout
Closes #15162
### Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms.
> [!NOTE]
> Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/20259
Author: @Classic298
Created: 12/30/2025
Status: ❌ Closed
Base:
dev← Head:redis-queue📝 Commits (10+)
710003binit8fa83a0Update main.py5a61241Update chat.pya0849d0Update +layout.svelte2cfd08bUpdate main.py44450e8Update main.py3950e22Update main.py296686dUpdate main.py5107f34Update main.py7db6d6cUpdate chat.py📊 Changes
3 files changed (+98 additions, -7 deletions)
View changed files
📝
backend/open_webui/main.py(+14 -0)📝
backend/open_webui/socket/main.py(+69 -0)📝
backend/open_webui/utils/chat.py(+15 -7)📄 Description
Problem
When using direct connection mode with multiple backend workers (uvicorn workers > 1 or multiple k8s pods), streaming chat requests intermittently timeout and never complete.
Root Cause: The generate_direct_chat_completion function dynamically registers a Socket.IO event handler using sio.on(channel, handler) for each streaming request. This handler only exists in the local worker's memory. When the browser's WebSocket connection lives on a different worker, streaming chunks arrive at that worker but no handler exists there - causing messages to be silently dropped and the original request to wait forever.
Solution
Replace per-request dynamic handler registration with a global static handler and Redis pub/sub routing:
Flow After Fix
Files Changed
Testing
Closes #15162
Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.