Non-streaming responses never print #3101

New Issue

2025-11-11T15:22:30-06:00

GiteaMirror commented

2025-11-11 15:22:30 -06:00

Originally created by @samling on GitHub (Dec 26, 2024).

Bug Report

Installation Method

Kubernetes

Environment

Open WebUI Version: 0.5.0
Ollama (if applicable): n/a
Operating System: Host is Ubuntu 24.04.1
Browser (if applicable): Chrome 131.0.6778.140

Confirmation:

I have read and followed all the instructions provided in the README.md.
I am on the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.

Expected Behavior:

The application prints the complete response when streaming responses is disabled.

Actual Behavior:

The response never returns and the message never breaks (i.e. there is still a "stop" button displayed).

Description

Bug Summary:
The latest update changed the backend behavior of chats to be asynchronous and had some cascading effects on some pipes I wrote. The simplest fix for most was to remove async behavior in my pipes (as that causes all chats, streaming or otherwise, to never break). This works for streaming responses, but I noticed for non-streaming responses nothing is ever returned and there are no errors in the logs or console. In fact, both the console and logs show the complete response, it's just never returned in the chat.

Reproduction Details

Steps to Reproduce:

With a pipe:

Download a simple OpenAI manifold pipe example from here
Configure the API key in both the global valves and user valves (seems like a bug in the pipe)
Enable the pipe and create a new chat
In the chat settings > advanced params, set "Stream Chat Response" to "off"
Send a message
Observe the console and note the returned message json
Observe the logs and note lack of errors

My logs show a lot of errors like "GET /ws/socket.io/?EIO=4&transport=polling&t=PG1aKig HTTP/1.1" 400 Bad Request, but I'm not sure if they're related or even new

With built-in openai connection:

Create new openai connection and configure API key
Enable the pipe and create a new chat
In the chat settings > advanced params, set "Stream Chat Response" to "off"
Send a message
Observe the console and note the returned message json
Observe the logs and note lack of errors

Logs and Screenshots

Browser Console Logs:

Successful completion log from non-streaming message

{
    "id": "chatcmpl-AibGfyy5rTPk6NfYMYH8wXl8c8XC7",
    "object": "chat.completion",
    "created": 1735193053,
    "model": "gpt-4o-2024-11-20",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "Sure! If you’re testing, feel free to ask me anything or let me know how I can help. 😊",
                "refusal": null
            },
            "logprobs": null,
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 26,
        "completion_tokens": 24,
        "total_tokens": 50,
        "prompt_tokens_details": {
            "cached_tokens": 0,
            "audio_tokens": 0
        },
        "completion_tokens_details": {
            "reasoning_tokens": 0,
            "audio_tokens": 0,
            "accepted_prediction_tokens": 0,
            "rejected_prediction_tokens": 0
        }
    },
    "system_fingerprint": "fp_41926dc6d6"
}

Docker Container Logs:

Successful request logs:

open-webui-74fcbff86c-l7pxb open-webui INFO:     10.42.1.0:0 - "POST /api/v1/chats/f1c26b73-34f8-40e2-aef2-a93e6431919a HTTP/1.1" 200 OK
open-webui-74fcbff86c-l7pxb open-webui INFO:     10.42.1.0:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK
open-webui-74fcbff86c-l7pxb open-webui INFO:     10.42.1.0:0 - "POST /api/v1/chats/f1c26b73-34f8-40e2-aef2-a93e6431919a HTTP/1.1" 200 OK
open-webui-74fcbff86c-l7pxb open-webui INFO:     10.42.1.0:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK
open-webui-74fcbff86c-l7pxb open-webui pipe:function_openai
open-webui-74fcbff86c-l7pxb open-webui {'id': '81a607b0-6231-42d7-b7fc-4d0aaa7e2770', 'email': '(my email)', 'name': 'Sam', 'role': 'admin', 'valves': UserValves(OPENAI_API_KEY='sk-my-key')}
open-webui-74fcbff86c-l7pxb open-webui {'stream': False, 'model': 'gpt-4o-2024-11-20', 'messages': [{'role': 'user', 'content': 'Test'}, {'role': 'assistant', 'content': 'Hi! How can I assist you today?'}, {'role': 'user', 'content': 'Test'}]}
open-webui-74fcbff86c-l7pxb open-webui INFO:     10.42.1.0:0 - "POST /api/chat/completions HTTP/1.1" 200 OK
open-webui-74fcbff86c-l7pxb open-webui INFO:     10.42.1.0:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK

Screenshots/Screen Recordings (if applicable):

Additional Information

This seems to also occur with the built-in openai connection.
Streaming responses do work and do complete (when the pipe is reconfigured to be synchronous).
My Open-WebUI is behind Traefik, but Traefik should handle websockets without any special configuration.

Note

If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!

Originally created by @samling on GitHub (Dec 26, 2024). # Bug Report ## Installation Method Kubernetes ## Environment - **Open WebUI Version:** 0.5.0 - **Ollama (if applicable):** n/a - **Operating System:** Host is Ubuntu 24.04.1 - **Browser (if applicable):** Chrome 131.0.6778.140 **Confirmation:** - [x] I have read and followed all the instructions provided in the README.md. - [x] I am on the latest version of both Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below. ## Expected Behavior: The application prints the complete response when streaming responses is disabled. ## Actual Behavior: The response never returns and the message never breaks (i.e. there is still a "stop" button displayed). ## Description **Bug Summary:** The latest update changed the backend behavior of chats to be asynchronous and had some cascading effects on some pipes I wrote. The simplest fix for most was to remove async behavior in my pipes (as that causes all chats, streaming or otherwise, to never break). This works for streaming responses, but I noticed for non-streaming responses nothing is ever returned and there are no errors in the logs or console. In fact, both the console and logs show the complete response, it's just never returned in the chat. ## Reproduction Details **Steps to Reproduce:** _With a pipe:_ 1. Download a simple OpenAI manifold pipe example from [here](https://openwebui.com/f/hub/openai_manifold) 2. Configure the API key in both the global valves and user valves (seems like a bug in the pipe) 3. Enable the pipe and create a new chat 4. In the chat settings > advanced params, set "Stream Chat Response" to "off" 5. Send a message 6. Observe the console and note the returned message json 7. Observe the logs and note lack of errors - My logs show a lot of errors like `"GET /ws/socket.io/?EIO=4&transport=polling&t=PG1aKig HTTP/1.1" 400 Bad Request`, but I'm not sure if they're related or even new _With built-in openai connection:_ 1. Create new openai connection and configure API key 2. Enable the pipe and create a new chat 3. In the chat settings > advanced params, set "Stream Chat Response" to "off" 4. Send a message 5. Observe the console and note the returned message json 6. Observe the logs and note lack of errors ## Logs and Screenshots **Browser Console Logs:** Successful completion log from non-streaming message ```bash { "id": "chatcmpl-AibGfyy5rTPk6NfYMYH8wXl8c8XC7", "object": "chat.completion", "created": 1735193053, "model": "gpt-4o-2024-11-20", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "Sure! If you’re testing, feel free to ask me anything or let me know how I can help. 😊", "refusal": null }, "logprobs": null, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 26, "completion_tokens": 24, "total_tokens": 50, "prompt_tokens_details": { "cached_tokens": 0, "audio_tokens": 0 }, "completion_tokens_details": { "reasoning_tokens": 0, "audio_tokens": 0, "accepted_prediction_tokens": 0, "rejected_prediction_tokens": 0 } }, "system_fingerprint": "fp_41926dc6d6" } ```` **Docker Container Logs:** Successful request logs: ```bash open-webui-74fcbff86c-l7pxb open-webui INFO: 10.42.1.0:0 - "POST /api/v1/chats/f1c26b73-34f8-40e2-aef2-a93e6431919a HTTP/1.1" 200 OK open-webui-74fcbff86c-l7pxb open-webui INFO: 10.42.1.0:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK open-webui-74fcbff86c-l7pxb open-webui INFO: 10.42.1.0:0 - "POST /api/v1/chats/f1c26b73-34f8-40e2-aef2-a93e6431919a HTTP/1.1" 200 OK open-webui-74fcbff86c-l7pxb open-webui INFO: 10.42.1.0:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK open-webui-74fcbff86c-l7pxb open-webui pipe:function_openai open-webui-74fcbff86c-l7pxb open-webui {'id': '81a607b0-6231-42d7-b7fc-4d0aaa7e2770', 'email': '(my email)', 'name': 'Sam', 'role': 'admin', 'valves': UserValves(OPENAI_API_KEY='sk-my-key')} open-webui-74fcbff86c-l7pxb open-webui {'stream': False, 'model': 'gpt-4o-2024-11-20', 'messages': [{'role': 'user', 'content': 'Test'}, {'role': 'assistant', 'content': 'Hi! How can I assist you today?'}, {'role': 'user', 'content': 'Test'}]} open-webui-74fcbff86c-l7pxb open-webui INFO: 10.42.1.0:0 - "POST /api/chat/completions HTTP/1.1" 200 OK open-webui-74fcbff86c-l7pxb open-webui INFO: 10.42.1.0:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK ``` **Screenshots/Screen Recordings (if applicable):** ![image](https://github.com/user-attachments/assets/d0e6da3c-9e9f-4271-8e3c-f0b7026c281a) ## Additional Information - This seems to also occur with the built-in openai connection. - Streaming responses _do_ work and _do_ complete (when the pipe is reconfigured to be synchronous). - My Open-WebUI is behind Traefik, but Traefik should handle websockets without any special configuration. ## Note If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!

GiteaMirror closed this issue

2025-11-11 15:22:30 -06:00

GiteaMirror commented

2025-11-11 15:22:30 -06:00

@samling commented on GitHub (Dec 26, 2024):

On some further reading it seems like that websocket error in the logs is related. I'll do some research and report back for anyone else experiencing anything similar.

@samling commented on GitHub (Dec 26, 2024): On some further reading it seems like that websocket error in the logs is related. I'll do some research and report back for anyone else experiencing anything similar.

GiteaMirror commented

2025-11-11 15:22:31 -06:00