github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-07-23 03:23:08 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[GH-ISSUE #22141] issue: Tokens dropped with streaming enabled #58309

New Issue
Closed
opened 2026-05-05 22:50:56 -05:00 by GiteaMirror · 39 comments
GiteaMirror commented 2026-05-05 22:50:56 -05:00
Owner
Copy Link
Copy Source

Originally created by @anselor on GitHub (Mar 2, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/22141

Check Existing Issues

  • I have searched for any existing and/or related issues.
  • I have searched for any existing and/or related discussions.
  • I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!).
  • I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

0.8.7

Ollama Version (if applicable)

No response

Operating System

ghcr.io/open-webui/open-webui:latest

Browser (if applicable)

Chrome Version 145.0.7632.117

Confirmation

  • I have read and followed all instructions in README.md.
  • I am using the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.
  • I have provided every relevant configuration, setting, and environment variable used in my setup.
  • I have clearly listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc).
  • I have documented step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation. My steps:
  • Start with the initial platform/version/OS and dependencies used,
  • Specify exact install/launch/configure commands,
  • List URLs visited, user input (incl. example values/emails/passwords if needed),
  • Describe all options and toggles enabled or changed,
  • Include any files or environmental changes,
  • Identify the expected and actual result at each stage,
  • Ensure any reasonably skilled user can follow and hit the same issue.

Expected Behavior

Streamed chat completions output does not drop tokens

Actual Behavior

Streamed chat completions output drops tokens. Especially noticeable with dates.

Steps to Reproduce

I'm running open webui 0.8.7 from docker (ghcr.io/open-webui/open-webui:latest)
I'm using a recent vllm nightlhy docker image: version 0.16.0rc2.dev496+g4a9c07a0a
With this model: cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit

When using chat completions with streaming enabled, tokens are randomly dropped.
If I disable streaming, tokens are not dropped but tool calling consistently fails.
If I switch to responses api with streaming enabled, tokens are not dropped but tool calling fails.
If I switch to responses API with streaming disabled, no results are displayed.

The dropped tokens happen most consistently when it needs to process dates. When I try streaming using curl I can see from the output that dates result in a long series of single-character tokens in a row which is probably what hits a bug in how open webui buffers the streaming input.
Command:

curl -N http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{"model": "cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit", "messages": [{"role": "user", "content": "List these dates precisely: January 15, 2024. February 28, 1999. March 3, 2026. December 31, 2000."}], "max_tokens": 512, "stream": true}'

Output:

data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"\n\n"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"1"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" January"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"1"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"5"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"2"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"0"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"2"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"4"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"\n"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"2"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" February"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"2"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"8"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"1"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"9"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"9"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"9"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"\n"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"3"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" March"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"3"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"2"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"0"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"2"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"6"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"\n"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"4"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" December"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"3"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"1"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"2"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"0"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"0"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"0"},"logprobs":null,"finish_reason":null,"token_ids":null}]}
data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}]}

Logs & Screenshots

Chat Completions with streaming enabled:
Image

2026-03-02 14:49:04.963 | INFO     | open_webui.routers.openai:get_all_models:487 - get_all_models()
2026-03-02 14:49:04.963 | INFO     | open_webui.routers.ollama:get_all_models:322 - get_all_models()
2026-03-02 14:49:04.969 | ERROR    | open_webui.routers.ollama:send_get_request:104 - Connection error: Cannot connect to host host.docker.internal:11434 ssl:default [Name or service not known]
2026-03-02 14:49:04.972 | ERROR    | open_webui.routers.ollama:send_get_request:104 - Connection error: Cannot connect to host host.docker.internal:11434 ssl:default [Name or service not known]
2026-03-02 14:49:04.983 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/models HTTP/1.1" 200
2026-03-02 14:49:07.869 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/v1/chats/ef081710-8bca-4196-9de8-273dcfc95a7b HTTP/1.1" 200
2026-03-02 14:49:07.886 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200
2026-03-02 14:49:07.905 | WARNING  | open_webui.utils.oauth:get_oauth_token:980 - No OAuth session found for user 13cadb86-fe97-4ad0-9fe4-29b27c017c75, session 6f44e795-a5e1-4d26-8c9a-92579009e7b2
2026-03-02 14:49:07.923 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/chat/completions HTTP/1.1" 200
2026-03-02 14:49:07.953 | WARNING  | open_webui.utils.oauth:get_oauth_token:980 - No OAuth session found for user 13cadb86-fe97-4ad0-9fe4-29b27c017c75, session 6f44e795-a5e1-4d26-8c9a-92579009e7b2
2026-03-02 14:49:07.960 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200
2026-03-02 14:49:08.828 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/chat/completed HTTP/1.1" 200
2026-03-02 14:49:08.858 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/v1/chats/ef081710-8bca-4196-9de8-273dcfc95a7b HTTP/1.1" 200
2026-03-02 14:49:08.875 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200
2026-03-02 14:49:27.506 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200
2026-03-02 14:49:27.520 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200
2026-03-02 14:49:29.677 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /_app/version.json HTTP/1.1" 200

Chat Completions streaming disabled:
Image

2026-03-02 14:51:29.681 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /_app/version.json HTTP/1.1" 200
2026-03-02 14:51:40.954 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/v1/chats/ef081710-8bca-4196-9de8-273dcfc95a7b HTTP/1.1" 200
2026-03-02 14:51:40.973 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200
2026-03-02 14:51:40.994 | WARNING  | open_webui.utils.oauth:get_oauth_token:980 - No OAuth session found for user 13cadb86-fe97-4ad0-9fe4-29b27c017c75, session 6f44e795-a5e1-4d26-8c9a-92579009e7b2
2026-03-02 14:51:41.012 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/chat/completions HTTP/1.1" 200
2026-03-02 14:51:41.034 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200
2026-03-02 14:51:42.460 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/chat/completed HTTP/1.1" 200
2026-03-02 14:51:42.492 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/v1/chats/ef081710-8bca-4196-9de8-273dcfc95a7b HTTP/1.1" 200
2026-03-02 14:51:42.510 | INFO     | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200

Responses API streaming enabled:
Image

Responses API streaming disabled:
Image

Additional Information

No response

Originally created by @anselor on GitHub (Mar 2, 2026). Original GitHub issue: https://github.com/open-webui/open-webui/issues/22141 ### Check Existing Issues - [x] I have searched for any existing and/or related issues. - [x] I have searched for any existing and/or related discussions. - [x] I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!). - [x] I am using the latest version of Open WebUI. ### Installation Method Docker ### Open WebUI Version 0.8.7 ### Ollama Version (if applicable) _No response_ ### Operating System ghcr.io/open-webui/open-webui:latest ### Browser (if applicable) Chrome Version 145.0.7632.117 ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have **provided every relevant configuration, setting, and environment variable used in my setup.** - [x] I have clearly **listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup** (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc). - [x] I have documented **step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation**. My steps: - Start with the initial platform/version/OS and dependencies used, - Specify exact install/launch/configure commands, - List URLs visited, user input (incl. example values/emails/passwords if needed), - Describe all options and toggles enabled or changed, - Include any files or environmental changes, - Identify the expected and actual result at each stage, - Ensure any reasonably skilled user can follow and hit the same issue. ### Expected Behavior Streamed chat completions output does not drop tokens ### Actual Behavior Streamed chat completions output drops tokens. Especially noticeable with dates. ### Steps to Reproduce I'm running open webui 0.8.7 from docker (ghcr.io/open-webui/open-webui:latest) I'm using a recent vllm nightlhy docker image: version 0.16.0rc2.dev496+g4a9c07a0a With this model: cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit When using chat completions with streaming enabled, tokens are randomly dropped. If I disable streaming, tokens are not dropped but tool calling consistently fails. If I switch to responses api with streaming enabled, tokens are not dropped but tool calling fails. If I switch to responses API with streaming disabled, no results are displayed. The dropped tokens happen most consistently when it needs to process dates. When I try streaming using curl I can see from the output that dates result in a long series of single-character tokens in a row which is probably what hits a bug in how open webui buffers the streaming input. Command: ``` curl -N http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{"model": "cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit", "messages": [{"role": "user", "content": "List these dates precisely: January 15, 2024. February 28, 1999. March 3, 2026. December 31, 2000."}], "max_tokens": 512, "stream": true}' ``` Output: ``` data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"\n\n"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"1"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" January"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"1"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"5"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"2"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"0"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"2"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"4"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"\n"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"2"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" February"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"2"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"8"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"1"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"9"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"9"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"9"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"\n"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"3"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" March"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"3"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"2"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"0"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"2"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"6"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"\n"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"4"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" December"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"3"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"1"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":","},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":" "},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"2"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"0"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"0"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{"content":"0"},"logprobs":null,"finish_reason":null,"token_ids":null}]} data: {"id":"chatcmpl-a5e91d98347f6309","object":"chat.completion.chunk","created":1772463447,"model":"cyankiwi/Qwen3.5-35B-A3B-AWQ-4bit","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop","stop_reason":null,"token_ids":null}]} ``` ### Logs & Screenshots Chat Completions with streaming enabled: <img width="1494" height="806" alt="Image" src="https://github.com/user-attachments/assets/eb99161a-2f2d-4c71-8a3a-563c712ec76b" /> ``` 2026-03-02 14:49:04.963 | INFO | open_webui.routers.openai:get_all_models:487 - get_all_models() 2026-03-02 14:49:04.963 | INFO | open_webui.routers.ollama:get_all_models:322 - get_all_models() 2026-03-02 14:49:04.969 | ERROR | open_webui.routers.ollama:send_get_request:104 - Connection error: Cannot connect to host host.docker.internal:11434 ssl:default [Name or service not known] 2026-03-02 14:49:04.972 | ERROR | open_webui.routers.ollama:send_get_request:104 - Connection error: Cannot connect to host host.docker.internal:11434 ssl:default [Name or service not known] 2026-03-02 14:49:04.983 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/models HTTP/1.1" 200 2026-03-02 14:49:07.869 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/v1/chats/ef081710-8bca-4196-9de8-273dcfc95a7b HTTP/1.1" 200 2026-03-02 14:49:07.886 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 2026-03-02 14:49:07.905 | WARNING | open_webui.utils.oauth:get_oauth_token:980 - No OAuth session found for user 13cadb86-fe97-4ad0-9fe4-29b27c017c75, session 6f44e795-a5e1-4d26-8c9a-92579009e7b2 2026-03-02 14:49:07.923 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/chat/completions HTTP/1.1" 200 2026-03-02 14:49:07.953 | WARNING | open_webui.utils.oauth:get_oauth_token:980 - No OAuth session found for user 13cadb86-fe97-4ad0-9fe4-29b27c017c75, session 6f44e795-a5e1-4d26-8c9a-92579009e7b2 2026-03-02 14:49:07.960 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 2026-03-02 14:49:08.828 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/chat/completed HTTP/1.1" 200 2026-03-02 14:49:08.858 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/v1/chats/ef081710-8bca-4196-9de8-273dcfc95a7b HTTP/1.1" 200 2026-03-02 14:49:08.875 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 2026-03-02 14:49:27.506 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 2026-03-02 14:49:27.520 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 2026-03-02 14:49:29.677 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /_app/version.json HTTP/1.1" 200 ``` Chat Completions streaming disabled: <img width="1509" height="508" alt="Image" src="https://github.com/user-attachments/assets/6e2a7a69-5519-438a-9f8a-03091feddde6" /> ``` 2026-03-02 14:51:29.681 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /_app/version.json HTTP/1.1" 200 2026-03-02 14:51:40.954 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/v1/chats/ef081710-8bca-4196-9de8-273dcfc95a7b HTTP/1.1" 200 2026-03-02 14:51:40.973 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 2026-03-02 14:51:40.994 | WARNING | open_webui.utils.oauth:get_oauth_token:980 - No OAuth session found for user 13cadb86-fe97-4ad0-9fe4-29b27c017c75, session 6f44e795-a5e1-4d26-8c9a-92579009e7b2 2026-03-02 14:51:41.012 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/chat/completions HTTP/1.1" 200 2026-03-02 14:51:41.034 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 2026-03-02 14:51:42.460 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/chat/completed HTTP/1.1" 200 2026-03-02 14:51:42.492 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "POST /api/v1/chats/ef081710-8bca-4196-9de8-273dcfc95a7b HTTP/1.1" 200 2026-03-02 14:51:42.510 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.152:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 ``` Responses API streaming enabled: <img width="1532" height="835" alt="Image" src="https://github.com/user-attachments/assets/2c1b8750-4fc8-48ba-96fd-dddf3c5bc93a" /> Responses API streaming disabled: <img width="1514" height="237" alt="Image" src="https://github.com/user-attachments/assets/5066ff4a-0c7c-45c0-9e33-6068c2511598" /> ### Additional Information _No response_
GiteaMirror added the bug label 2026-05-05 22:50:56 -05:00
GiteaMirror commented 2026-05-05 22:51:04 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

Cannot reproduce with API models from vertex ai, openrouter, openai and azure. Do you use nginx or any other reverse proxy for open webui? if so, what's the configuration there? Since you are saying you are using a nightly version of vLLM - does that maybe have a bug for your model during streaming?

<!-- gh-comment-id:3984953617 --> @Classic298 commented on GitHub (Mar 2, 2026): Cannot reproduce with API models from vertex ai, openrouter, openai and azure. Do you use nginx or any other reverse proxy for open webui? if so, what's the configuration there? Since you are saying you are using a _nightly version of vLLM_ - does that maybe have a bug for your model during streaming?
GiteaMirror commented 2026-05-05 22:51:06 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

Cannot reproduce with API models from vertex ai, openrouter, openai and azure. Do you use nginx or any other reverse proxy for open webui? if so, what's the configuration there? Since you are saying you are using a nightly version of vLLM - does that maybe have a bug for your model during streaming?

If you read the report, I demonstrated that streaming works fine using curl outside of open WebUI. This problem only occurs in Open WebUI streaming.

@Classic298

<!-- gh-comment-id:3984967546 --> @anselor commented on GitHub (Mar 2, 2026): > Cannot reproduce with API models from vertex ai, openrouter, openai and azure. Do you use nginx or any other reverse proxy for open webui? if so, what's the configuration there? Since you are saying you are using a _nightly version of vLLM_ - does that maybe have a bug for your model during streaming? If you read the report, I demonstrated that streaming works fine using curl outside of open WebUI. This problem only occurs in Open WebUI streaming. @Classic298
GiteaMirror commented 2026-05-05 22:51:07 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

@anselor there might still be a difference in HOW vLLM streams the tokens.
Indubitably streaming works elsewhere, but if the way vLLM streams tokens changed then we have the culprit here.

I try MiniMax (which has notoriously weird streaming behaviour) and openrouter, vertex ai, anthropic, and others and streaming works fine there. Maybe you can try a stable vLLM version and tell us if the issue persists there?

<!-- gh-comment-id:3985002689 --> @Classic298 commented on GitHub (Mar 2, 2026): @anselor there might still be a difference in HOW vLLM streams the tokens. Indubitably streaming works elsewhere, but if the way vLLM streams tokens changed then we have the culprit here. I try MiniMax (which has notoriously weird streaming behaviour) and openrouter, vertex ai, anthropic, and others and streaming works fine there. Maybe you can try a stable vLLM version and tell us if the issue persists there?
GiteaMirror commented 2026-05-05 22:51:08 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

Chat completions is chat completions. The If the problem were in vllm then it would show up consistently with the same protocol.
@Classic298

Here's the situation:

  • Same vLLM instance, same model, same streaming
  • curl to vLLM Chat Completions + streaming endpoint: correct output
  • Open WebUI Chat Completions + streaming: drops characters
  • Open WebUI Responses API + streaming: correct output
<!-- gh-comment-id:3985039046 --> @anselor commented on GitHub (Mar 2, 2026): Chat completions is chat completions. The If the problem were in vllm then it would show up consistently with the same protocol. @Classic298 Here's the situation: - Same vLLM instance, same model, same streaming - curl to vLLM Chat Completions + streaming endpoint: correct output - Open WebUI Chat Completions + streaming: drops characters - Open WebUI Responses API + streaming: correct output
GiteaMirror commented 2026-05-05 22:51:09 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

chat completions is not chat completions.

We have had cases not too long ago (in fact just 14 days ago) where minimax, also "just chat completions is chat completions" acted up weirdly because their streaming is weird.

This is fixed now

Chat completions is not chat completions if the way the tokens are streamed is weird (buffered tokens for example)

Can you please tell us if the same happens on a non-nightly version? Thanks

<!-- gh-comment-id:3985071427 --> @Classic298 commented on GitHub (Mar 2, 2026): chat completions is not chat completions. We have had cases not too long ago (in fact just 14 days ago) where minimax, also "just chat completions is chat completions" acted up weirdly because their streaming is weird. This is fixed now Chat completions is not chat completions if the way the tokens are streamed is weird (buffered tokens for example) Can you please tell us if the same happens on a non-nightly version? Thanks
GiteaMirror commented 2026-05-05 22:51:09 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

I just set up Librechat and streaming works fine. If streaming works fine through multiple other methods and only fails in open webUI it really points to open webUI having a buggy implementation and not vllm "streams weird"

Honestly, "streaming is weird" just sounds like making excuses for a buggy implementation.

<!-- gh-comment-id:3985553866 --> @anselor commented on GitHub (Mar 2, 2026): I just set up Librechat and streaming works fine. If streaming works fine through multiple other methods and only fails in open webUI it really points to open webUI having a buggy implementation and not vllm "streams weird" Honestly, "streaming is weird" just sounds like making excuses for a buggy implementation.
GiteaMirror commented 2026-05-05 22:51:10 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

If multiple other providers work without issue, then it seems vLLM is doing something different to other providers.

And streaming is weird is not an excuse 😁
MiniMax genuinely has weird streaming that is different to how OpenAI does it.

Let me find out the root cause back then...

Ah right found it.

So MiniMax did not send a closing </think> token (and still doesn't) when doing tool calls. I opened up PRs for this to fix it, tim ended up fixing it himself.

this caused token duplication and the entire chat history appeared 10 times if there were 9 tool calls for example.

This is weird streaming behaviour. Because MiniMax is too lazy to send a closing thinking token.

and if vLLM does anything "weird" then that's a vLLM issue (mostly).
The fact i cannot replicate your issue with neither OpenAI, openrouter, litellm, vertex ai, anthropic or azure should be proof enough that vLLM is doing something different here.

Did you try a stable version of vLLM yet?

<!-- gh-comment-id:3985582689 --> @Classic298 commented on GitHub (Mar 2, 2026): If multiple other providers work without issue, then it seems vLLM is doing something different to other providers. And streaming is weird is not an excuse 😁 MiniMax genuinely has weird streaming that is different to how OpenAI does it. Let me find out the root cause back then... Ah right found it. So MiniMax did not send a closing `</think>` token (and still doesn't) when doing tool calls. I opened up PRs for this to fix it, tim ended up fixing it himself. this caused token duplication and the entire chat history appeared 10 times if there were 9 tool calls for example. This is weird streaming behaviour. Because MiniMax is too lazy to send a closing thinking token. and if vLLM does anything "weird" then that's a vLLM issue (mostly). The fact i cannot replicate your issue with neither OpenAI, openrouter, litellm, vertex ai, anthropic or azure should be proof enough that vLLM is doing something different here. Did you try a stable version of vLLM yet?
GiteaMirror commented 2026-05-05 22:51:12 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

Just for reference @anselor this one is a bug report in vLLM and looks just all to similar to what you are describing.

https://github.com/vllm-project/vllm/issues/23335

<!-- gh-comment-id:3985611825 --> @Classic298 commented on GitHub (Mar 2, 2026): Just for reference @anselor this one is a bug report in vLLM and looks just all to similar to what you are describing. https://github.com/vllm-project/vllm/issues/23335
GiteaMirror commented 2026-05-05 22:51:12 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

same here

https://github.com/vllm-project/vllm/issues/15545

<!-- gh-comment-id:3985620863 --> @Classic298 commented on GitHub (Mar 2, 2026): same here https://github.com/vllm-project/vllm/issues/15545
GiteaMirror commented 2026-05-05 22:51:13 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

a user described setting the temperature to 0 may help

<!-- gh-comment-id:3985622011 --> @Classic298 commented on GitHub (Mar 2, 2026): a user described setting the temperature to 0 may help
GiteaMirror commented 2026-05-05 22:51:14 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

I'm aware of the gpt-oss specific bug and with that it happens consistently through all clients. It doesn't just happen inside of open webui.

<!-- gh-comment-id:3985624853 --> @anselor commented on GitHub (Mar 2, 2026): I'm aware of the gpt-oss specific bug and with that it happens consistently through all clients. It doesn't just happen inside of open webui.
GiteaMirror commented 2026-05-05 22:51:17 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

@anselor can you please tell me if you have open webui or vllm or both behind any reverse proxy? if yes, which, if yes, which config?

<!-- gh-comment-id:3985644780 --> @Classic298 commented on GitHub (Mar 2, 2026): @anselor can you please tell me if you have open webui or vllm or both behind any reverse proxy? if yes, which, if yes, which config?
GiteaMirror commented 2026-05-05 22:51:19 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

I use traefik for both.

<!-- gh-comment-id:3985654944 --> @anselor commented on GitHub (Mar 2, 2026): I use traefik for both.
GiteaMirror commented 2026-05-05 22:51:25 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

ok and again: what configs please? specifically re: cache, buffering, TCP settings.

Unfortunately i have no traefik tutorial to share, but there are a LOT of pitfalls with for example NGINX, which, when configured incorrectly can lead to such behaviour during streaming

https://docs.openwebui.com/reference/https/nginx

<!-- gh-comment-id:3985669312 --> @Classic298 commented on GitHub (Mar 2, 2026): ok and again: what configs please? specifically re: cache, buffering, TCP settings. Unfortunately i have no traefik tutorial to share, but there are a LOT of pitfalls with for example NGINX, which, when configured incorrectly can lead to such behaviour during streaming https://docs.openwebui.com/reference/https/nginx
GiteaMirror commented 2026-05-05 22:51:28 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

whats your traefik config?

<!-- gh-comment-id:3985701896 --> @Classic298 commented on GitHub (Mar 2, 2026): whats your traefik config?
GiteaMirror commented 2026-05-05 22:51:30 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

@anselor this is quite likely a reverse proxy buffering issue with Traefik. Here's what's happening under the hood:

When vLLM streams single-character tokens rapidly (like dates: "2", "0", "2", "4"), each SSE event is very small (~150 bytes). Traefik can buffer multiple of these tiny events into a single TCP segment before forwarding them. When Open WebUI's backend receives a chunk containing multiple SSE events at once, the streaming parser treats the entire chunk as a single line and attempts to JSON-parse it, which fails because it contains more than one event. The parser then silently drops the entire chunk via a continue statement in the exception handler.

This explains every observation in your report:

  • curl to vLLM works: no proxy in the path, events arrive individually
  • Chat Completions + streaming drops tokens: tiny single-char delta events get batched by Traefik
  • Responses API + streaming works: events carry more metadata, are larger, and less likely to get batched together in a single TCP segment
  • Dates are most affected: they produce the longest sequences of single-character tokens, maximizing the chance of proxy batching

This is the same class of issue we document for Nginx (proxy_buffering off, X-Accel-Buffering header, etc.). Traefik has equivalent settings.

Could you try the following:

  1. Add response buffering disable to your Traefik config for the Open WebUI and/or vLLM services. In Traefik, this is typically done via middleware or service-level configuration:

traefik.http.middlewares.no-buffer.buffering.maxResponseBodyBytes: "0"

Or if you're using file-based config:

   http:
     middlewares:
       streaming:
         buffering:
           maxResponseBodyBytes: 0
           retryExpression: "IsNetworkError()"

  1. Alternatively, as a quick test, try accessing Open WebUI directly (bypassing Traefik and also adding vLLM directly without Traefik) and see if the token dropping stops.

  2. You can also try setting the environment variable CHAT_STREAM_RESPONSE_CHUNK_MAX_BUFFER_SIZE (e.g. to 16384) on your Open WebUI container. This activates a built-in SSE line splitter that properly handles buffered/batched events by splitting on newline boundaries before processing.

Let us know what Traefik config you're using and whether any of the above helps. We've seen this pattern many times with various reverse proxies and it's always been a proxy buffering misconfiguration.

<!-- gh-comment-id:3985811711 --> @Classic298 commented on GitHub (Mar 2, 2026): @anselor this is quite likely a reverse proxy buffering issue with Traefik. Here's what's happening under the hood: When vLLM streams single-character tokens rapidly (like dates: "2", "0", "2", "4"), each SSE event is very small (~150 bytes). Traefik can buffer multiple of these tiny events into a single TCP segment before forwarding them. When Open WebUI's backend receives a chunk containing multiple SSE events at once, the streaming parser treats the entire chunk as a single line and attempts to JSON-parse it, which fails because it contains more than one event. The parser then silently drops the entire chunk via a continue statement in the exception handler. This explains every observation in your report: - curl to vLLM works: no proxy in the path, events arrive individually - Chat Completions + streaming drops tokens: tiny single-char delta events get batched by Traefik - Responses API + streaming works: events carry more metadata, are larger, and less likely to get batched together in a single TCP segment - Dates are most affected: they produce the longest sequences of single-character tokens, maximizing the chance of proxy batching This is the same class of issue we document for Nginx (proxy_buffering off, X-Accel-Buffering header, etc.). Traefik has equivalent settings. Could you try the following: 1. Add response buffering disable to your Traefik config for the Open WebUI and/or vLLM services. In Traefik, this is typically done via middleware or service-level configuration: ` traefik.http.middlewares.no-buffer.buffering.maxResponseBodyBytes: "0"` Or if you're using file-based config: ``` http: middlewares: streaming: buffering: maxResponseBodyBytes: 0 retryExpression: "IsNetworkError()" ``` 2. Alternatively, as a quick test, **try accessing Open WebUI directly (bypassing Traefik and also adding vLLM directly without Traefik) and see if the token dropping stops.** 3. You can also try setting the environment variable CHAT_STREAM_RESPONSE_CHUNK_MAX_BUFFER_SIZE (e.g. to 16384) on your Open WebUI container. This activates a built-in SSE line splitter that properly handles buffered/batched events by splitting on newline boundaries before processing. Let us know what Traefik config you're using and whether any of the above helps. We've seen this pattern many times with various reverse proxies and it's always been a proxy buffering misconfiguration.
GiteaMirror commented 2026-05-05 22:51:31 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

curl to vllm through the proxy yields the same correct streaming output. I tried it.

I just added the following to vllm's settings - no change in behavior:

      - "traefik.http.middlewares.qwen-nobuffer.buffering.maxRequestBodyBytes=0"
      - "traefik.http.middlewares.qwen-nobuffer.buffering.maxResponseBodyBytes=0"
      - "traefik.http.middlewares.qwen-nobuffer.buffering.retryExpression="
      - "traefik.http.middlewares.qwen-headers.headers.customResponseHeaders.X-Accel-Buffering=no"
      - "traefik.http.routers.qwen35.middlewares=qwen-nobuffer,qwen-headers"

I added the following to open webui's settings, also no improvement in behavior:

      # Disable buffering for streaming
      - "traefik.http.middlewares.openwebui-nobuffer.buffering.maxRequestBodyBytes=0"
      - "traefik.http.middlewares.openwebui-nobuffer.buffering.maxResponseBodyBytes=0"
      - "traefik.http.middlewares.openwebui-nobuffer.buffering.retryExpression="
      - "traefik.http.routers.openwebui.middlewares=openwebui-nobuffer"
      # Headers for SSE
      - "traefik.http.middlewares.openwebui-headers.headers.customResponseHeaders.X-Accel-Buffering=no"
      - "traefik.http.routers.openwebui.middlewares=openwebui-nobuffer,openwebui-headers"
<!-- gh-comment-id:3985847875 --> @anselor commented on GitHub (Mar 2, 2026): curl to vllm through the proxy yields the same correct streaming output. I tried it. I just added the following to vllm's settings - no change in behavior: ``` - "traefik.http.middlewares.qwen-nobuffer.buffering.maxRequestBodyBytes=0" - "traefik.http.middlewares.qwen-nobuffer.buffering.maxResponseBodyBytes=0" - "traefik.http.middlewares.qwen-nobuffer.buffering.retryExpression=" - "traefik.http.middlewares.qwen-headers.headers.customResponseHeaders.X-Accel-Buffering=no" - "traefik.http.routers.qwen35.middlewares=qwen-nobuffer,qwen-headers" ``` I added the following to open webui's settings, also no improvement in behavior: ``` # Disable buffering for streaming - "traefik.http.middlewares.openwebui-nobuffer.buffering.maxRequestBodyBytes=0" - "traefik.http.middlewares.openwebui-nobuffer.buffering.maxResponseBodyBytes=0" - "traefik.http.middlewares.openwebui-nobuffer.buffering.retryExpression=" - "traefik.http.routers.openwebui.middlewares=openwebui-nobuffer" # Headers for SSE - "traefik.http.middlewares.openwebui-headers.headers.customResponseHeaders.X-Accel-Buffering=no" - "traefik.http.routers.openwebui.middlewares=openwebui-nobuffer,openwebui-headers" ```
GiteaMirror commented 2026-05-05 22:51:32 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

Tried connecting openwebui directly to vllm on port 8000, bypassing traefik, same problem.

<!-- gh-comment-id:3985869581 --> @anselor commented on GitHub (Mar 2, 2026): Tried connecting openwebui directly to vllm on port 8000, bypassing traefik, same problem.
GiteaMirror commented 2026-05-05 22:51:33 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

@anselor if you add this to open webui? does anything change

CHAT_STREAM_RESPONSE_CHUNK_MAX_BUFFER_SIZE=16384

<!-- gh-comment-id:3985879445 --> @Classic298 commented on GitHub (Mar 2, 2026): @anselor if you add this to open webui? does anything change CHAT_STREAM_RESPONSE_CHUNK_MAX_BUFFER_SIZE=16384
GiteaMirror commented 2026-05-05 22:51:34 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

Just tried it. No improvement.

<!-- gh-comment-id:3985899957 --> @anselor commented on GitHub (Mar 2, 2026): Just tried it. No improvement.
GiteaMirror commented 2026-05-05 22:51:34 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

Open WebUI
Image

LibreChat (both directly connected to vllm on port 8000):
Image

<!-- gh-comment-id:3985908563 --> @anselor commented on GitHub (Mar 2, 2026): Open WebUI <img width="1501" height="882" alt="Image" src="https://github.com/user-attachments/assets/22c7cd5e-0765-48c7-9202-e2dc5c3a14b9" /> LibreChat (both directly connected to vllm on port 8000): <img width="1270" height="602" alt="Image" src="https://github.com/user-attachments/assets/3bdeef25-76af-483a-9054-44e7e849eb84" />
GiteaMirror commented 2026-05-05 22:51:34 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

@anselor thanks for testing. Since we've ruled out proxy buffering, could you help us narrow down whether the issue is in the backend processing or frontend display?

Could you open browser DevTools (F12) → Console, reproduce the issue, and check the console output? The chat completion events are logged there (you'll see objects with a "content" field). We need to know if the "content" values arriving via socket already have missing characters, or if the full text arrives correctly and something goes wrong during rendering.

If you can share the last few logged event objects (specifically the "content" field values) right before the response finishes, that would narrow this down significantly.

<!-- gh-comment-id:3985945799 --> @Classic298 commented on GitHub (Mar 2, 2026): @anselor thanks for testing. Since we've ruled out proxy buffering, could you help us narrow down whether the issue is in the backend processing or frontend display? Could you open browser DevTools (F12) → Console, reproduce the issue, and check the console output? The chat completion events are logged there (you'll see objects with a "content" field). We need to know if the "content" values arriving via socket already have missing characters, or if the full text arrives correctly and something goes wrong during rendering. If you can share the last few logged event objects (specifically the "content" field values) right before the response finishes, that would narrow this down significantly.
GiteaMirror commented 2026-05-05 22:51:35 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

I'm not a web developer so I don't really know the ins and outs of the debug console. This is what I see:

Image
<!-- gh-comment-id:3986058638 --> @anselor commented on GitHub (Mar 2, 2026): I'm not a web developer so I don't really know the ins and outs of the debug console. This is what I see: <img width="2874" height="329" alt="Image" src="https://github.com/user-attachments/assets/2a9d3f23-8683-43b9-945c-f5bb48a39a95" />
GiteaMirror commented 2026-05-05 22:51:36 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

Okay, that is the console

Switch to the network tab (away from the console) and filter by "completion" and then look at the content of that

<!-- gh-comment-id:3986067231 --> @Classic298 commented on GitHub (Mar 2, 2026): Okay, that is the console Switch to the network tab (away from the console) and filter by "completion" and then look at the content of that
GiteaMirror commented 2026-05-05 22:51:37 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

completions:

[
  {
    "role": "user",
    "content": "List these dates precisely: January 15, 2024. February 28, 1999. March 3, 2026. December 31, 2000."
  }
]

completed:

[
  {
    "id": "0c05a317-abb4-4574-9936-34b431d8b9f6",
    "role": "user",
    "content": "List these dates precisely: January 15, 2024. February 28, 1999. March 3, 2026. December 31, 2000.",
    "timestamp": 1772475614
  },
  {
    "id": "da08602b-61c2-4a33-8ad4-681188ede6df",
    "role": "assistant",
    "content": "<details type=\"reasoning\" done=\"true\" duration=\"0\">\n<summary>Thought for 0 seconds</summary>\n&gt; The user is asking me to list these dates precisely. They&#x27;ve given me four dates:\n&gt; 1. January 15, 2024\n&gt; 2. February 28, 1999\n&gt; 3. March 3, 2026\n&gt; 4. December 31, 2000\n&gt; \n&gt; This is a straightforward request - they just want me to list these dates. I don&#x27;t need to use any tools for this, as it&#x27;s just a simple text response. I should format these dates clearly.\n</details>\nHere are the dates listed precisely:\n\n1. **January 15, 2024**\n2. **February 28, 19**\n3. **March 3, 2026**\n4. **December 31, 20**",
    "timestamp": 1772475614
  }
]
<!-- gh-comment-id:3986105832 --> @anselor commented on GitHub (Mar 2, 2026): completions: ``` [ { "role": "user", "content": "List these dates precisely: January 15, 2024. February 28, 1999. March 3, 2026. December 31, 2000." } ] ``` completed: ``` [ { "id": "0c05a317-abb4-4574-9936-34b431d8b9f6", "role": "user", "content": "List these dates precisely: January 15, 2024. February 28, 1999. March 3, 2026. December 31, 2000.", "timestamp": 1772475614 }, { "id": "da08602b-61c2-4a33-8ad4-681188ede6df", "role": "assistant", "content": "<details type=\"reasoning\" done=\"true\" duration=\"0\">\n<summary>Thought for 0 seconds</summary>\n&gt; The user is asking me to list these dates precisely. They&#x27;ve given me four dates:\n&gt; 1. January 15, 2024\n&gt; 2. February 28, 1999\n&gt; 3. March 3, 2026\n&gt; 4. December 31, 2000\n&gt; \n&gt; This is a straightforward request - they just want me to list these dates. I don&#x27;t need to use any tools for this, as it&#x27;s just a simple text response. I should format these dates clearly.\n</details>\nHere are the dates listed precisely:\n\n1. **January 15, 2024**\n2. **February 28, 19**\n3. **March 3, 2026**\n4. **December 31, 20**", "timestamp": 1772475614 } ] ```
GiteaMirror commented 2026-05-05 22:51:38 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

@anselor Thanks for this!

the completed response confirms the tokens are already lost in the backend before they reach your browser. The pattern is interesting - "1999" becomes "19" and "2000" becomes "20", suggesting vLLM tokenizes these as two tokens (e.g. "19"+"99", "20"+"00") and the second token in each pair is being dropped during stream processing.

Could you restart Open WebUI with GLOBAL_LOG_LEVEL=DEBUG, and also with this temporary environment variable to enable more granular stream logging:

ENABLE_REALTIME_CHAT_SAVE=true

Then reproduce the issue and share the backend logs showing the full streaming activity (try to have the AI answer concisely otherwise you will see A LOT of logs (you will see a lot of logs anyways, but still))

This will help pinpoint exactly where in the stream processing pipeline the tokens are being lost.

Also - quick question: does this ONLY happen with the Qwen 3.5 model, or have you tested other models on the same vLLM instance? Specifically, does it happen with a model that doesn't use thinking/reasoning tags? That would help narrow down whether the tag detection code is involved.

Please also test with other models!

<!-- gh-comment-id:3986148021 --> @Classic298 commented on GitHub (Mar 2, 2026): @anselor Thanks for this! the completed response confirms the tokens are already lost in the backend before they reach your browser. The pattern is interesting - "1999" becomes "19" and "2000" becomes "20", suggesting vLLM tokenizes these as two tokens (e.g. "19"+"99", "20"+"00") and the second token in each pair is being dropped during stream processing. Could you restart Open WebUI with GLOBAL_LOG_LEVEL=DEBUG, and also with this temporary environment variable to enable more granular stream logging: ENABLE_REALTIME_CHAT_SAVE=true Then reproduce the issue and share the backend logs showing the full streaming activity (try to have the AI answer concisely otherwise you will see A LOT of logs (you will see a lot of logs anyways, but still)) This will help pinpoint exactly where in the stream processing pipeline the tokens are being lost. Also - quick question: does this ONLY happen with the Qwen 3.5 model, or have you tested other models on the same vLLM instance? Specifically, does it happen with a model that doesn't use thinking/reasoning tags? That would help narrow down whether the tag detection code is involved. **Please also test with other models!**
GiteaMirror commented 2026-05-05 22:51:39 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

I didn't see any useful logs out of open webui so instead I logged the generation output from vllm:

(APIServer pid=1) INFO:     Started server process [1]
(APIServer pid=1) INFO:     Waiting for application startup.
(APIServer pid=1) INFO:     Application startup complete.
(APIServer pid=1) INFO 03-02 18:46:45 [qwen3coder_tool_parser.py:85] vLLM Successfully import tool parser Qwen3CoderToolParser !
(APIServer pid=1) INFO 03-02 18:46:45 [logger.py:49] Received request chatcmpl-b1f7fd6e394e173b: params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=27750, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, structured_outputs=None, extra_args=None), lora_request: None.
(APIServer pid=1) INFO:     192.168.10.83:38142 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO 03-02 18:46:45 [qwen3coder_tool_parser.py:85] vLLM Successfully import tool parser Qwen3CoderToolParser !
(APIServer pid=1) INFO 03-02 18:46:45 [async_llm.py:419] Added request chatcmpl-b1f7fd6e394e173b-a8bd997f.
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: The]', output_token_ids: [760], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  user]', output_token_ids: [1156], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  is]', output_token_ids: [369], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  asking]', output_token_ids: [9859], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  me]', output_token_ids: [728], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  to]', output_token_ids: [310], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  list]', output_token_ids: [1103], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  specific]', output_token_ids: [3050], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  dates]', output_token_ids: [12348], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  precisely]', output_token_ids: [22898], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: .]', output_token_ids: [13], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  They]', output_token_ids: [2302], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: "[reasoning: 've]", output_token_ids: [2908], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  given]', output_token_ids: [2574], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  me]', output_token_ids: [728], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: :]', output_token_ids: [25], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: \n]', output_token_ids: [198], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 1]', output_token_ids: [16], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: .]', output_token_ids: [13], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  January]', output_token_ids: [5869], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  ]', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 1]', output_token_ids: [16], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 5]', output_token_ids: [20], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ,]', output_token_ids: [11], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  ]', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 2]', output_token_ids: [17], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 0]', output_token_ids: [15], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 2]', output_token_ids: [17], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 4]', output_token_ids: [19], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: \n]', output_token_ids: [198], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 2]', output_token_ids: [17], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: .]', output_token_ids: [13], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  February]', output_token_ids: [7165], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  ]', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 2]', output_token_ids: [17], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 8]', output_token_ids: [23], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ,]', output_token_ids: [11], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  ]', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 1]', output_token_ids: [16], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 9]', output_token_ids: [24], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 9]', output_token_ids: [24], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 9]', output_token_ids: [24], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: \n]', output_token_ids: [198], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 3]', output_token_ids: [18], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: .]', output_token_ids: [13], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  March]', output_token_ids: [5297], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  ]', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 3]', output_token_ids: [18], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ,]', output_token_ids: [11], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  ]', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 2]', output_token_ids: [17], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 0]', output_token_ids: [15], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 2]', output_token_ids: [17], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 6]', output_token_ids: [21], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: \n]', output_token_ids: [198], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 4]', output_token_ids: [19], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: .]', output_token_ids: [13], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  December]', output_token_ids: [6443], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  ]', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 3]', output_token_ids: [18], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 1]', output_token_ids: [16], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ,]', output_token_ids: [11], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  ]', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 2]', output_token_ids: [17], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 0]', output_token_ids: [15], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 0]', output_token_ids: [15], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 0]', output_token_ids: [15], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: \n\n]', output_token_ids: [271], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: This]', output_token_ids: [1919], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  is]', output_token_ids: [369], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  a]', output_token_ids: [264], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  straightforward]', output_token_ids: [29350], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  request]', output_token_ids: [1622], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  -]', output_token_ids: [471], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  I]', output_token_ids: [353], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  don]', output_token_ids: [1459], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: "[reasoning: 't]", output_token_ids: [914], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  need]', output_token_ids: [1144], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  to]', output_token_ids: [310], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  use]', output_token_ids: [958], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  any]', output_token_ids: [866], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  tools]', output_token_ids: [7141], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  for]', output_token_ids: [364], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  this]', output_token_ids: [411], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: .]', output_token_ids: [13], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  I]', output_token_ids: [353], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  just]', output_token_ids: [1066], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  need]', output_token_ids: [1144], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  to]', output_token_ids: [310], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  format]', output_token_ids: [3443], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  and]', output_token_ids: [321], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  present]', output_token_ids: [2945], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  these]', output_token_ids: [1439], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  dates]', output_token_ids: [12348], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning:  clearly]', output_token_ids: [9077], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: .]', output_token_ids: [13], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: \n]', output_token_ids: [198], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '\n\n', output_token_ids: [271], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: 'Here', output_token_ids: [8160], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' are', output_token_ids: [513], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' the', output_token_ids: [279], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' dates', output_token_ids: [12348], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' you', output_token_ids: [488], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' requested', output_token_ids: [10897], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ',', output_token_ids: [11], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' listed', output_token_ids: [9711], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' precisely', output_token_ids: [22898], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ':', output_token_ids: [25], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '\n\n', output_token_ids: [271], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '1', output_token_ids: [16], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '.', output_token_ids: [13], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' **', output_token_ids: [2972], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: 'January', output_token_ids: [31164], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '1', output_token_ids: [16], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '5', output_token_ids: [20], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ',', output_token_ids: [11], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '2', output_token_ids: [17], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '0', output_token_ids: [15], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '2', output_token_ids: [17], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '4', output_token_ids: [19], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '**', output_token_ids: [332], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '\n', output_token_ids: [198], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '2', output_token_ids: [17], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '.', output_token_ids: [13], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' **', output_token_ids: [2972], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: 'February', output_token_ids: [31691], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '2', output_token_ids: [17], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '8', output_token_ids: [23], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ',', output_token_ids: [11], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '1', output_token_ids: [16], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '9', output_token_ids: [24], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '9', output_token_ids: [24], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '9', output_token_ids: [24], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '**', output_token_ids: [332], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '\n', output_token_ids: [198], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '3', output_token_ids: [18], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '.', output_token_ids: [13], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' **', output_token_ids: [2972], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: 'March', output_token_ids: [26631], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '3', output_token_ids: [18], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ',', output_token_ids: [11], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '2', output_token_ids: [17], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '0', output_token_ids: [15], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '2', output_token_ids: [17], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '6', output_token_ids: [21], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '**', output_token_ids: [332], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '\n', output_token_ids: [198], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '4', output_token_ids: [19], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '.', output_token_ids: [13], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' **', output_token_ids: [2972], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: 'December', output_token_ids: [31087], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '3', output_token_ids: [18], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '1', output_token_ids: [16], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ',', output_token_ids: [11], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '2', output_token_ids: [17], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '0', output_token_ids: [15], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '0', output_token_ids: [15], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '0', output_token_ids: [15], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '**', output_token_ids: [332], finish_reason: None
(APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming complete): output: '\n\nHere are the dates you requested, listed precisely:\n\n1. **January 15, 2024**\n2. **February 28, 1999**\n3. **March 3, 2026**\n4. **December 31, 2000**', output_token_ids: None, finish_reason: streaming_complete
(APIServer pid=1) INFO 03-02 18:47:19 [loggers.py:259] Engine 000: Avg prompt throughput: 501.8 tokens/s, Avg generation throughput: 16.9 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=1) INFO 03-02 18:47:20 [logger.py:49] Received request chatcmpl-b83927f13171723c: params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=32424, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, structured_outputs=None, extra_args=None), lora_request: None.
Image
<!-- gh-comment-id:3986216025 --> @anselor commented on GitHub (Mar 2, 2026): I didn't see any useful logs out of open webui so instead I logged the generation output from vllm: ``` (APIServer pid=1) INFO: Started server process [1] (APIServer pid=1) INFO: Waiting for application startup. (APIServer pid=1) INFO: Application startup complete. (APIServer pid=1) INFO 03-02 18:46:45 [qwen3coder_tool_parser.py:85] vLLM Successfully import tool parser Qwen3CoderToolParser ! (APIServer pid=1) INFO 03-02 18:46:45 [logger.py:49] Received request chatcmpl-b1f7fd6e394e173b: params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=27750, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, structured_outputs=None, extra_args=None), lora_request: None. (APIServer pid=1) INFO: 192.168.10.83:38142 - "POST /v1/chat/completions HTTP/1.1" 200 OK (APIServer pid=1) INFO 03-02 18:46:45 [qwen3coder_tool_parser.py:85] vLLM Successfully import tool parser Qwen3CoderToolParser ! (APIServer pid=1) INFO 03-02 18:46:45 [async_llm.py:419] Added request chatcmpl-b1f7fd6e394e173b-a8bd997f. (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: The]', output_token_ids: [760], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: user]', output_token_ids: [1156], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: is]', output_token_ids: [369], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: asking]', output_token_ids: [9859], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: me]', output_token_ids: [728], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: to]', output_token_ids: [310], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: list]', output_token_ids: [1103], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: specific]', output_token_ids: [3050], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: dates]', output_token_ids: [12348], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: precisely]', output_token_ids: [22898], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: .]', output_token_ids: [13], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: They]', output_token_ids: [2302], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: "[reasoning: 've]", output_token_ids: [2908], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: given]', output_token_ids: [2574], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: me]', output_token_ids: [728], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: :]', output_token_ids: [25], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: \n]', output_token_ids: [198], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 1]', output_token_ids: [16], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: .]', output_token_ids: [13], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: January]', output_token_ids: [5869], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ]', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 1]', output_token_ids: [16], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 5]', output_token_ids: [20], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ,]', output_token_ids: [11], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ]', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 2]', output_token_ids: [17], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 0]', output_token_ids: [15], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 2]', output_token_ids: [17], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 4]', output_token_ids: [19], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: \n]', output_token_ids: [198], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 2]', output_token_ids: [17], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: .]', output_token_ids: [13], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: February]', output_token_ids: [7165], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ]', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 2]', output_token_ids: [17], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 8]', output_token_ids: [23], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ,]', output_token_ids: [11], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ]', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 1]', output_token_ids: [16], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 9]', output_token_ids: [24], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 9]', output_token_ids: [24], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 9]', output_token_ids: [24], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: \n]', output_token_ids: [198], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 3]', output_token_ids: [18], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: .]', output_token_ids: [13], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: March]', output_token_ids: [5297], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ]', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 3]', output_token_ids: [18], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ,]', output_token_ids: [11], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ]', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 2]', output_token_ids: [17], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 0]', output_token_ids: [15], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 2]', output_token_ids: [17], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 6]', output_token_ids: [21], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: \n]', output_token_ids: [198], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 4]', output_token_ids: [19], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: .]', output_token_ids: [13], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: December]', output_token_ids: [6443], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ]', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 3]', output_token_ids: [18], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 1]', output_token_ids: [16], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ,]', output_token_ids: [11], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: ]', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 2]', output_token_ids: [17], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 0]', output_token_ids: [15], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 0]', output_token_ids: [15], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: 0]', output_token_ids: [15], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: \n\n]', output_token_ids: [271], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: This]', output_token_ids: [1919], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: is]', output_token_ids: [369], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: a]', output_token_ids: [264], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: straightforward]', output_token_ids: [29350], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: request]', output_token_ids: [1622], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: -]', output_token_ids: [471], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: I]', output_token_ids: [353], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: don]', output_token_ids: [1459], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: "[reasoning: 't]", output_token_ids: [914], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: need]', output_token_ids: [1144], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: to]', output_token_ids: [310], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: use]', output_token_ids: [958], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: any]', output_token_ids: [866], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: tools]', output_token_ids: [7141], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: for]', output_token_ids: [364], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: this]', output_token_ids: [411], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: .]', output_token_ids: [13], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: I]', output_token_ids: [353], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: just]', output_token_ids: [1066], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: need]', output_token_ids: [1144], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: to]', output_token_ids: [310], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: format]', output_token_ids: [3443], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: and]', output_token_ids: [321], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: present]', output_token_ids: [2945], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: these]', output_token_ids: [1439], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: dates]', output_token_ids: [12348], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: clearly]', output_token_ids: [9077], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: .]', output_token_ids: [13], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '[reasoning: \n]', output_token_ids: [198], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '\n\n', output_token_ids: [271], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: 'Here', output_token_ids: [8160], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' are', output_token_ids: [513], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' the', output_token_ids: [279], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' dates', output_token_ids: [12348], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' you', output_token_ids: [488], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' requested', output_token_ids: [10897], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ',', output_token_ids: [11], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' listed', output_token_ids: [9711], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' precisely', output_token_ids: [22898], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ':', output_token_ids: [25], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:18 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '\n\n', output_token_ids: [271], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '1', output_token_ids: [16], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '.', output_token_ids: [13], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' **', output_token_ids: [2972], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: 'January', output_token_ids: [31164], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '1', output_token_ids: [16], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '5', output_token_ids: [20], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ',', output_token_ids: [11], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '2', output_token_ids: [17], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '0', output_token_ids: [15], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '2', output_token_ids: [17], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '4', output_token_ids: [19], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '**', output_token_ids: [332], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '\n', output_token_ids: [198], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '2', output_token_ids: [17], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '.', output_token_ids: [13], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' **', output_token_ids: [2972], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: 'February', output_token_ids: [31691], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '2', output_token_ids: [17], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '8', output_token_ids: [23], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ',', output_token_ids: [11], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '1', output_token_ids: [16], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '9', output_token_ids: [24], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '9', output_token_ids: [24], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '9', output_token_ids: [24], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '**', output_token_ids: [332], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '\n', output_token_ids: [198], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '3', output_token_ids: [18], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '.', output_token_ids: [13], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' **', output_token_ids: [2972], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: 'March', output_token_ids: [26631], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '3', output_token_ids: [18], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ',', output_token_ids: [11], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '2', output_token_ids: [17], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '0', output_token_ids: [15], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '2', output_token_ids: [17], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '6', output_token_ids: [21], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '**', output_token_ids: [332], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '\n', output_token_ids: [198], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '4', output_token_ids: [19], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '.', output_token_ids: [13], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' **', output_token_ids: [2972], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: 'December', output_token_ids: [31087], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '3', output_token_ids: [18], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '1', output_token_ids: [16], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ',', output_token_ids: [11], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: ' ', output_token_ids: [220], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '2', output_token_ids: [17], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '0', output_token_ids: [15], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '0', output_token_ids: [15], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '0', output_token_ids: [15], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming delta): output: '**', output_token_ids: [332], finish_reason: None (APIServer pid=1) INFO 03-02 18:47:19 [logger.py:78] Generated response chatcmpl-b1f7fd6e394e173b (streaming complete): output: '\n\nHere are the dates you requested, listed precisely:\n\n1. **January 15, 2024**\n2. **February 28, 1999**\n3. **March 3, 2026**\n4. **December 31, 2000**', output_token_ids: None, finish_reason: streaming_complete (APIServer pid=1) INFO 03-02 18:47:19 [loggers.py:259] Engine 000: Avg prompt throughput: 501.8 tokens/s, Avg generation throughput: 16.9 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0% (APIServer pid=1) INFO 03-02 18:47:20 [logger.py:49] Received request chatcmpl-b83927f13171723c: params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.0, temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=32424, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, structured_outputs=None, extra_args=None), lora_request: None. ``` <img width="1499" height="543" alt="Image" src="https://github.com/user-attachments/assets/e7fd3425-1cca-47da-b37e-9f6a87558786" />
GiteaMirror commented 2026-05-05 22:51:41 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

did you try with a different model also?

<!-- gh-comment-id:3986227408 --> @Classic298 commented on GitHub (Mar 2, 2026): did you try with a different model also?
GiteaMirror commented 2026-05-05 22:51:42 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

Besides testing w/ a different model, please also do the following:

Could you try adding ENABLE_REALTIME_CHAT_SAVE=true as an environment variable on your Open WebUI container? This changes how the stream processing saves content and would help us determine if the issue is in the delta batching/emission path or the content accumulation itself.

<!-- gh-comment-id:3986315856 --> @Classic298 commented on GitHub (Mar 2, 2026): Besides testing w/ a different model, please also do the following: Could you try adding ENABLE_REALTIME_CHAT_SAVE=true as an environment variable on your Open WebUI container? This changes how the stream processing saves content and would help us determine if the issue is in the delta batching/emission path or the content accumulation itself.
GiteaMirror commented 2026-05-05 22:51:44 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

I did enable that and didn't see any useful logs.

<!-- gh-comment-id:3986353212 --> @anselor commented on GitHub (Mar 2, 2026): I did enable that and didn't see any useful logs.
GiteaMirror commented 2026-05-05 22:51:46 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

@anselor to clarify - ENABLE_REALTIME_CHAT_SAVE isn't about generating logs, it changes the internal code path for how streaming content is persisted. The question is: with ENABLE_REALTIME_CHAT_SAVE=true set, does the token dropping still occur? Same missing characters in the output? Or does it behave differently?

Also, two things that would really help narrow this down:

  1. Have you tested with a different model on the same vLLM instance? Specifically a model that does NOT use thinking/reasoning tags (i.e. not a Qwen reasoning model). This would tell us if the issue is related to the reasoning tag detection code path.

  2. During streaming in your browser, open DevTools (F12) → Console. You should see objects being logged for each streaming event. While the response is actively streaming (before it finishes), do you see the dates appearing correctly and then getting truncated at the end? Or are the characters never appearing at all during streaming? This tells us whether tokens are being received and then lost during finalization, vs never making it through the stream processing.

<!-- gh-comment-id:3986378213 --> @Classic298 commented on GitHub (Mar 2, 2026): @anselor to clarify - ENABLE_REALTIME_CHAT_SAVE isn't about generating logs, it changes the internal code path for how streaming content is persisted. The question is: with ENABLE_REALTIME_CHAT_SAVE=true set, does the token dropping still occur? Same missing characters in the output? Or does it behave differently? Also, two things that would really help narrow this down: 1. Have you tested with a different model on the same vLLM instance? Specifically a model that does NOT use thinking/reasoning tags (i.e. not a Qwen reasoning model). This would tell us if the issue is related to the reasoning tag detection code path. 2. During streaming in your browser, open DevTools (F12) → Console. You should see objects being logged for each streaming event. While the response is actively streaming (before it finishes), do you see the dates appearing correctly and then getting truncated at the end? Or are the characters never appearing at all during streaming? This tells us whether tokens are being received and then lost during finalization, vs never making it through the stream processing.
GiteaMirror commented 2026-05-05 22:51:51 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

I've not tried a different model.
realtime chat save doesn't change the output - same errors.

I don't know what you're expecting to find in the browser. I see 2 things come in on the network:
completions then completed.
I don't see individual chunks coming in through the network tab.

<!-- gh-comment-id:3986408198 --> @anselor commented on GitHub (Mar 2, 2026): I've not tried a different model. realtime chat save doesn't change the output - same errors. I don't know what you're expecting to find in the browser. I see 2 things come in on the network: completions then completed. I don't see individual chunks coming in through the network tab.
GiteaMirror commented 2026-05-05 22:51:57 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

I definitely do not see the year appearing and then disappearing.

https://github.com/user-attachments/assets/ebc26d1c-ff7b-4c93-8ee4-b5c9554c37c3

<!-- gh-comment-id:3986432108 --> @anselor commented on GitHub (Mar 2, 2026): I definitely do not see the year appearing and then disappearing. https://github.com/user-attachments/assets/ebc26d1c-ff7b-4c93-8ee4-b5c9554c37c3
GiteaMirror commented 2026-05-05 22:52:05 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

@anselor the streaming events don't show up in the Network tab because they go through WebSocket (socket.io), not HTTP. That's why you only see the initial "completions" request and the final "completed" call.

To see the individual streaming events, switch to the Console tab instead. You should see objects being logged as each token arrives during streaming. Look at these logged objects - specifically any that have a "content" field - and check whether the dates appear correctly in those content values.

Also, two more questions that would really help:
1. Do you have any pipeline filters or functions installed and active in your Open WebUI instance?

2. Can you test with a non-reasoning model (one that doesn't use think tags)? This will tell us if the reasoning/thinking tag detection is involved.

<!-- gh-comment-id:3986451510 --> @Classic298 commented on GitHub (Mar 2, 2026): @anselor the streaming events don't show up in the Network tab because they go through WebSocket (socket.io), not HTTP. That's why you only see the initial "completions" request and the final "completed" call. To see the individual streaming events, switch to the Console tab instead. You should see objects being logged as each token arrives during streaming. Look at these logged objects - specifically any that have a "content" field - and check whether the dates appear correctly in those content values. Also, two more questions that would really help: <ins>**1. Do you have any pipeline filters or functions installed and active in your Open WebUI instance?**</ins> <ins>**2. Can you test with a non-reasoning model (one that doesn't use think tags)? This will tell us if the reasoning/thinking tag detection is involved.**</ins>
GiteaMirror commented 2026-05-05 22:52:08 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

https://github.com/user-attachments/assets/ab8868a0-5005-48a4-aa9b-4faf58ebcf6b

<!-- gh-comment-id:3986461385 --> @anselor commented on GitHub (Mar 2, 2026): https://github.com/user-attachments/assets/ab8868a0-5005-48a4-aa9b-4faf58ebcf6b
GiteaMirror commented 2026-05-05 22:52:09 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

We really need you to test with a non-reasoning model to narrow this down. This is the single most useful test you can run right now. If the issue only happens with models that use thinking tags, we know exactly which code path to focus on. If it also happens with a standard model, that rules out a large chunk of the code. Can you load any standard (non-thinking) non-qwen model on your vLLM instance and try the same date prompt?

Re: console logging, be right back

<!-- gh-comment-id:3986473662 --> @Classic298 commented on GitHub (Mar 2, 2026): We really need you to test with a non-reasoning model to narrow this down. This is the single most useful test you can run right now. If the issue only happens with models that use thinking tags, we know exactly which code path to focus on. If it also happens with a standard model, that rules out a large chunk of the code. Can you load any standard (non-thinking) non-qwen model on your vLLM instance and try the same date prompt? Re: console logging, be right back
GiteaMirror commented 2026-05-05 22:52:10 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

ok for console logging you'd need to run a development setup. That would be helpful but i suppose you cannot do that?

<!-- gh-comment-id:3986488380 --> @Classic298 commented on GitHub (Mar 2, 2026): ok for console logging you'd need to run a development setup. That would be helpful but i suppose you cannot do that?
GiteaMirror commented 2026-05-05 22:52:12 -05:00
Author
Owner
Copy Link
Copy Source

@anselor commented on GitHub (Mar 2, 2026):

  1. Do you have any pipeline filters or functions installed and active in your Open WebUI instance?

This was it! I had some thinking stream fixup function active from when gpt-oss was first released. Thank you

<!-- gh-comment-id:3986517145 --> @anselor commented on GitHub (Mar 2, 2026): > 1. Do you have any pipeline filters or functions installed and active in your Open WebUI instance? This was it! I had some thinking stream fixup function active from when gpt-oss was first released. Thank you
GiteaMirror commented 2026-05-05 22:52:13 -05:00
Author
Owner
Copy Link
Copy Source

@Classic298 commented on GitHub (Mar 2, 2026):

.................

Glad we found a solution but next time don't say I am just looking for excuses ;)

Sometimes I may be wrong, but most of the time I know when something is a setup/config issue versus actual Open WebUI bug

<!-- gh-comment-id:3986529029 --> @Classic298 commented on GitHub (Mar 2, 2026): ................. Glad we found a solution but next time don't say I am just looking for excuses ;) Sometimes I may be wrong, but most of the time I know when something is a setup/config issue versus actual Open WebUI bug
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No labels bug
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#58309