github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-07-16 14:39:31 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[GH-ISSUE #24697] issue: reasoning_content from vLLM streaming dropped after first thinking block #107375

New Issue
Closed
opened 2026-05-18 06:10:23 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-05-18 06:10:23 -05:00
Owner
Copy Link

Originally created by @brandons209 on GitHub (May 14, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/24697

Check Existing Issues

  • I have searched for any existing and/or related issues.
  • I have searched for any existing and/or related discussions.
  • I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!).
  • I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

v0.9.5

Ollama Version (if applicable)

I am using vLLM version 0.20.2 proxied through LiteLLM

Operating System

Ubuntu 24.04.4 LTS

Browser (if applicable)

No response

Confirmation

  • I have read and followed all instructions in README.md.
  • I am using the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.
  • I have provided every relevant configuration, setting, and environment variable used in my setup.
  • I have clearly listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc).
  • I have documented step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation. My steps:
  • Start with the initial platform/version/OS and dependencies used,
  • Specify exact install/launch/configure commands,
  • List URLs visited, user input (incl. example values/emails/passwords if needed),
  • Describe all options and toggles enabled or changed,
  • Include any files or environmental changes,
  • Identify the expected and actual result at each stage,
  • Ensure any reasonably skilled user can follow and hit the same issue.

Expected Behavior

When a vLLM upstream returns a streaming response containing both delta.reasoning_content chunks (from --reasoning-parser qwen3, tested with qwen3.6-27b) and subsequent delta.content chunks ending with finish_reason: "stop", Open WebUI should:

Render the reasoning content in a collapsible "Thinking" section above the final answer.
Render the subsequent content chunks as the visible final answer below the thinking section.
Persist both fields correctly to the chat record for multi-turn continuity.

Actual Behavior

Only the "Thinking" section renders. The subsequent content chunks are dropped by Open WebUI's frontend stream reconciliation, they never appear as an answer nor does the model continue with the next turn / tool call. This occurs both in API requests and using the chat interface.

The raw stream from LiteLLM contains both phases correctly (verified with curl directly against LiteLLM). The bug is purely in Open WebUI's handling of the separated reasoning_content + content stream from vLLM.

This is essentially the same symptom class as closed issue #23173 and open issue #24295, but specifically for the vLLM --reasoning-parser qwen3. Discussion #23895 has multiple users confirming the Provider: llama.cpp connection-dropdown workaround does not apply to vLLM upstreams (which I confirmed myself as well.)

Steps to Reproduce

Tested on a Ubuntu Server 24.04 with two RTX 3090s, but should reproduce on any vLLM 0.20+ deployment serving any Qwen3.6 model.

  1. Deploy vLLM 0.20.2 serving any Qwen3.6 family model (tested with Qwen/Qwen3.6-27B-FP8).

  2. Launch vLLM with the Qwen3 reasoning parser so reasoning is split out of content:

    vllm serve /models/qwen3.6-27b-fp8 \
  --served-model-name qwen3.6-27b \
  --tensor-parallel-size 2 \
  --max-model-len 72000 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

  3. (Optional) Put LiteLLM in front of vLLM with this entry in config.yaml:

    model_list:
  - model_name: qwen3.6-27b
    litellm_params:
      model: openai/qwen3.6-27b
      api_base: http://vllm:8000/v1
      api_key: os.environ/VLLM_KEY
      max_tokens: 16000
    model_info:
      supports_reasoning: true

    Restart LiteLLM. Do not set merge_reasoning_content_in_choices: true for this repro.

  4. In Open WebUI v0.9.5, configure a Connection to the LiteLLM endpoint:

    • Admin → Settings → Connections → OpenAI API
    • URL: http://litellm:4000/v1
    • Provider dropdown: Default (leaving as-is; switching to llama.cpp does not help vLLM upstreams, per discussion #23895)

  5. Verify the stream is correctly shaped by hitting LiteLLM directly:

    curl -sS -N -H "Authorization: Bearer $LITELLM_KEY" \
  -H 'Content-Type: application/json' \
  http://litellm:4000/v1/chat/completions \
  -d '{
    "model": "qwen3.6-27b",
    "messages": [{"role":"user","content":"What is 2+2? Be brief."}],
    "stream": true
  }'

    Expected (and what I actually see): a sequence of chunks like:

    data: {"choices":[{"delta":{"reasoning_content":"Here","role":"assistant"}}]}
data: {"choices":[{"delta":{"reasoning_content":"'s"}}]}
... (many reasoning_content chunks) ...
data: {"choices":[{"delta":{"content":"4"}}]}
data: {"choices":[{"finish_reason":"stop","delta":{}}]}
data: [DONE]

    Both phases are present, finish_reason: stop.

  6. In the Open WebUI chat UI, select the qwen3.6-27b model and send: What is 2+2? Be brief.

  7. Observe: the UI shows a "Thinking" section that fills in correctly, then renders nothing else after thinking finishes. The expected answer (4) never appears. No error is visible to the user.

  8. Inspect the rendered chat record:

    curl -sS -H "Authorization: Bearer $OWUI_KEY" \
  "$OWUI_URL/api/v1/chats/<chat_id>" | jq '.chat.messages[-1]'

    The assistant message has populated reasoning but content is empty or missing.

Logs & Screenshots

Response from model in open-webui:

Image

Open WebUI Logs:
open-webui | 2026-05-13 20:00:16.483 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 23.234.111.26:0 - "POST /api/v1/chats/f9bbbd19-2f29-49bb-8363-53cc298d6077 HTTP/1.1" 200
open-webui | 2026-05-13 20:00:19.384 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.169:57274 - "GET / HTTP/1.1" 200
open-webui | 2026-05-13 20:00:19.724 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 23.234.111.26:0 - "GET /api/v1/terminals/open-terminal/ports HTTP/1.1" 200
open-webui | 2026-05-13 20:00:23.277 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 23.234.111.26:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200
open-webui | 2026-05-13 20:00:24.388 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.169:35074 - "GET / HTTP/1.1" 200

Correct output from LiteLLM proxying the vLLM endpoint:

{
    "id": "chatcmpl-9c21a8b97e07ab9a",
    "created": 1778716971,
    "model": "qwen3.6-27b",
    "object": "chat.completion",
    "choices": [
        {
            "finish_reason": "stop",
            "index": 0,
            "message": {
                "content": "\n\n4.",
                "role": "assistant",
                "reasoning_content": "Here's a thinking process:\n\n1.  **Analyze User Input:**\n   - Question: \"What is 2+2?\"\n   - Constraint: \"Be brief.\"\n\n2.  **Identify Core Task:**\n   - Perform simple arithmetic: 2 + 2 = 4.\n   - Adhere to brevity constraint.\n\n3.  **Formulate Response:**\n   - Direct answer: \"4\"\n   - Check constraint: Very brief.\n   - Could add minimal context if needed, but \"4\" or \"4.\" is sufficient.\n\n4.  **Final Output Generation:**\n   - \"4\" (or \"4.\") matches the request perfectly. I'll go with just \"4.\" for clarity and brevity.\u2705\n",
                "provider_specific_fields": {
                    "refusal": null,
                    "reasoning": "Here's a thinking process:\n\n1.  **Analyze User Input:**\n   - Question: \"What is 2+2?\"\n   - Constraint: \"Be brief.\"\n\n2.  **Identify Core Task:**\n   - Perform simple arithmetic: 2 + 2 = 4.\n   - Adhere to brevity constraint.\n\n3.  **Formulate Response:**\n   - Direct answer: \"4\"\n   - Check constraint: Very brief.\n   - Could add minimal context if needed, but \"4\" or \"4.\" is sufficient.\n\n4.  **Final Output Generation:**\n   - \"4\" (or \"4.\") matches the request perfectly. I'll go with just \"4.\" for clarity and brevity.\u2705\n",
                    "reasoning_content": "Here's a thinking process:\n\n1.  **Analyze User Input:**\n   - Question: \"What is 2+2?\"\n   - Constraint: \"Be brief.\"\n\n2.  **Identify Core Task:**\n   - Perform simple arithmetic: 2 + 2 = 4.\n   - Adhere to brevity constraint.\n\n3.  **Formulate Response:**\n   - Direct answer: \"4\"\n   - Check constraint: Very brief.\n   - Could add minimal context if needed, but \"4\" or \"4.\" is sufficient.\n\n4.  **Final Output Generation:**\n   - \"4\" (or \"4.\") matches the request perfectly. I'll go with just \"4.\" for clarity and brevity.\u2705\n"
                }
            },
            "provider_specific_fields": {
                "stop_reason": null,
                "token_ids": null
            }
        }
    ],
    "usage": {
        "completion_tokens": 168,
        "prompt_tokens": 20,
        "total_tokens": 188
    }
}

Additional Information

According the the Qwen3.6 models official docs, the opening tag for reasoning is pre-filled by the chat template, the model will only output the closing tag, so the workaround of injecting the reasoning content directly into the content key instead of the separate reasoning_content also will not work since open-webui will see the ending think tag and stop prematurely.

Workarounds I tried:

Workaround Helps?
Setting merge_reasoning_content_in_choices: true in LiteLLM per their tutorial Partial - content arrives wrapped, but Open WebUI then renders the model's emitted </think> as literal text because the prompt-injected opening tag is stripped from output (Qwen3.6+ template behavior). I see more of the content but it still stops prematurely
Switching the Open WebUI Connection's Provider dropdown to llama.cpp No
Dropping --reasoning-parser qwen3 so vLLM emits raw thinking blocks No
Calling the workspace model from a custom tool/router that reads message.content directly via the API (bypassing the UI renderer) No
Originally created by @brandons209 on GitHub (May 14, 2026). Original GitHub issue: https://github.com/open-webui/open-webui/issues/24697 ### Check Existing Issues - [x] I have searched for any existing and/or related issues. - [x] I have searched for any existing and/or related discussions. - [x] I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!). - [x] I am using the latest version of Open WebUI. ### Installation Method Docker ### Open WebUI Version v0.9.5 ### Ollama Version (if applicable) I am using vLLM version 0.20.2 proxied through LiteLLM ### Operating System Ubuntu 24.04.4 LTS ### Browser (if applicable) _No response_ ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have **provided every relevant configuration, setting, and environment variable used in my setup.** - [x] I have clearly **listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup** (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc). - [x] I have documented **step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation**. My steps: - Start with the initial platform/version/OS and dependencies used, - Specify exact install/launch/configure commands, - List URLs visited, user input (incl. example values/emails/passwords if needed), - Describe all options and toggles enabled or changed, - Include any files or environmental changes, - Identify the expected and actual result at each stage, - Ensure any reasonably skilled user can follow and hit the same issue. ### Expected Behavior When a vLLM upstream returns a streaming response containing both delta.reasoning_content chunks (from --reasoning-parser qwen3, tested with qwen3.6-27b) and subsequent delta.content chunks ending with finish_reason: "stop", Open WebUI should: Render the reasoning content in a collapsible "Thinking" section above the final answer. Render the subsequent content chunks as the visible final answer below the thinking section. Persist both fields correctly to the chat record for multi-turn continuity. ### Actual Behavior Only the "Thinking" section renders. The subsequent content chunks are dropped by Open WebUI's frontend stream reconciliation, they never appear as an answer nor does the model continue with the next turn / tool call. This occurs both in API requests and using the chat interface. The raw stream from LiteLLM contains both phases correctly (verified with curl directly against LiteLLM). The bug is purely in Open WebUI's handling of the separated reasoning_content + content stream from vLLM. This is essentially the same symptom class as closed issue #23173 and open issue #24295, but specifically for the vLLM --reasoning-parser qwen3. Discussion #23895 has multiple users confirming the Provider: llama.cpp connection-dropdown workaround does not apply to vLLM upstreams (which I confirmed myself as well.) ### Steps to Reproduce Tested on a Ubuntu Server 24.04 with two RTX 3090s, but should reproduce on any vLLM 0.20+ deployment serving any Qwen3.6 model. 1. **Deploy vLLM 0.20.2** serving any Qwen3.6 family model (tested with `Qwen/Qwen3.6-27B-FP8`). 2. **Launch vLLM with the Qwen3 reasoning parser** so reasoning is split out of `content`: ```bash vllm serve /models/qwen3.6-27b-fp8 \ --served-model-name qwen3.6-27b \ --tensor-parallel-size 2 \ --max-model-len 72000 \ --reasoning-parser qwen3 \ --enable-auto-tool-choice \ --tool-call-parser hermes ``` 3. (Optional) **Put LiteLLM in front of vLLM** with this entry in `config.yaml`: ```yaml model_list: - model_name: qwen3.6-27b litellm_params: model: openai/qwen3.6-27b api_base: http://vllm:8000/v1 api_key: os.environ/VLLM_KEY max_tokens: 16000 model_info: supports_reasoning: true ``` Restart LiteLLM. Do **not** set `merge_reasoning_content_in_choices: true` for this repro. 4. **In Open WebUI v0.9.5**, configure a Connection to the LiteLLM endpoint: - Admin → Settings → Connections → OpenAI API - URL: `http://litellm:4000/v1` - Provider dropdown: **Default** (leaving as-is; switching to `llama.cpp` does not help vLLM upstreams, per discussion #23895) 5. **Verify the stream is correctly shaped** by hitting LiteLLM directly: ```bash curl -sS -N -H "Authorization: Bearer $LITELLM_KEY" \ -H 'Content-Type: application/json' \ http://litellm:4000/v1/chat/completions \ -d '{ "model": "qwen3.6-27b", "messages": [{"role":"user","content":"What is 2+2? Be brief."}], "stream": true }' ``` Expected (and what I actually see): a sequence of chunks like: ``` data: {"choices":[{"delta":{"reasoning_content":"Here","role":"assistant"}}]} data: {"choices":[{"delta":{"reasoning_content":"'s"}}]} ... (many reasoning_content chunks) ... data: {"choices":[{"delta":{"content":"4"}}]} data: {"choices":[{"finish_reason":"stop","delta":{}}]} data: [DONE] ``` Both phases are present, `finish_reason: stop`. 6. **In the Open WebUI chat UI**, select the `qwen3.6-27b` model and send: `What is 2+2? Be brief.` 7. **Observe:** the UI shows a "Thinking" section that fills in correctly, then renders nothing else after thinking finishes. The expected answer (`4`) never appears. No error is visible to the user. 8. **Inspect the rendered chat record:** ```bash curl -sS -H "Authorization: Bearer $OWUI_KEY" \ "$OWUI_URL/api/v1/chats/<chat_id>" | jq '.chat.messages[-1]' ``` The assistant message has populated reasoning but `content` is empty or missing. ### Logs & Screenshots Response from model in open-webui: <img width="1326" height="274" alt="Image" src="https://github.com/user-attachments/assets/6f03bbcc-9935-4e4a-9a40-3b115f6d0e49" /> Open WebUI Logs: open-webui | 2026-05-13 20:00:16.483 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 23.234.111.26:0 - "POST /api/v1/chats/f9bbbd19-2f29-49bb-8363-53cc298d6077 HTTP/1.1" 200 open-webui | 2026-05-13 20:00:19.384 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.169:57274 - "GET / HTTP/1.1" 200 open-webui | 2026-05-13 20:00:19.724 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 23.234.111.26:0 - "GET /api/v1/terminals/open-terminal/ports HTTP/1.1" 200 open-webui | 2026-05-13 20:00:23.277 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 23.234.111.26:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 open-webui | 2026-05-13 20:00:24.388 | INFO | uvicorn.protocols.http.httptools_impl:send:483 - 192.168.0.169:35074 - "GET / HTTP/1.1" 200 Correct output from LiteLLM proxying the vLLM endpoint: ```json { "id": "chatcmpl-9c21a8b97e07ab9a", "created": 1778716971, "model": "qwen3.6-27b", "object": "chat.completion", "choices": [ { "finish_reason": "stop", "index": 0, "message": { "content": "\n\n4.", "role": "assistant", "reasoning_content": "Here's a thinking process:\n\n1. **Analyze User Input:**\n - Question: \"What is 2+2?\"\n - Constraint: \"Be brief.\"\n\n2. **Identify Core Task:**\n - Perform simple arithmetic: 2 + 2 = 4.\n - Adhere to brevity constraint.\n\n3. **Formulate Response:**\n - Direct answer: \"4\"\n - Check constraint: Very brief.\n - Could add minimal context if needed, but \"4\" or \"4.\" is sufficient.\n\n4. **Final Output Generation:**\n - \"4\" (or \"4.\") matches the request perfectly. I'll go with just \"4.\" for clarity and brevity.\u2705\n", "provider_specific_fields": { "refusal": null, "reasoning": "Here's a thinking process:\n\n1. **Analyze User Input:**\n - Question: \"What is 2+2?\"\n - Constraint: \"Be brief.\"\n\n2. **Identify Core Task:**\n - Perform simple arithmetic: 2 + 2 = 4.\n - Adhere to brevity constraint.\n\n3. **Formulate Response:**\n - Direct answer: \"4\"\n - Check constraint: Very brief.\n - Could add minimal context if needed, but \"4\" or \"4.\" is sufficient.\n\n4. **Final Output Generation:**\n - \"4\" (or \"4.\") matches the request perfectly. I'll go with just \"4.\" for clarity and brevity.\u2705\n", "reasoning_content": "Here's a thinking process:\n\n1. **Analyze User Input:**\n - Question: \"What is 2+2?\"\n - Constraint: \"Be brief.\"\n\n2. **Identify Core Task:**\n - Perform simple arithmetic: 2 + 2 = 4.\n - Adhere to brevity constraint.\n\n3. **Formulate Response:**\n - Direct answer: \"4\"\n - Check constraint: Very brief.\n - Could add minimal context if needed, but \"4\" or \"4.\" is sufficient.\n\n4. **Final Output Generation:**\n - \"4\" (or \"4.\") matches the request perfectly. I'll go with just \"4.\" for clarity and brevity.\u2705\n" } }, "provider_specific_fields": { "stop_reason": null, "token_ids": null } } ], "usage": { "completion_tokens": 168, "prompt_tokens": 20, "total_tokens": 188 } } ``` ### Additional Information According the the Qwen3.6 models official docs, the opening tag for reasoning is pre-filled by the chat template, the model will only output the closing tag, so the workaround of injecting the reasoning content directly into the content key instead of the separate reasoning_content also will not work since open-webui will see the ending think tag and stop prematurely. Workarounds I tried: | Workaround | Helps? | |---|---| | Setting `merge_reasoning_content_in_choices: true` in LiteLLM per their tutorial | Partial - content arrives wrapped, but Open WebUI then renders the model's emitted `</think>` as literal text because the prompt-injected opening tag is stripped from output (Qwen3.6+ template behavior). I see more of the content but it still stops prematurely | | Switching the Open WebUI Connection's Provider dropdown to `llama.cpp` | No | | Dropping `--reasoning-parser qwen3` so vLLM emits raw thinking blocks| No | | Calling the workspace model from a custom tool/router that reads `message.content` directly via the API (bypassing the UI renderer) | No |
GiteaMirror added the bug label 2026-05-18 06:10:23 -05:00
GiteaMirror commented 2026-05-18 06:10:26 -05:00
Author
Owner
Copy Link

@owui-terminator[bot] commented on GitHub (May 14, 2026):

🔍 Related Issues Found

I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions:

  1. 🟣 #23173 issue: After native tool calls, streaming reasoning_content is truncated to the first token in the UI
    Very similar stream-reconciliation bug: Open WebUI mishandles streamed reasoning_content after the first segment, causing the reasoning/final-answer split to break in the UI. Your report differs in that the final content is dropped after the thinking block, but it’s clearly the closest prior symptom class.
    by ianustec · bug

  2. 🟢 #24295 issue: Reasoning UI shows "Thought for less than a second" too early while streamed reasoning/tool execution is still running (LiteLLM Responses API + Azure OpenAI)
    Open issue about reasoning UI state with LiteLLM/OpenAI-compatible reasoning streams. It’s not the exact same failure mode, but it involves the same Open WebUI reasoning pipeline and LiteLLM proxying behavior.
    by DediCATeD88 · bug

  3. 🟣 #23879 issue: reasoning_content is ignored in non-stream openai API
    Closed issue showing Open WebUI’s OpenAI-compatible handling of reasoning_content can be wrong in non-streaming mode. This supports that the same parsing/rendering path around reasoning_content has had related bugs.
    by vanmilleru · bug

  4. 🟣 #18058 issue: handle thinking for Qwen3-VL models
    Older vLLM/Qwen thinking-handling issue in Open WebUI. It’s about Qwen3-VL/vLLM reasoning not being rendered in the correct place, so it’s relevant as an upstream/provider-specific precedent.
    by SlavikCA · bug

  5. 🟣 #23175 Issue #23175
    Related multi-turn/tool-call bug where reasoning_content is stripped from assistant messages, breaking follow-up turns. Your issue also affects persistence of reasoning vs content across turns, so this is adjacent and useful context.
    by unknown

💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead.

This comment was generated automatically. React with 👍 if helpful, 👎 if not.

<!-- gh-comment-id:4446187202 --> @owui-terminator[bot] commented on GitHub (May 14, 2026): <!-- terminator-bot:related-issues-reply --> 🔍 **Related Issues Found** I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions: 1. 🟣 [#23173](https://github.com/open-webui/open-webui/issues/23173) **issue: After native tool calls, streaming reasoning_content is truncated to the first token in the UI** *Very similar stream-reconciliation bug: Open WebUI mishandles streamed `reasoning_content` after the first segment, causing the reasoning/final-answer split to break in the UI. Your report differs in that the final `content` is dropped after the thinking block, but it’s clearly the closest prior symptom class.* *by ianustec · `bug`* 2. 🟢 [#24295](https://github.com/open-webui/open-webui/issues/24295) **issue: Reasoning UI shows "Thought for less than a second" too early while streamed reasoning/tool execution is still running (LiteLLM Responses API + Azure OpenAI)** *Open issue about reasoning UI state with LiteLLM/OpenAI-compatible reasoning streams. It’s not the exact same failure mode, but it involves the same Open WebUI reasoning pipeline and LiteLLM proxying behavior.* *by DediCATeD88 · `bug`* 3. 🟣 [#23879](https://github.com/open-webui/open-webui/issues/23879) **issue: reasoning_content is ignored in non-stream openai API** *Closed issue showing Open WebUI’s OpenAI-compatible handling of `reasoning_content` can be wrong in non-streaming mode. This supports that the same parsing/rendering path around `reasoning_content` has had related bugs.* *by vanmilleru · `bug`* 4. 🟣 [#18058](https://github.com/open-webui/open-webui/issues/18058) **issue: handle thinking for Qwen3-VL models** *Older vLLM/Qwen thinking-handling issue in Open WebUI. It’s about Qwen3-VL/vLLM reasoning not being rendered in the correct place, so it’s relevant as an upstream/provider-specific precedent.* *by SlavikCA · `bug`* 5. 🟣 [#23175](https://github.com/open-webui/open-webui/issues/23175) **Issue #23175** *Related multi-turn/tool-call bug where `reasoning_content` is stripped from assistant messages, breaking follow-up turns. Your issue also affects persistence of reasoning vs content across turns, so this is adjacent and useful context.* *by unknown* --- 💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead. *This comment was generated automatically.* React with 👍 if helpful, 👎 if not.
GiteaMirror commented 2026-05-18 06:10:26 -05:00
Author
Owner
Copy Link

@brandons209 commented on GitHub (May 14, 2026):

All the call signs seemed to point to this issue, but I looked over the fact that I was using the wrong tool parser in vLLM. When I saw the logs from the request it looked right but actually wasn't being parsed properly. Closing the issue, double check your configs against model cards!

<!-- gh-comment-id:4446503073 --> @brandons209 commented on GitHub (May 14, 2026): All the call signs seemed to point to this issue, but I looked over the fact that I was using the wrong tool parser in vLLM. When I saw the logs from the request it looked right but actually wasn't being parsed properly. Closing the issue, double check your configs against model cards!
GiteaMirror commented 2026-05-18 06:10:27 -05:00
Author
Owner
Copy Link

@j820301 commented on GitHub (May 15, 2026):

Hello, I would like to ask how to enable thought mode in vLLM? I am currently using vLLM gemma4:26b-it and don't see any thinking process.

openwebui how setting?

services:
vllm:
image: vllm/vllm-openai:v0.20.2
container_name: vllm
runtime: nvidia
restart: unless-stopped
ports:
- "8888:8000"
volumes:
- ./.cache/huggingface:/root/.cache/huggingface
ipc: host
shm_size: 16gb
command: >
--model google/gemma-4-26B-A4B-it
--tensor-parallel-size 1
--trust-remote-code
--max-num-seqs 32
--enforce-eager
--kv-cache-dtype fp8
--enable-auto-tool-choice
--reasoning-parser gemma4
--tool-call-parser gemma4
--host 0.0.0.0
--port 8000
--gpu-memory-utilization 0.95
--max-model-len 131072

deploy:
  resources:
    reservations:
      devices:
        - driver: nvidia
          count: all
          capabilities: [gpu]
<!-- gh-comment-id:4457072965 --> @j820301 commented on GitHub (May 15, 2026): Hello, I would like to ask how to enable thought mode in vLLM? I am currently using vLLM gemma4:26b-it and don't see any thinking process. openwebui how setting? services: vllm: image: vllm/vllm-openai:v0.20.2 container_name: vllm runtime: nvidia restart: unless-stopped ports: - "8888:8000" volumes: - ./.cache/huggingface:/root/.cache/huggingface ipc: host shm_size: 16gb command: > --model google/gemma-4-26B-A4B-it --tensor-parallel-size 1 --trust-remote-code --max-num-seqs 32 --enforce-eager --kv-cache-dtype fp8 --enable-auto-tool-choice --reasoning-parser gemma4 --tool-call-parser gemma4 --host 0.0.0.0 --port 8000 --gpu-memory-utilization 0.95 --max-model-len 131072 deploy: resources: reservations: devices: - driver: nvidia count: all capabilities: [gpu]
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#107375
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?