[GH-ISSUE #8738] Support for beautiful and functional rendering of "reasoning" model output (like deepseek-r1) #102229

New Issue

2026-05-17T23:38:41-05:00

GiteaMirror commented

2026-05-17 23:38:41 -05:00

Originally created by @kyr0 on GitHub (Jan 22, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/8738

Feature Request

Current state

Currently, Open WebUI renders the tokens indicating a reasoning ("thinking") process 1:1 as they are streamed:

Current behaviour for opening thinking tag rendering:

Current behaviour for closing thinking tag rendering:

Proposed solution

a) Let's stop the streamed rendering output once we hit a pre-configurable <think> token. Once the end token </think> is found, we restart the rendering.

b) In "thinking rendering mode", we render a status message (similar to OpenAI's UI). Therefore, we wait for each paragraph to finish. Once we have the paragraph, we could use a configurable model (the same or an alternative) to summarize the paragraph to show that it is currently 'thinking' about. We would display an animated "Thinking..." message until we found the first paragraph to summarize on or if summarizing is disabled.This message should be rendered with less contrast.

c) We use the performance API to measure the time between start and end of the process to replace the message with the Thought for n seconds message once we hit the end thinking token.

How it might look like / or similar

Opening

Closing

Technical issue 1: How do we know the right token name to stop for

Every model potentially uses a different, distinct token name. Some may go with <think> others with <reason> or the "ing"-form. However, there are only so many different tokens that make sense, and none of them are probable to be used in a different context. Because of that, we can't simply hardcode the token name.

However, currently, such token is not part of the modelfile. This means, that we can't read it from metadata easily via ollama. We also couldn't do this in a generalized way, even if it was defined in metadata, because remotely hosted models are not easily identifiable in many cases.

Therefore, it seems reasonable to add a field to the settings to configure it. To increase the usability, we could add a very simple heuristic like a list of RegExp to map between "thinking" token name and model name. Imagine: const thinkTokenMatchers = [{ match: /deepseek(.*)r1/i, 'think' }].

Technical issue 2: Inner tokens

A typical issue of such algorithms, that may lead to bugs is, when the text that is streaming in "thinking mode" contains a thinking end token that is not a real thinking end token but a content token.

The only way to track and solve this, is counting the opening tags and comparing to the count of closing tags. This is trivial because we're only looking for one token.

// pseudocode: 
let thinkingNesting = 1; 

if (thinkingOpeningTagFound) { 
    thinkingNesting++ 
} 

if (thinkingClosingTagFound) {
    thinkingNesting--
}

if (thinkingNesting === 0) {
    // replace message with final thinking status message and restart streaming rendering
}

Reproduction Example

Either use ollama and pull / serve https://ollama.com/library/deepseek-r1 or create an API token with https://platform.deepseek.com/ top-up and use this endpoint.

Originally created by @kyr0 on GitHub (Jan 22, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/8738 # Feature Request ## Current state Currently, Open WebUI renders the tokens indicating a reasoning ("thinking") process 1:1 as they are streamed: ### Current behaviour for opening thinking tag rendering: <img width="1044" alt="Image" src="https://github.com/user-attachments/assets/b8eaa1ee-fce3-4dbd-b834-1843b9909602" /> ### Current behaviour for closing thinking tag rendering: <img width="980" alt="Image" src="https://github.com/user-attachments/assets/a8021a6d-c5f1-43e2-8933-d21fb14408c9" /> ## Proposed solution a) Let's stop the streamed rendering output once we hit a pre-configurable `<think>` token. Once the end token `</think>` is found, we restart the rendering. b) In "thinking rendering mode", we render a status message (similar to OpenAI's UI). Therefore, we wait for each paragraph to finish. Once we have the paragraph, we could use a configurable model (the same or an alternative) to summarize the paragraph to show that it is currently 'thinking' about. We would display an animated "Thinking..." message until we found the first paragraph to summarize on or if summarizing is disabled.This message should be rendered with less contrast. c) We use the `performance` API to measure the time between start and end of the process to replace the message with the `Thought for n seconds` message once we hit the end thinking token. ### How it might look like / or similar #### Opening <img width="868" alt="Image" src="https://github.com/user-attachments/assets/cec067ca-11af-4181-8205-f67cdbc0bd36" /> #### Closing <img width="1167" alt="Image" src="https://github.com/user-attachments/assets/3926ca2f-4a5e-4ce4-8aca-aae9792fddb3" /> ### Technical issue 1: How do we know the right token name to stop for Every model potentially uses a different, distinct token name. Some may go with `<think>` others with `<reason>` or the "ing"-form. However, there are only so many different tokens that make sense, and none of them are probable to be used in a different context. Because of that, we can't simply hardcode the token name. However, currently, such token is **not** part of the `modelfile`. This means, that we can't read it from metadata easily via ollama. We also couldn't do this in a generalized way, even if it was defined in metadata, because remotely hosted models are not easily identifiable in many cases. Therefore, it seems reasonable to add a field to the settings to configure it. To increase the usability, we could add a very simple heuristic like a list of RegExp to map between "thinking" token name and model name. Imagine: `const thinkTokenMatchers = [{ match: /deepseek(.*)r1/i, 'think' }]`. ### Technical issue 2: Inner tokens A typical issue of such algorithms, that may lead to bugs is, when the text that is streaming in "thinking mode" contains a thinking end token that is **not** a real thinking end token but a content token. <img width="1032" alt="Image" src="https://github.com/user-attachments/assets/cfed9281-d06c-43c1-ba09-535a92754956" /> The only way to track and solve this, is counting the opening tags and comparing to the count of closing tags. This is trivial because we're only looking for one token. ``` // pseudocode: let thinkingNesting = 1; if (thinkingOpeningTagFound) { thinkingNesting++ } if (thinkingClosingTagFound) { thinkingNesting-- } if (thinkingNesting === 0) { // replace message with final thinking status message and restart streaming rendering } ``` ## Reproduction Example Either use `ollama` and pull / serve https://ollama.com/library/deepseek-r1 or create an API token with https://platform.deepseek.com/ top-up and use this endpoint.

GiteaMirror closed this issue

2026-05-17 23:38:42 -05:00

GiteaMirror commented

2026-05-17 23:38:43 -05:00