issue: Dont support new response/workflow for GPT open models #5967

@Shawneau commented on GitHub (Aug 5, 2025):

Also Ollama loads these models 100% CPU even when GPU is available when called from Open WebUI, but works correctly calling directly via Ollama CLI

@Shawneau commented on GitHub (Aug 5, 2025): Also Ollama loads these models 100% CPU even when GPU is available when called from Open WebUI, but works correctly calling directly via Ollama CLI

GiteaMirror commented

@light-and-ray commented on GitHub (Aug 5, 2025):

Also Ollama loads these models 100% CPU even when GPU is available when called from Open WebUI, but works correctly calling directly via Ollama CLI

In LM-Studio for me this model doesn't work with CUDA, but it does with Vulkan. Maybe you can also change somehow ollama from CUDA to Vulkan

@light-and-ray commented on GitHub (Aug 5, 2025): > Also Ollama loads these models 100% CPU even when GPU is available when called from Open WebUI, but works correctly calling directly via Ollama CLI In LM-Studio for me this model doesn't work with CUDA, but it does with Vulkan. Maybe you can also change somehow ollama from CUDA to Vulkan

GiteaMirror commented

@enmanuelmag commented on GitHub (Aug 5, 2025):

@light-and-ray

What GPU you all have? I have a RTX 4080 and the gpt-oss-20b model works and load successfully on GPU. I tested loading all layers (24 layers), GPU is used for generation as expected.

I saw just after download the model, that LM Studio ask for update CUDA Runtime, check if you have te lasted version, I'm using CUDA 12

@enmanuelmag commented on GitHub (Aug 5, 2025): @light-and-ray What GPU you all have? I have a RTX 4080 and the gpt-oss-20b model works and load successfully on GPU. I tested loading all layers (24 layers), GPU is used for generation as expected. I saw just after download the model, that LM Studio ask for update CUDA Runtime, check if you have te lasted version, I'm using CUDA 12

GiteaMirror commented

@light-and-ray commented on GitHub (Aug 5, 2025):

@enmanuelmag I see, it suggest me updating the cuda runtime from 1.43 to 1.45. So it will work

On Vulkan in 12 GB 3060 I was able to fit only 22 layers out of 24 with default 4k context. Does your 4080 also have 12GB?

@light-and-ray commented on GitHub (Aug 5, 2025): @enmanuelmag I see, it suggest me updating the cuda runtime from 1.43 to 1.45. So it will work On Vulkan in 12 GB 3060 I was able to fit only 22 layers out of 24 with default 4k context. Does your 4080 also have 12GB?

GiteaMirror commented

@enmanuelmag commented on GitHub (Aug 5, 2025):

@light-and-ray Good. The RTX 4080 has 16GB, so with 12GB is expected load fewer layers

@enmanuelmag commented on GitHub (Aug 5, 2025): @light-and-ray Good. The RTX 4080 has 16GB, so with 12GB is expected load fewer layers

GiteaMirror commented

2025-11-11 16:40:28 -06:00

@light-and-ray commented on GitHub (Aug 5, 2025):

I see that 4080 has 16 GB. And it looks like this model is designed for 16 GB cards, so my 3060 is even more outdated now with this model 😂

@light-and-ray commented on GitHub (Aug 5, 2025): I see that 4080 has 16 GB. And it looks like this model is designed for 16 GB cards, so my 3060 is even more outdated now with this model 😂

GiteaMirror commented

@light-and-ray commented on GitHub (Aug 5, 2025):

Okay, now it works with cuda. You just need to have llama.cpp of version 1.44+

My RTX 3060 speed with 22/24 layers loaded: 19-20 tok/sec (cuda), 16-17 tok/sec (vulkan)

@light-and-ray commented on GitHub (Aug 5, 2025): Okay, now it works with cuda. You just need to have llama.cpp of version 1.44+ My RTX 3060 speed with 22/24 layers loaded: 19-20 tok/sec (cuda), 16-17 tok/sec (vulkan)

GiteaMirror commented

2025-11-11 16:40:28 -06:00

@light-and-ray commented on GitHub (Aug 5, 2025):

I assume that 4080 has around 60 tok/sec, right? Or more? @enmanuelmag

Unfortunately whatever GPU you buy, you wont be able to run something significantly better. I thought about spending more money, but even if I buy RTX 5090 I won't be able to run even Gemma 3 27B at FP8, not talking about GPT-OSS 120B

@light-and-ray commented on GitHub (Aug 5, 2025): I assume that 4080 has around 60 tok/sec, right? Or more? @enmanuelmag Unfortunately whatever GPU you buy, you wont be able to run something significantly better. I thought about spending more money, but even if I buy RTX 5090 I won't be able to run even Gemma 3 27B at FP8, not talking about GPT-OSS 120B

GiteaMirror commented

2025-11-11 16:40:28 -06:00

@enmanuelmag commented on GitHub (Aug 5, 2025):

@light-and-ray Actually, I saw performance up to 120-200 tokens per second on models like gemma3-12b and qwen-14b. On gpt model seems be even more optimized because i saw this (request generated from openwebui, still facing same error that I mentioned before on this issue):

2025-08-05 15:43:01 [DEBUG]
Target model llama_perf stats:
  llama_perf_context_print:        load time =    5627.42 ms
  llama_perf_context_print: prompt eval time =     180.78 ms /   367 tokens (    0.49 ms per token,  2030.11 tokens per second)
  llama_perf_context_print:        eval time =     296.84 ms /    44 runs   (    6.75 ms per token,   148.23 tokens per second)
  llama_perf_context_print:       total time =     507.80 ms /   411 tokens
  llama_perf_context_print:    graphs reused =         42

@enmanuelmag commented on GitHub (Aug 5, 2025): @light-and-ray Actually, I saw performance up to 120-200 tokens per second on models like gemma3-12b and qwen-14b. On gpt model seems be even more optimized because i saw this (request generated from openwebui, still facing same error that I mentioned before on this issue): ``` 2025-08-05 15:43:01 [DEBUG] Target model llama_perf stats: llama_perf_context_print: load time = 5627.42 ms llama_perf_context_print: prompt eval time = 180.78 ms / 367 tokens ( 0.49 ms per token, 2030.11 tokens per second) llama_perf_context_print: eval time = 296.84 ms / 44 runs ( 6.75 ms per token, 148.23 tokens per second) llama_perf_context_print: total time = 507.80 ms / 411 tokens llama_perf_context_print: graphs reused = 42 ```

GiteaMirror commented

2025-11-11 16:40:29 -06:00

@light-and-ray commented on GitHub (Aug 5, 2025):

Wow, nice! This speed is actually worth the money

@light-and-ray commented on GitHub (Aug 5, 2025): Wow, nice! This speed is actually worth the money

GiteaMirror commented

2025-11-11 16:40:29 -06:00

@vekovius commented on GitHub (Aug 5, 2025):

I'm experiencing the exact same issue with the newly released OpenAI open-source reasoning models (released today, August 5, 2025). This appears to be affecting multiple new OpenAI reasoning models, including the gpt-oss-20b mentioned in the original issue.

My Setup

Backend: LM Studio (local model serving)
Frontend: Open WebUI v0.6.18 connected via OpenAI-compatible API
Models: New OpenAI open-source reasoning models
Same behavior: Works perfectly in LM Studio native interface, broken in Open WebUI

Token Structure I'm Seeing

The raw reasoning tokens being displayed follow this pattern:

<|channel|>analysis<|message|>User says "hi" repeatedly. We should respond warmly and ask what they want.<|end|><|start|>assistant<|channel|>final<|message|>Hey! 😊 Anything fun or interesting you'd like to chat about, test, or explore right now?

Expected vs Actual

Expected: Clean reasoning display like DeepSeek R1 (which works beautifully in Open WebUI with its collapsible reasoning dropdown - looks absolutely awesome!)
Actual: All the internal reasoning tokens displayed directly to user

Key Observation

Since the same models work correctly in LM Studio's interface (reasoning properly hidden/formatted), this confirms it's specifically an Open WebUI token parsing issue rather than a model output problem.

DeepSeek R1 comparison: Open WebUI already handles DeepSeek R1's reasoning perfectly with that clean, collapsible UI component. We'd love to see the same elegant treatment for these new OpenAI open-source reasoning models!

Impact

With these being OpenAI's new open-source reasoning models, they'll likely see significant adoption in the self-hosted/local model community. This issue affects the core user experience for what will probably become commonly used reasoning models in Open WebUI installations.

This seems to be a broader issue with Open WebUI not recognizing the new reasoning token format that multiple new OpenAI open-source models are using. Would love to help test any fixes!

@vekovius commented on GitHub (Aug 5, 2025): I'm experiencing the exact same issue with the newly released **OpenAI open-source reasoning models** (released today, August 5, 2025). This appears to be affecting multiple new OpenAI reasoning models, including the gpt-oss-20b mentioned in the original issue. ## My Setup - **Backend**: LM Studio (local model serving) - **Frontend**: Open WebUI v0.6.18 connected via OpenAI-compatible API - **Models**: New OpenAI open-source reasoning models - **Same behavior**: Works perfectly in LM Studio native interface, broken in Open WebUI ## Token Structure I'm Seeing The raw reasoning tokens being displayed follow this pattern: ``` <|channel|>analysis<|message|>User says "hi" repeatedly. We should respond warmly and ask what they want.<|end|><|start|>assistant<|channel|>final<|message|>Hey! 😊 Anything fun or interesting you'd like to chat about, test, or explore right now? ``` ## Expected vs Actual - **Expected**: Clean reasoning display like **DeepSeek R1** (which works beautifully in Open WebUI with its collapsible reasoning dropdown - looks absolutely awesome!) - **Actual**: All the internal reasoning tokens displayed directly to user ## Key Observation Since the same models work correctly in LM Studio's interface (reasoning properly hidden/formatted), this confirms it's specifically an Open WebUI token parsing issue rather than a model output problem. **DeepSeek R1 comparison**: Open WebUI already handles DeepSeek R1's reasoning perfectly with that clean, collapsible UI component. We'd love to see the same elegant treatment for these new OpenAI open-source reasoning models! ## Impact With these being OpenAI's new open-source reasoning models, they'll likely see significant adoption in the self-hosted/local model community. This issue affects the core user experience for what will probably become commonly used reasoning models in Open WebUI installations. This seems to be a broader issue with Open WebUI not recognizing the new reasoning token format that multiple new OpenAI open-source models are using. Would love to help test any fixes!

GiteaMirror commented

https://openwebui.com/f/sub01/openai_think_patcher

@Sub0X commented on GitHub (Aug 5, 2025):

I’ve just wrote a quick temporary pipe/function that solves this issue of GPT OSS's different token formatting for now until openwebui adds it natively! Tool calls aren’t tested yet but normal conversations should appear fine! (Currently having GPT OSS 20B running on my Macbook and 120B running on Linux both via LM Studio)

@Sub0X commented on GitHub (Aug 5, 2025): I’ve just wrote a quick temporary pipe/function that solves this issue of GPT OSS's different token formatting for now until openwebui adds it natively! Tool calls aren’t tested yet but normal conversations should appear fine! (Currently having GPT OSS 20B running on my Macbook and 120B running on Linux both via LM Studio) https://openwebui.com/f/sub01/openai_think_patcher <img width="2234" height="1818" alt="Image" src="https://github.com/user-attachments/assets/08f57b7d-14a2-4a59-8613-a0dbd8e45172" /> <img width="2207" height="1807" alt="Image" src="https://github.com/user-attachments/assets/168d5733-21b0-4bdf-a4d6-8581556685ca" />

GiteaMirror commented

@swein commented on GitHub (Aug 5, 2025):

I see the same issue, like jinja harmony based template?

@swein commented on GitHub (Aug 5, 2025): I see the same issue, like jinja harmony based template?

GiteaMirror commented

https://github.com/openai/harmony

@gaby commented on GitHub (Aug 6, 2025):

Basically Open-WebUI needs to add support for harmony response format. It's a python dependency:

I have no idea how to integrate this into Open-WebUI, else I would submit a PR.

@gaby commented on GitHub (Aug 6, 2025): Basically `Open-WebUI` needs to add support for `harmony response format`. It's a python dependency: https://github.com/openai/harmony I have no idea how to integrate this into `Open-WebUI`, else I would submit a PR.

GiteaMirror commented

@light-and-ray commented on GitHub (Aug 6, 2025):

I have no idea how to integrate this into Open-WebUI, else I would submit a PR.

I heard that it's supported via OpenAI API. But I'm not sure in general

@light-and-ray commented on GitHub (Aug 6, 2025): > I have no idea how to integrate this into Open-WebUI, else I would submit a PR. I heard that it's supported via OpenAI API. But I'm not sure in general

GiteaMirror commented

@gaby commented on GitHub (Aug 6, 2025):

@light-and-ray Pretty sure that OpenWebUI has to support the format too to be able to render it

@gaby commented on GitHub (Aug 6, 2025): @light-and-ray Pretty sure that OpenWebUI has to support the format too to be able to render it

GiteaMirror commented

@light-and-ray commented on GitHub (Aug 6, 2025):

@gaby
This is the field I heard of: https://platform.openai.com/docs/guides/reasoning?api-mode=responses

from openai import OpenAI

client = OpenAI()

prompt = """
Write a bash script that takes a matrix represented as a string with 
format '[1,2],[3,4],[5,6]' and prints the transpose in the same format.
"""

response = client.responses.create(
    model="o4-mini",
    reasoning={"effort": "medium"},
    input=[
        {
            "role": "user", 
            "content": prompt
        }
    ]
)

print(response.output_text)

@light-and-ray commented on GitHub (Aug 6, 2025): @gaby This is the field I heard of: https://platform.openai.com/docs/guides/reasoning?api-mode=responses > > ```python > from openai import OpenAI > > client = OpenAI() > > prompt = """ > Write a bash script that takes a matrix represented as a string with > format '[1,2],[3,4],[5,6]' and prints the transpose in the same format. > """ > > response = client.responses.create( > model="o4-mini", > reasoning={"effort": "medium"}, > input=[ > { > "role": "user", > "content": prompt > } > ] > ) > > print(response.output_text) > ```

GiteaMirror commented

@gaby commented on GitHub (Aug 6, 2025):

@gaby This is the field I heard of: https://platform.openai.com/docs/guides/reasoning?api-mode=responses

from openai import OpenAI

client = OpenAI()

prompt = """
Write a bash script that takes a matrix represented as a string with
format '[1,2],[3,4],[5,6]' and prints the transpose in the same format.
"""

response = client.responses.create(
model="o4-mini",
reasoning={"effort": "medium"},
input=[
{
"role": "user",
"content": prompt
}
]
)

print(response.output_text)

Different thing, the new models state they need openai-harmony according to the Model Card.

These are not o3/o4

@gaby commented on GitHub (Aug 6, 2025): > [@gaby](https://github.com/gaby) This is the field I heard of: https://platform.openai.com/docs/guides/reasoning?api-mode=responses > > > from openai import OpenAI > > > > client = OpenAI() > > > > prompt = """ > > Write a bash script that takes a matrix represented as a string with > > format '[1,2],[3,4],[5,6]' and prints the transpose in the same format. > > """ > > > > response = client.responses.create( > > model="o4-mini", > > reasoning={"effort": "medium"}, > > input=[ > > { > > "role": "user", > > "content": prompt > > } > > ] > > ) > > > > print(response.output_text) Different thing, the new models state they need openai-harmony according to the Model Card. These are not o3/o4

GiteaMirror commented

2025-11-11 16:40:32 -06:00

@danielfl commented on GitHub (Aug 6, 2025):

Also Ollama loads these models 100% CPU even when GPU is available when called from Open WebUI, but works correctly calling directly via Ollama CLI

I’ve experienced the same here, using Ubuntu + an AMD RX 7900 XTX. The CLI works and is very fast. If I send "Hi" on the web UI, the model unloads from the GPU and loads again into the RAM.

PS: I am not experiencing it using other models.

@danielfl commented on GitHub (Aug 6, 2025): > Also Ollama loads these models 100% CPU even when GPU is available when called from Open WebUI, but works correctly calling directly via Ollama CLI I’ve experienced the same here, using Ubuntu + an AMD RX 7900 XTX. The CLI works and is very fast. If I send "Hi" on the web UI, the model unloads from the GPU and loads again into the RAM. PS: I am not experiencing it using other models.

GiteaMirror commented

@beavis82 commented on GitHub (Aug 6, 2025):

I was experiencing the same problem with LM Studio, this was a bug in the LMStudio API which has been fixed in version 0.3.22. Update to the latest version which was released a few hours ago. You may also need to go to LM Studio App Settings (gear icon in bottom right of LM Studio), click the Developer tab, then enable "When applicable, separate reasoning_content and content in API responses". Doing this resolved the problem for me, hopefully it will work for you.

@beavis82 commented on GitHub (Aug 6, 2025): I was experiencing the same problem with LM Studio, this was a bug in the LMStudio API which has been fixed in version 0.3.22. Update to the latest version which was released a few hours ago. You may also need to go to LM Studio App Settings (gear icon in bottom right of LM Studio), click the Developer tab, then enable "When applicable, separate reasoning_content and content in API responses". Doing this resolved the problem for me, hopefully it will work for you.

GiteaMirror commented

2025-11-11 16:40:32 -06:00

@tjbck commented on GitHub (Aug 6, 2025):

Seems to have been addressed with the latest lmstudio, testing wanted here!

@tjbck commented on GitHub (Aug 6, 2025): Seems to have been addressed with the latest lmstudio, testing wanted here! <img width="1070" height="467" alt="Image" src="https://github.com/user-attachments/assets/6059bb88-7641-476d-93bd-f1b3dff40544" />

GiteaMirror commented

2025-11-11 16:40:32 -06:00

@danielfl commented on GitHub (Aug 6, 2025):

I am using Open WebUI 0.6.18 (created a conda env and pip installed it) with Ollama 0.11.0 backend. I am referencing Shawneau problem with Ollama and I am suspecting the issue is in the ollama api or ollama itself so I just made a comment there as well.

@danielfl commented on GitHub (Aug 6, 2025): I am using Open WebUI 0.6.18 (created a conda env and pip installed it) with Ollama 0.11.0 backend. I am referencing [Shawneau](https://github.com/Shawneau) problem with Ollama and I am suspecting the issue is in the ollama api or ollama itself so I just made a comment there as well.

GiteaMirror commented

2025-11-11 16:40:33 -06:00

@enmanuelmag commented on GitHub (Aug 6, 2025):

Seems to have been addressed with the latest lmstudio, testing wanted here!

I just tested, LM Studio 0.3.22 and Open WebUI v0.6.18 but still facing the same error:

@enmanuelmag commented on GitHub (Aug 6, 2025): > Seems to have been addressed with the latest lmstudio, testing wanted here! > > <img alt="Image" width="1070" height="467" src="https://private-user-images.githubusercontent.com/25473318/474952768-6059bb88-7641-476d-93bd-f1b3dff40544.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTQ0ODM2OTIsIm5iZiI6MTc1NDQ4MzM5MiwicGF0aCI6Ii8yNTQ3MzMxOC80NzQ5NTI3NjgtNjA1OWJiODgtNzY0MS00NzZkLTkzYmQtZjFiM2RmZjQwNTQ0LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA4MDYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwODA2VDEyMjk1MlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPThmZmJlODAxMDQwMWFiNmY5MzNmNzZiMzgzZThmNmNkODljOGRmODI4ZTkwOWI5NTg2MzdiMmY4YWE4Yzk0OTImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.N9SYmXWxUcd1Rr0mClX1UbT2MzGP63fBNHCMwoT0PLY"> I just tested, LM Studio 0.3.22 and Open WebUI v0.6.18 but still facing the same error: <img width="1424" height="398" alt="Image" src="https://github.com/user-attachments/assets/5ea5acb6-98f7-4749-b3cc-aaf7b2f5dfe5" />

GiteaMirror commented

2025-11-11 16:40:33 -06:00

@tjbck commented on GitHub (Aug 6, 2025):

@enmanuelmag could you share the exact step by steps to reproduce? the more detailed the better.

@tjbck commented on GitHub (Aug 6, 2025): @enmanuelmag could you share the exact step by steps to reproduce? the more detailed the better.

GiteaMirror commented

2025-11-11 16:40:33 -06:00

@light-and-ray commented on GitHub (Aug 6, 2025):

I updated LM-Studio, and now it doesn't show tags, but also it doesn't show reasoning block either

@light-and-ray commented on GitHub (Aug 6, 2025): I updated LM-Studio, and now it doesn't show tags, but also it doesn't show reasoning block either <img width="2476" height="1588" alt="Image" src="https://github.com/user-attachments/assets/f5f63bcf-e1d6-4912-a22e-c2ed756193e6" />

GiteaMirror commented

2025-11-11 16:40:35 -06:00

@light-and-ray commented on GitHub (Aug 6, 2025):

Seems to have been addressed with the latest lmstudio, testing wanted here!

Open webui also needs "reasoning effort"/"reasoning level" switch

@light-and-ray commented on GitHub (Aug 6, 2025): > Seems to have been addressed with the latest lmstudio, testing wanted here! Open webui also needs "reasoning effort"/"reasoning level" switch <img width="746" height="336" alt="Image" src="https://github.com/user-attachments/assets/07dc38e1-31b1-4317-ad28-77605792f31a" /> <img width="1386" height="678" alt="Image" src="https://github.com/user-attachments/assets/429b6dde-d366-49f2-b73f-cb5a4d7fe101" />

GiteaMirror commented

@enmanuelmag commented on GitHub (Aug 6, 2025):

@enmanuelmag could you share the exact step by steps to reproduce? the more detailed the better.

As mentioned on Issue details:

Install open web ui via pip
Install gpt model on LM Studio and serve it
Connect open webui to LM Studio server using OpenAI connection
Select the gpt model to chat
Sent a prompt and see the response

@enmanuelmag commented on GitHub (Aug 6, 2025): > @enmanuelmag could you share the exact step by steps to reproduce? the more detailed the better. As mentioned on Issue details: 1. Install open web ui via pip 2. Install gpt model on LM Studio and serve it 3. Connect open webui to LM Studio server using OpenAI connection 4. Select the gpt model to chat 5. Sent a prompt and see the response

GiteaMirror commented

@beavis82 commented on GitHub (Aug 6, 2025):

I updated LM-Studio, and now it doesn't show tags, but also it doesn't show reasoning block either

I needed to enable the following setting in LM Studio to resolve this:

In LM Studio, go to App Settings (gear icon in bottom right of LM Studio), click the Developer tab, then enable "When applicable, separate reasoning_content and content in API responses".

@beavis82 commented on GitHub (Aug 6, 2025): > I updated LM-Studio, and now it doesn't show tags, but also it doesn't show reasoning block either > > <img alt="Image" width="2000" height="1588" src="https://private-user-images.githubusercontent.com/33491867/475114217-f5f63bcf-e1d6-4912-a22e-c2ed756193e6.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTQ0OTgxNjgsIm5iZiI6MTc1NDQ5Nzg2OCwicGF0aCI6Ii8zMzQ5MTg2Ny80NzUxMTQyMTctZjVmNjNiY2YtZTFkNi00OTEyLWEyMmUtYzJlZDc1NjE5M2U2LnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA4MDYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwODA2VDE2MzEwOFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTZmN2FlYTg0MWFiNjYzODhmMmY0MDRhYjRhMmFkYTc0YTQyYzU0NjFmY2QzNWUzMDdhM2JjZGIzYTY5NGZlYmImWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.qlFCElGYes6hG6Cu_yb_hP_WSudTGYsi67bBk1VBpYc"> I needed to enable the following setting in LM Studio to resolve this: In LM Studio, go to App Settings (gear icon in bottom right of LM Studio), click the Developer tab, then enable "When applicable, separate reasoning_content and content in API responses".

GiteaMirror commented

@beavis82 commented on GitHub (Aug 6, 2025):

Seems to have been addressed with the latest lmstudio, testing wanted here!

Open webui also needs "reasoning effort"/"reasoning level" switch

In Open WebUI it's possible to set the reasoning effort in Model Params > Advanced Params.

Set up a model in Workspace > Models using GPT-OSS as the base model, then in Model Params, select Advanced Params and scroll down to Reasoning Effort.

I'm not sure how effective this is though, I've not noticed any difference when changing this setting.

@beavis82 commented on GitHub (Aug 6, 2025): > > Seems to have been addressed with the latest lmstudio, testing wanted here! > > Open webui also needs "reasoning effort"/"reasoning level" switch > > <img alt="Image" width="746" height="336" src="https://private-user-images.githubusercontent.com/33491867/475114911-07dc38e1-31b1-4317-ad28-77605792f31a.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTQ0OTgxNjgsIm5iZiI6MTc1NDQ5Nzg2OCwicGF0aCI6Ii8zMzQ5MTg2Ny80NzUxMTQ5MTEtMDdkYzM4ZTEtMzFiMS00MzE3LWFkMjgtNzc2MDU3OTJmMzFhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA4MDYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwODA2VDE2MzEwOFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTU2ZjdhODVhZmQ4NWU4M2E2ZDhlZmIzMjZiNjk0MWI5MTViMGI0MzA3NTcyYzdlZTc1NjQ1YjlhOWM1NGNmZmUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.QvXugjzGrhPfQH-8SB8Ume4gMLCpFxsw5sj5_OHp8wA"> <img alt="Image" width="1386" height="678" src="https://private-user-images.githubusercontent.com/33491867/475115388-429b6dde-d366-49f2-b73f-cb5a4d7fe101.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTQ0OTgxNjgsIm5iZiI6MTc1NDQ5Nzg2OCwicGF0aCI6Ii8zMzQ5MTg2Ny80NzUxMTUzODgtNDI5YjZkZGUtZDM2Ni00OWYyLWI3M2YtY2I1YTRkN2ZlMTAxLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA4MDYlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwODA2VDE2MzEwOFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTU1ZWRjYzhlNmFkNzMwZTE0Y2JiMzI4MGI5MTdjYzM0NmJlYmYzODU2MjIzNzY2NmQwYWMxOTgyMGRjNzczODgmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.9YfsmP4MGfQKjLVv6OVsJvaVMpL4CiNIT2lNGzeIHz4"> In Open WebUI it's possible to set the reasoning effort in Model Params > Advanced Params. Set up a model in Workspace > Models using GPT-OSS as the base model, then in Model Params, select Advanced Params and scroll down to Reasoning Effort. I'm not sure how effective this is though, I've not noticed any difference when changing this setting.

GiteaMirror commented

@enmanuelmag commented on GitHub (Aug 6, 2025):

Update

Only show tags when model try to call tool, for normal chat works good. The last message on screenshot was sent with Web search tool enabled (tool native from Open WebUI, also fails with external tool loaded from MCPO server)

@enmanuelmag commented on GitHub (Aug 6, 2025): ### Update Only show tags when model try to call tool, for normal chat works good. The last message on screenshot was sent with `Web search` tool enabled (tool native from Open WebUI, also fails with external tool loaded from MCPO server) <img width="1375" height="1089" alt="Image" src="https://github.com/user-attachments/assets/ac250839-2a9d-480a-835d-0fe471860a12" />

GiteaMirror commented

@light-and-ray commented on GitHub (Aug 6, 2025):

I updated LM-Studio, and now it doesn't show tags, but also it doesn't show reasoning block either

I needed to enable the following setting in LM Studio to resolve this:

In LM Studio, go to App Settings (gear icon in bottom right of LM Studio), click the Developer tab, then enable "When applicable, separate reasoning_content and content in API responses".

Thanks, it works now

@light-and-ray commented on GitHub (Aug 6, 2025): > > I updated LM-Studio, and now it doesn't show tags, but also it doesn't show reasoning block either > > I needed to enable the following setting in LM Studio to resolve this: > > In LM Studio, go to App Settings (gear icon in bottom right of LM Studio), click the Developer tab, then enable "When applicable, separate reasoning_content and content in API responses". Thanks, it works now

GiteaMirror commented

@light-and-ray commented on GitHub (Aug 6, 2025):

I'm not sure how effective this is though, I've not noticed any difference when changing this setting.

Me either, I tried high and low in the demo, and the results are very similar

@light-and-ray commented on GitHub (Aug 6, 2025): > I'm not sure how effective this is though, I've not noticed any difference when changing this setting. Me either, I tried high and low in the demo, and the results are very similar

GiteaMirror commented

@light-and-ray commented on GitHub (Aug 6, 2025):

I've found one more bug. If you stop generation, then continue, it starts reasoning again in the middle of the answer

@light-and-ray commented on GitHub (Aug 6, 2025): I've found one more bug. If you stop generation, then continue, it starts reasoning again in the middle of the answer ![Screenshot_2025-08-06-20-50-12-045_org.mozilla.firefox-edit.jpg](https://github.com/user-attachments/assets/dd86bcb7-2c85-4e76-bf81-1d738a80c6db)

GiteaMirror commented

2025-11-11 16:40:38 -06:00

@enmanuelmag commented on GitHub (Aug 7, 2025):

Update

Now works successfully with models server from LM Studio (v0.3.22 Build 2). Please check on your envs to confirm this.

Take in mind that surely Open WebUI still requiring a parsing pipe or integrate Harmony parser to be able to render messages that no sent from LM Studio.

Disclaimer:

Seems that still throw same error if no tools enables (web search) and tried to make a call (maybe a RAG request)

@enmanuelmag commented on GitHub (Aug 7, 2025): ### Update Now works successfully with models server from LM Studio (v0.3.22 Build 2). Please check on your envs to confirm this. Take in mind that surely Open WebUI still requiring a parsing pipe or integrate Harmony parser to be able to render messages that no sent from LM Studio. <img width="1488" height="937" alt="Image" src="https://github.com/user-attachments/assets/c5f3f47b-4c72-4637-81c8-7a4d2d77900e" /> Disclaimer: Seems that still throw same error if no tools enables (web search) and tried to make a call (maybe a RAG request) <img width="1360" height="257" alt="Image" src="https://github.com/user-attachments/assets/0b3b10be-7a15-4713-ae1b-e7d1476dc647" />

GiteaMirror commented

@tblattner commented on GitHub (Aug 7, 2025):

It would be good if OWUI supported/shifted to the responses API. I have had limited success with tool calling and the new models with chat completions.

To experiment, I created a functional chatbot that interacts with vLLM and the new OSS models.

Here is a snippet for processing the responses using the openai API, with tool calling using fastmcp:

async def get_llm_response_native(client: openai.OpenAI, fastmcp_client: Client, messages: list[dict], model: str, tools: list):
    last_response = None
    response_text = ''
    
    while response_text == '':
        try:            
            last_response = client.responses.create(
                model=model,
                input=messages,
                stream=False,
                reasoning={'effort': 'high'},
                tools=tools
            )
                          
            response_text = last_response.output_text
            
            for response in last_response.output:                
                if isinstance(response, ResponseReasoningItem):
                    messages.append(response)
                    print('Reasoning: {}'.format(response.content[0].text))   
                                                         
                elif isinstance(response, ResponseOutputMessage):
                    response_message = { 'role': 'assistant' }                    
                    response_message['content'] = response.content[0].text                      
                    messages.append(response_message)

                elif isinstance(response, ResponseFunctionToolCall):
                    messages.append(response)
                    if response.type == 'function_call':
                        function_name = response.name
                        function_args = json.loads(response.arguments)                        
                        
                        print(f"Tool call: {function_name} with args: {function_args}")
                        tool_result = await fastmcp_client.call_tool(function_name, function_args)
                        print(f"Tool result: {tool_result.structured_content}")
                        
                        # Add response message
                        response_message = {
                            'type': 'function_call_output',
                            'call_id': response.call_id,
                            'output': tool_result.data                            
                        }
                        messages.append(response_message)                                
                    else:
                        print(f"Unexpected response type: {response.type}")
                        continue
                                        
                    
                else:
                    messages.append(response)
                    print(f"Unexpected response type: {type(response)}")
            
            
        except Exception as e:
            print(f"An error occurred while calling the LLM: {e}")
            raise e
            
    return response_text, messages

Another nice feature of the responses API is it specifies a "previous_response_id" (https://platform.openai.com/docs/guides/conversation-state?api-mode=responses#openai-apis-for-conversation-state). Although vllm does not currently provide a good solution for this yet: https://github.com/vllm-project/vllm/blob/139d155781c187b6d38ac6d84a516c97ff66bb1f/vllm/entrypoints/openai/serving_responses.py#L115

@tblattner commented on GitHub (Aug 7, 2025): It would be good if OWUI supported/shifted to the responses API. I have had limited success with tool calling and the new models with chat completions. To experiment, I created a functional chatbot that interacts with vLLM and the new OSS models. Here is a snippet for processing the responses using the openai API, with tool calling using fastmcp: ``` async def get_llm_response_native(client: openai.OpenAI, fastmcp_client: Client, messages: list[dict], model: str, tools: list): last_response = None response_text = '' while response_text == '': try: last_response = client.responses.create( model=model, input=messages, stream=False, reasoning={'effort': 'high'}, tools=tools ) response_text = last_response.output_text for response in last_response.output: if isinstance(response, ResponseReasoningItem): messages.append(response) print('Reasoning: {}'.format(response.content[0].text)) elif isinstance(response, ResponseOutputMessage): response_message = { 'role': 'assistant' } response_message['content'] = response.content[0].text messages.append(response_message) elif isinstance(response, ResponseFunctionToolCall): messages.append(response) if response.type == 'function_call': function_name = response.name function_args = json.loads(response.arguments) print(f"Tool call: {function_name} with args: {function_args}") tool_result = await fastmcp_client.call_tool(function_name, function_args) print(f"Tool result: {tool_result.structured_content}") # Add response message response_message = { 'type': 'function_call_output', 'call_id': response.call_id, 'output': tool_result.data } messages.append(response_message) else: print(f"Unexpected response type: {response.type}") continue else: messages.append(response) print(f"Unexpected response type: {type(response)}") except Exception as e: print(f"An error occurred while calling the LLM: {e}") raise e return response_text, messages ``` Another nice feature of the responses API is it specifies a "previous_response_id" (https://platform.openai.com/docs/guides/conversation-state?api-mode=responses#openai-apis-for-conversation-state). Although vllm does not currently provide a good solution for this yet: https://github.com/vllm-project/vllm/blob/139d155781c187b6d38ac6d84a516c97ff66bb1f/vllm/entrypoints/openai/serving_responses.py#L115

GiteaMirror commented

2025-11-11 16:40:38 -06:00

@tristan-k commented on GitHub (Aug 8, 2025):

I’ve just wrote a quick temporary pipe/function that solves this issue of GPT OSS's different token formatting for now until openwebui adds it natively! Tool calls aren’t tested yet but normal conversations should appear fine! (Currently having GPT OSS 20B running on my Macbook and 120B running on Linux both via LM Studio)

@Sub0X Can you do a version which is compatible with llama.cpp. If I use your filter the reasoning is still part of the answer:

@tristan-k commented on GitHub (Aug 8, 2025): > I’ve just wrote a quick temporary pipe/function that solves this issue of GPT OSS's different token formatting for now until openwebui adds it natively! Tool calls aren’t tested yet but normal conversations should appear fine! (Currently having GPT OSS 20B running on my Macbook and 120B running on Linux both via LM Studio) @Sub0X Can you do a version which is compatible with `llama.cpp`. If I use your filter the reasoning is still part of the answer: <img width="1062" height="1002" alt="Image" src="https://github.com/user-attachments/assets/6c10d077-c599-4314-b4e3-16760f2bc4d2" />

GiteaMirror commented

2025-11-11 16:40:38 -06:00

@light-and-ray commented on GitHub (Aug 8, 2025):

@tristan-k the problem is similar to my problem when separate reasoning stream was disabled in LM-Studio. Maybe you have the same problem. I'm curious, does llama.cpp itself have open-ai compatible server, or what do you mean?

@light-and-ray commented on GitHub (Aug 8, 2025): @tristan-k the problem is similar to my problem when separate reasoning stream was disabled in LM-Studio. Maybe you have the same problem. I'm curious, does llama.cpp itself have open-ai compatible server, or what do you mean?

GiteaMirror commented

@tristan-k commented on GitHub (Aug 8, 2025):

Yes, it does have a OpenAI API compatible server but to my knowledge it does not have a built‑in way to "split" the model’s internal reasoning from the final answer.

@tristan-k commented on GitHub (Aug 8, 2025): Yes, it does have a OpenAI API compatible server but to my knowledge it does not have a built‑in way to "split" the model’s internal reasoning from the final answer.

GiteaMirror commented

@tblattner commented on GitHub (Aug 8, 2025):

Yes, it does have a OpenAI API compatible server but to my knowledge it does not have a built‑in way to "split" the model’s internal reasoning from the final answer.

The new responses API implementation from OpenAI does split using "ResponseReasingItem" versus "ResponseOutputMessage" in the output from the client.responses. Also includes a component for tool calling as well. There's a boat load of types that they added: https://github.com/openai/openai-python/blob/main/api.md#responses Here's a snippet that I wrote that handles a few of them:

            last_response = client.responses.create(
                model=model,
                input=messages,
                stream=False,
                reasoning={'effort': 'high'},
                tools=tools
            )
                          
            response_text = last_response.output_text
            
            for response in last_response.output:                
                if isinstance(response, ResponseReasoningItem):
                    messages.append(response)
                    print('Reasoning: {}'.format(response.content[0].text))   
                                                         
                elif isinstance(response, ResponseOutputMessage):
                    response_message = { 'role': 'assistant' }                    
                    response_message['content'] = response.content[0].text                      
                    messages.append(response_message)

                elif isinstance(response, ResponseFunctionToolCall):
                    messages.append(response)
                    if response.type == 'function_call':
                        function_name = response.name
                        function_args = json.loads(response.arguments)                        
                        
                        print(f"Tool call: {function_name} with args: {function_args}")
                        tool_result = await fastmcp_client.call_tool(function_name, function_args)
                        print(f"Tool result: {tool_result.structured_content}")
                        
                        # Add response message
                        response_message = {
                            'type': 'function_call_output',
                            'call_id': response.call_id,
                            'output': tool_result.data                            
                        }
                        messages.append(response_message)                                
                    else:
                        print(f"Unexpected response type: {response.type}")
                        continue
                                        
                    
                else:
                    messages.append(response)
                    print(f"Unexpected response type: {type(response)}")

@tblattner commented on GitHub (Aug 8, 2025): > Yes, it does have a OpenAI API compatible server but to my knowledge it does not have a built‑in way to "split" the model’s internal reasoning from the final answer. The new responses API implementation from OpenAI does split using "ResponseReasingItem" versus "ResponseOutputMessage" in the output from the client.responses. Also includes a component for tool calling as well. There's a boat load of types that they added: https://github.com/openai/openai-python/blob/main/api.md#responses Here's a snippet that I wrote that handles a few of them: ``` last_response = client.responses.create( model=model, input=messages, stream=False, reasoning={'effort': 'high'}, tools=tools ) response_text = last_response.output_text for response in last_response.output: if isinstance(response, ResponseReasoningItem): messages.append(response) print('Reasoning: {}'.format(response.content[0].text)) elif isinstance(response, ResponseOutputMessage): response_message = { 'role': 'assistant' } response_message['content'] = response.content[0].text messages.append(response_message) elif isinstance(response, ResponseFunctionToolCall): messages.append(response) if response.type == 'function_call': function_name = response.name function_args = json.loads(response.arguments) print(f"Tool call: {function_name} with args: {function_args}") tool_result = await fastmcp_client.call_tool(function_name, function_args) print(f"Tool result: {tool_result.structured_content}") # Add response message response_message = { 'type': 'function_call_output', 'call_id': response.call_id, 'output': tool_result.data } messages.append(response_message) else: print(f"Unexpected response type: {response.type}") continue else: messages.append(response) print(f"Unexpected response type: {type(response)}") ```

GiteaMirror commented

@tristan-k commented on GitHub (Aug 8, 2025):

The new responses API implementation from OpenAI does split using "ResponseReasingItem" versus "ResponseOutputMessage" in the output from the client.responses. Also includes a component for tool calling as well. There's a boat load of types that they added: https://github.com/openai/openai-python/blob/main/api.md#responses

So essentially llama.cpp does yet have to adapt the new reasoning additions in the OpenAI API spec to it's server.

@tristan-k commented on GitHub (Aug 8, 2025): > The new responses API implementation from OpenAI does split using "ResponseReasingItem" versus "ResponseOutputMessage" in the output from the client.responses. Also includes a component for tool calling as well. There's a boat load of types that they added: https://github.com/openai/openai-python/blob/main/api.md#responses So essentially llama.cpp does yet have to adapt the new reasoning additions in the OpenAI API spec to it's server.

GiteaMirror commented

@tblattner commented on GitHub (Aug 8, 2025):

So essentially llama.cpp does yet have to adapt the new reasoning additions in the OpenAI API spec to it's server.

Forgot to mention that I am using vLLM as the provider. But yea ideally all the backend providers would support more of these types of features.

Edit:
vLLM is using the harmony library: https://github.com/openai/harmony

@tblattner commented on GitHub (Aug 8, 2025): > So essentially llama.cpp does yet have to adapt the new reasoning additions in the OpenAI API spec to it's server. Forgot to mention that I am using vLLM as the provider. But yea ideally all the backend providers would support more of these types of features. Edit: vLLM is using the harmony library: https://github.com/openai/harmony

GiteaMirror commented

@Startrek666 commented on GitHub (Aug 10, 2025):

I’ve just wrote a quick temporary pipe/function that solves this issue of GPT OSS's different token formatting for now until openwebui adds it natively! Tool calls aren’t tested yet but normal conversations should appear fine! (Currently having GPT OSS 20B running on my Macbook and 120B running on Linux both via LM Studio)

https://openwebui.com/f/sub01/openai_think_patcher

Hello, I just started using open-webui. I installed the function you wrote, but it only removed those tags and didn't display the thinking area starting with "Thought for...". Could you please tell me how to solve this problem? The version of open-webui I'm using is v0.6.18. Thank you!

@Startrek666 commented on GitHub (Aug 10, 2025): > I’ve just wrote a quick temporary pipe/function that solves this issue of GPT OSS's different token formatting for now until openwebui adds it natively! Tool calls aren’t tested yet but normal conversations should appear fine! (Currently having GPT OSS 20B running on my Macbook and 120B running on Linux both via LM Studio) > > https://openwebui.com/f/sub01/openai_think_patcher > > <img alt="Image" width="2000" height="1818" src="https://private-user-images.githubusercontent.com/74270733/474736502-08f57b7d-14a2-4a59-8613-a0dbd8e45172.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTQ4NDA5OTAsIm5iZiI6MTc1NDg0MDY5MCwicGF0aCI6Ii83NDI3MDczMy80NzQ3MzY1MDItMDhmNTdiN2QtMTRhMi00YTU5LTg2MTMtYTBkYmQ4ZTQ1MTcyLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA4MTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwODEwVDE1NDQ1MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTJhZDk4MGE2YzNhM2ExYTFkYWEwOGUzY2E3OGM2ZjY5YjY5YjBjOTQ3MDE1ZGI3NjJmNDg4NDQwMDU1ZWNkNTcmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.MPjBpIh5s-bnX3NDPWJhL8LWkTak23efebzrpFcTAnM"> <img alt="Image" width="2000" height="1807" src="https://private-user-images.githubusercontent.com/74270733/474737064-168d5733-21b0-4bdf-a4d6-8581556685ca.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3NTQ4NDA5OTAsIm5iZiI6MTc1NDg0MDY5MCwicGF0aCI6Ii83NDI3MDczMy80NzQ3MzcwNjQtMTY4ZDU3MzMtMjFiMC00YmRmLWE0ZDYtODU4MTU1NjY4NWNhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNTA4MTAlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjUwODEwVDE1NDQ1MFomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWVlYTMzZWJmY2ZlZGU2NmU5MWU2ZTgwOGRiZThmNjNlZjhkODlhNDRhNDNkODRjNzA2YThmMWQ2MTUwNmU4Y2YmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0In0.G-Khh4isG0uTg3XuUpmThm1FjNdTK9rJh06VDeKP1PY"> Hello, I just started using open-webui. I installed the function you wrote, but it only removed those tags and didn't display the thinking area starting with "Thought for...". Could you please tell me how to solve this problem? The version of open-webui I'm using is v0.6.18. Thank you! <img width="1710" height="327" alt="Image" src="https://github.com/user-attachments/assets/413c3a0d-abec-4c9a-8cef-803f54df09fb" />

GiteaMirror commented

@gaby commented on GitHub (Aug 10, 2025):

@tblattner I'm using vLLM too and it works fine now. You have to use the gptoss docker image tag.

@gaby commented on GitHub (Aug 10, 2025): @tblattner I'm using vLLM too and it works fine now. You have to use the `gptoss` docker image tag.

GiteaMirror commented

@thr-4 commented on GitHub (Aug 10, 2025):

I'm using Lemonade as server and the thinking tags aren't parsed in Open WebUI:

In Jan.ai, using same server (Lemonade), the response is parsed correcty:

Are there plans to implement corrent parsing in Open WebUI, or does this need to be addressed on server side?

@thr-4 commented on GitHub (Aug 10, 2025): I'm using Lemonade as server and the thinking tags aren't parsed in Open WebUI: <img width="1266" height="265" alt="Image" src="https://github.com/user-attachments/assets/1ff40141-ff17-4c47-8f95-e7e69f75c55f" /> In Jan.ai, using same server (Lemonade), the response is parsed correcty: <img width="529" height="288" alt="Image" src="https://github.com/user-attachments/assets/de97aef6-446d-458f-b62b-70a693289b4e" /> Are there plans to implement corrent parsing in Open WebUI, or does this need to be addressed on server side?

GiteaMirror commented

@tblattner commented on GitHub (Aug 11, 2025):

@tblattner I'm using vLLM too and it works fine now. You have to use the gptoss docker image tag.

I was able to install via a wheel.

uv pip install --pre vllm==0.10.1+gptoss \
    --extra-index-url https://wheels.vllm.ai/gpt-oss/ \
    --extra-index-url https://download.pytorch.org/whl/nightly/cu128 \
    --index-strategy unsafe-best-match

vllm serve openai/gpt-oss-120b

@tblattner commented on GitHub (Aug 11, 2025): > [@tblattner](https://github.com/tblattner) I'm using vLLM too and it works fine now. You have to use the `gptoss` docker image tag. I was able to install via a wheel. ``` uv pip install --pre vllm==0.10.1+gptoss \ --extra-index-url https://wheels.vllm.ai/gpt-oss/ \ --extra-index-url https://download.pytorch.org/whl/nightly/cu128 \ --index-strategy unsafe-best-match vllm serve openai/gpt-oss-120b ```

GiteaMirror commented

@recallmenot commented on GitHub (Aug 13, 2025):

Yeah we need proper support for harmony in open-webui.
The function is nice but with OSS 20b running in llama.cpp, for me, the role tags are removed but the thoughts doesn't get inserted into a "thinking"-block, so I'm currently better-off without it.

@recallmenot commented on GitHub (Aug 13, 2025): Yeah we need proper support for [harmony](https://github.com/openai/harmony) in open-webui. The [function](https://openwebui.com/f/sub01/openai_think_patcher) is nice but with OSS 20b running in llama.cpp, for me, the role tags are removed but the thoughts doesn't get inserted into a "thinking"-block, so I'm currently better-off without it.

GiteaMirror commented

@ghost commented on GitHub (Aug 13, 2025):

I've ditched open-webui for that reason. A chat webui that is copy-pasted on chatgpt that does not work properly with an openai model.....is beyond me!

@ghost commented on GitHub (Aug 13, 2025): I've ditched open-webui for that reason. A chat webui that is copy-pasted on chatgpt that does not work properly with an openai model.....is beyond me!

GiteaMirror commented

@enmanuelmag commented on GitHub (Aug 13, 2025):

@CybergNetwork Open WebUI is not a crude copy. It is an open source project that democratizes and provides space for all of us to have similar and very useful features that different providers have, and some that they don't even have. It gives you a workspace environment, pipeline customization, tools (client and server side), user administration, and more. All from a very simple plug-and-play UX.

To dismiss it just because of this issue is not doing justice to everything this product is.

@enmanuelmag commented on GitHub (Aug 13, 2025): @CybergNetwork Open WebUI is not a crude copy. It is an open source project that democratizes and provides space for all of us to have similar and very useful features that different providers have, and some that they don't even have. It gives you a workspace environment, pipeline customization, tools (client and server side), user administration, and more. All from a very simple plug-and-play UX. To dismiss it just because of this issue is not doing justice to everything this product is.

GiteaMirror commented

@etlweather commented on GitHub (Aug 14, 2025):

@enmanuelmag

I've ditched open-webui for that reason. A chat webui that is copy-pasted on chatgpt that does not work properly with an openai model.....is beyond me!

No need to be mean. How much did you pay for it? How many contributions did you make to open source? I don't see any project under your account.

Just saying....

@etlweather commented on GitHub (Aug 14, 2025): @enmanuelmag > I've ditched open-webui for that reason. A chat webui that is copy-pasted on chatgpt that does not work properly with an openai model.....is beyond me! No need to be mean. How much did you pay for it? How many contributions did you make to open source? I don't see any project under your account. Just saying....

GiteaMirror commented

@etlweather commented on GitHub (Aug 14, 2025):

I run gpt-oss-20b via llama.cpp and gguf file. Tried various gguf files and all come out just fine with the latest version except for one artifact - <|channel|>analysis as seen here:

For those interested - as I see other people seem to have a lot more artifacts and incorrect parsing - my llama.cpp parameters are:

docker run ... ghcr.io/ggml-org/llama.cpp:server-cuda --port 8080 --host 0.0.0.0 -m /models/unsloth_gpt-oss-20b-Q4_K_M.gguf -c 16384 -ngl 99 --jinja --threads -1 --temp 1.0 --top-p 1.0 --top-k 0

@etlweather commented on GitHub (Aug 14, 2025): I run `gpt-oss-20b` via `llama.cpp` and `gguf` file. Tried various `gguf` files and all come out just fine with the latest version except for one artifact - `<|channel|>analysis` as seen here: <img width="547" height="145" alt="Image" src="https://github.com/user-attachments/assets/72fbb522-b147-4a9b-be3e-bcabb89b282e" /> For those interested - as I see other people seem to have a lot more artifacts and incorrect parsing - my `llama.cpp` parameters are: ``` docker run ... ghcr.io/ggml-org/llama.cpp:server-cuda --port 8080 --host 0.0.0.0 -m /models/unsloth_gpt-oss-20b-Q4_K_M.gguf -c 16384 -ngl 99 --jinja --threads -1 --temp 1.0 --top-p 1.0 --top-k 0 ```

GiteaMirror commented

@tristan-k commented on GitHub (Aug 16, 2025):

According to this comment, when using --reasoning-format auto with llama-server it should fix the issue. Can someone confirm this?

@tristan-k commented on GitHub (Aug 16, 2025): According to this [comment](https://github.com/ggml-org/llama.cpp/pull/15181#issuecomment-3193831022), when using `--reasoning-format auto` with `llama-server` it should fix the issue. Can someone confirm this?

GiteaMirror commented

@etlweather commented on GitHub (Aug 16, 2025):

According to this comment, when using --reasoning-format auto with llama-server it should fix the issue. Can someone confirm this?

Tried it. I still get the <|channel|>analysis artifact but the rest remained good as shown in my previous post.

@etlweather commented on GitHub (Aug 16, 2025): > According to this [comment](https://github.com/ggml-org/llama.cpp/pull/15181#issuecomment-3193831022), when using `--reasoning-format auto` with `llama-server` it should fix the issue. Can someone confirm this? Tried it. I still get the `<|channel|>analysis` artifact but the rest remained good as shown in my previous post.

GiteaMirror commented

@tristan-k commented on GitHub (Aug 16, 2025):

Same here except my OpenWebUI installation doesnt even seperate the reasoning from the answer response.

I'm running with:

      llama-server
      --model unsloth_gpt-oss-20b-GGUF_gpt-oss-20b-Q4_K_M.gguf
      --threads 8
      --cache-type-k q4_0
      --cache-type-v q4_0
      --ctx-size 16384
      --temp 1.0
      --min-p 0.0
      --top-p 1.0
      --top-k 0.0
      --reasoning-format auto

@tristan-k commented on GitHub (Aug 16, 2025): Same here except my OpenWebUI installation doesnt even seperate the reasoning from the answer response. <img width="1614" height="1676" alt="Image" src="https://github.com/user-attachments/assets/9a4a52cf-fa33-4216-8efb-f3bb7f0fd1ca" /> I'm running with: ``` llama-server --model unsloth_gpt-oss-20b-GGUF_gpt-oss-20b-Q4_K_M.gguf --threads 8 --cache-type-k q4_0 --cache-type-v q4_0 --ctx-size 16384 --temp 1.0 --min-p 0.0 --top-p 1.0 --top-k 0.0 --reasoning-format auto ```

GiteaMirror commented

2025-11-11 16:40:43 -06:00

@aldehir commented on GitHub (Aug 16, 2025):

@tristan-k try adding --jinja in the server.

@etlweather the container is running an older version due to a broken build https://github.com/ggml-org/llama.cpp/issues/15356. The server-cuda tag does not contain the PR.

@aldehir commented on GitHub (Aug 16, 2025): @tristan-k try adding --jinja in the server. @etlweather the container is running an older version due to a broken build https://github.com/ggml-org/llama.cpp/issues/15356. The `server-cuda` tag does not contain the PR.

GiteaMirror commented

@semidark commented on GitHub (Aug 17, 2025):

Using the latest commit of the llama.cpp master branch, everything works fine in Open WebUI with the following command:

/usr/local/bin/llama-server \
    --host 0.0.0.0 \
    --port 80 \
    --gpu-layers 25 \
    -m gpt-oss-20b \
    -c 0 -fa \
    --ctx-size 131072 \
    --temp 1.0 \
    --top-p 1.0 \
    --top-k 0 \
    --chat-template-kwargs '{"reasoning_effort":"high"}' \
    --jinja \
    --reasoning-format auto

/usr/local/bin/llama-server --version
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 2 CUDA devices:
  Device 0: NVIDIA GeForce RTX 3060, compute capability 8.6, VMM: yes
  Device 1: NVIDIA GeForce RTX 3060, compute capability 8.6, VMM: yes
register_backend: registered backend CUDA (2 devices)
register_device: registered device CUDA0 (NVIDIA GeForce RTX 3060)
register_device: registered device CUDA1 (NVIDIA GeForce RTX 3060)
register_backend: registered backend CPU (1 devices)
register_device: registered device CPU (Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz)
version: 6183 (65349f26)
built with cc (Debian 12.2.0-14+deb12u1) 12.2.0 for x86_64-linux-gnu

Also, the latest Ollama version correctly supports OpenAIs new Harmony Chat Format. However, the performance was only 50% of the TPS I get with llama.cpp. So, it’s a no-brainer for me to use llama.cpp instead.

@semidark commented on GitHub (Aug 17, 2025): Using the latest commit of the llama.cpp master branch, everything works fine in Open WebUI with the following command: ``` /usr/local/bin/llama-server \ --host 0.0.0.0 \ --port 80 \ --gpu-layers 25 \ -m gpt-oss-20b \ -c 0 -fa \ --ctx-size 131072 \ --temp 1.0 \ --top-p 1.0 \ --top-k 0 \ --chat-template-kwargs '{"reasoning_effort":"high"}' \ --jinja \ --reasoning-format auto ``` ``` /usr/local/bin/llama-server --version ggml_cuda_init: GGML_CUDA_FORCE_MMQ: no ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no ggml_cuda_init: found 2 CUDA devices: Device 0: NVIDIA GeForce RTX 3060, compute capability 8.6, VMM: yes Device 1: NVIDIA GeForce RTX 3060, compute capability 8.6, VMM: yes register_backend: registered backend CUDA (2 devices) register_device: registered device CUDA0 (NVIDIA GeForce RTX 3060) register_device: registered device CUDA1 (NVIDIA GeForce RTX 3060) register_backend: registered backend CPU (1 devices) register_device: registered device CPU (Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz) version: 6183 (65349f26) built with cc (Debian 12.2.0-14+deb12u1) 12.2.0 for x86_64-linux-gnu ``` <img width="381" height="287" alt="Image" src="https://github.com/user-attachments/assets/37f4c844-e6c4-4915-9fbd-819e7d1430bb" /> Also, the latest Ollama version correctly supports OpenAIs new [Harmony Chat Format](https://huggingface.co/blog/kuotient/chatml-vs-harmony). However, the performance was only 50% of the TPS I get with llama.cpp. So, it’s a no-brainer for me to use llama.cpp instead.

GiteaMirror commented

2025-11-11 16:40:43 -06:00

@tristan-k commented on GitHub (Aug 17, 2025):

Try adding --jinja in the server.

That did the trick. So --jinja --reasoning-format auto resolves the issue for me.

However, the performance was only 50% of the TPS I get with llama.cpp. So, it’s a no-brainer for me to use llama.cpp instead.

Yeah, the performance difference between Ollama and llama.cpp is huge. Only with llama.cpp I'm able to run gpt-oss:120bon a Mac Studio M1 Max 64GB fully from unified memory.

@tristan-k commented on GitHub (Aug 17, 2025): > Try adding --jinja in the server. That did the trick. So `--jinja --reasoning-format auto` resolves the issue for me. > However, the performance was only 50% of the TPS I get with llama.cpp. So, it’s a no-brainer for me to use llama.cpp instead. Yeah, the performance difference between `Ollama` and `llama.cpp` is huge. Only with `llama.cpp` I'm able to run `gpt-oss:120b`on a Mac Studio M1 Max 64GB fully from unified memory.

GiteaMirror commented

2025-11-11 16:40:43 -06:00

@GinkREAL commented on GitHub (Aug 18, 2025):

Anyone figure out if the reasoning effort in the chat controls actually work? I've tried checking between high and low but it doesn't seem to affect the thinking

@GinkREAL commented on GitHub (Aug 18, 2025): Anyone figure out if the reasoning effort in the chat controls actually work? I've tried checking between high and low but it doesn't seem to affect the thinking <img width="317" height="40" alt="Image" src="https://github.com/user-attachments/assets/067db153-d5ed-4448-87d3-1704e97b7ff6" />

GiteaMirror commented

@GinkREAL commented on GitHub (Aug 18, 2025):

I wrote a filter instead that seems to work. I set the default reasoning effort to low in the llama.cpp backend also via chat_template_kwargs, so that user queries use medium but interface tasks like title generation use low

from pydantic import BaseModel, Field
from typing import Optional

REASONING_EFFORT_VALUES = {"low": "low", "medium": "medium", "high": "high"}


class Filter:
    class Valves(BaseModel):
        pass

    class UserValves(BaseModel):
        reasoning_effort: str = Field(
            "medium",
            description="Adjust how hard the AI thinks when providing answers.",
            json_schema_extra={
                "enum": list(REASONING_EFFORT_VALUES.keys())
            },  # KEY: This creates the dropdown
        )
        pass

    def __init__(self):
        # Indicates custom file handling logic. This flag helps disengage default routines in favor of custom
        # implementations, informing the WebUI to defer file-related operations to designated methods within this class.
        # Alternatively, you can remove the files directly from the body in from the inlet hook
        # self.file_handler = True

        # Initialize 'valves' with specific configurations. Using 'Valves' instance helps encapsulate settings,
        # which ensures settings are managed cohesively and not confused with operational flags like 'file_handler'.
        self.valves = self.UserValves()
        pass

    def inlet(self, body: dict, __user__: Optional[dict] = None) -> dict:
        if __user__ is not None:
            body["chat_template_kwargs"] = {
                "reasoning_effort": __user__["valves"].reasoning_effort
            }
        else:
            body["chat_template_kwargs"] = "low"
        return body

    def outlet(self, body: dict, __user__: Optional[dict] = None) -> dict:
        return body

@GinkREAL commented on GitHub (Aug 18, 2025): I wrote a filter instead that seems to work. I set the default reasoning effort to low in the llama.cpp backend also via `chat_template_kwargs`, so that user queries use `medium` but interface tasks like title generation use `low` ``` from pydantic import BaseModel, Field from typing import Optional REASONING_EFFORT_VALUES = {"low": "low", "medium": "medium", "high": "high"} class Filter: class Valves(BaseModel): pass class UserValves(BaseModel): reasoning_effort: str = Field( "medium", description="Adjust how hard the AI thinks when providing answers.", json_schema_extra={ "enum": list(REASONING_EFFORT_VALUES.keys()) }, # KEY: This creates the dropdown ) pass def __init__(self): # Indicates custom file handling logic. This flag helps disengage default routines in favor of custom # implementations, informing the WebUI to defer file-related operations to designated methods within this class. # Alternatively, you can remove the files directly from the body in from the inlet hook # self.file_handler = True # Initialize 'valves' with specific configurations. Using 'Valves' instance helps encapsulate settings, # which ensures settings are managed cohesively and not confused with operational flags like 'file_handler'. self.valves = self.UserValves() pass def inlet(self, body: dict, __user__: Optional[dict] = None) -> dict: if __user__ is not None: body["chat_template_kwargs"] = { "reasoning_effort": __user__["valves"].reasoning_effort } else: body["chat_template_kwargs"] = "low" return body def outlet(self, body: dict, __user__: Optional[dict] = None) -> dict: return body ```

GiteaMirror commented

@etlweather commented on GitHub (Aug 19, 2025):

@tristan-k try adding --jinja in the server.

@etlweather the container is running an older version due to a broken build ggml-org/llama.cpp#15356. The server-cuda tag does not contain the PR.

Latest llama.cpp Docker CUDA build works perfectly with the --reasoning-format auto parameter - the output come out as expected.

@etlweather commented on GitHub (Aug 19, 2025): > [@tristan-k](https://github.com/tristan-k) try adding --jinja in the server. > > [@etlweather](https://github.com/etlweather) the container is running an older version due to a broken build [ggml-org/llama.cpp#15356](https://github.com/ggml-org/llama.cpp/issues/15356). The `server-cuda` tag does not contain the PR. Latest llama.cpp Docker CUDA build works perfectly with the `--reasoning-format auto` parameter - the output come out as expected.

GiteaMirror commented

@gaby commented on GitHub (Aug 20, 2025):

@tjbck The latest vLLM has full support for gpt-oss with tool calling unfortunately Open-WebUI doesn't support the responses API.

They documented that they will only support tool calling when using /v1/responses here: https://docs.vllm.ai/projects/recipes/en/latest/OpenAI/GPT-OSS.html#usage

@gaby commented on GitHub (Aug 20, 2025): @tjbck The latest `vLLM` has full support for gpt-oss with tool calling unfortunately Open-WebUI doesn't support the responses API. They documented that they will only support tool calling when using `/v1/responses` here: https://docs.vllm.ai/projects/recipes/en/latest/OpenAI/GPT-OSS.html#usage

GiteaMirror commented

@AbdullahMPrograms commented on GitHub (Aug 23, 2025):

I'm still not getting correct think parsing with llama.cpp latest and OWUI latest, with --reasoning-format auto I get:

with --reasoning-format none I get:

@AbdullahMPrograms commented on GitHub (Aug 23, 2025): I'm still not getting correct think parsing with llama.cpp latest and OWUI latest, with `--reasoning-format auto` I get: <img width="913" height="246" alt="Image" src="https://github.com/user-attachments/assets/08bdd021-3fa5-4029-8548-a2ce0457c19c" /> with `--reasoning-format none` I get: <img width="906" height="924" alt="Image" src="https://github.com/user-attachments/assets/a54ffba4-46a3-4aa4-a7dd-2bb216cac871" />

GiteaMirror commented