github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #8552] Adding Tools support to template causes erroneous removal of <think> tag in responses #67575

New Issue
Closed
opened 2026-05-04 10:54:17 -05:00 by GiteaMirror · 12 comments
GiteaMirror commented 2026-05-04 10:54:17 -05:00
Owner
Copy Link

Originally created by @odrobnik on GitHub (Jan 23, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/8552

What is the issue?

A model like deepseek-r1 emits thinking self-dialog between <think> and </think>. This is evident in deepseek-r1 with the standard chat template.

While trying to add tool support I added this after the system message, this minimal addition is enough to show the problem.

{{- if .Tools }}
# Tools
{{- end }}

This causes Ollama to accept tools definitions, but it causes an issue later on, because there's a bug causing the leading <think> to disappear from messages. You see the thinking followed by </think> and the normal response.

At first I thought that I might have some issue with whitespace removal of the tags, but it turns out, that the above addition to the original template is enough to cause the same issue, with the rest of the template being completely identical.

So my minimally modified template looks like this:

{{- if .System }}{{ .System }}{{ end }}

{{- if .Tools }}
# Tools
{{- end }}

{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1}}
{{- if eq .Role "user" }}<｜User｜>{{ .Content }}
{{- else if eq .Role "assistant" }}<｜Assistant｜>{{ .Content }}{{- if not $last }}<｜end▁of▁sentence｜>{{- end }}
{{- end }}
{{- if and $last (ne .Role "assistant") }}<｜Assistant｜>{{- end }}
{{- end }}

The expected behavior is that the tag should be preserved if it is in the response from the LLM.

OS

macOS

GPU

Apple

CPU

Apple

Ollama version

0.5.7

Originally created by @odrobnik on GitHub (Jan 23, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/8552 ### What is the issue? A model like `deepseek-r1` emits thinking self-dialog between `<think>` and `</think>`. This is evident in deepseek-r1 with the standard chat template. While trying to add tool support I added this after the system message, this minimal addition is enough to show the problem. ``` {{- if .Tools }} # Tools {{- end }} ``` This causes Ollama to accept tools definitions, but it causes an issue later on, because there's a bug causing the leading `<think>` to disappear from messages. You see the thinking followed by `</think>` and the normal response. At first I thought that I might have some issue with whitespace removal of the tags, but it turns out, that the above addition to the original template is enough to cause the same issue, with the rest of the template being completely identical. So my minimally modified template looks like this: ``` {{- if .System }}{{ .System }}{{ end }} {{- if .Tools }} # Tools {{- end }} {{- range $i, $_ := .Messages }} {{- $last := eq (len (slice $.Messages $i)) 1}} {{- if eq .Role "user" }}<｜User｜>{{ .Content }} {{- else if eq .Role "assistant" }}<｜Assistant｜>{{ .Content }}{{- if not $last }}<｜end▁of▁sentence｜>{{- end }} {{- end }} {{- if and $last (ne .Role "assistant") }}<｜Assistant｜>{{- end }} {{- end }} ``` The expected behavior is that the tag should be preserved if it is in the response from the LLM. ### OS macOS ### GPU Apple ### CPU Apple ### Ollama version 0.5.7
GiteaMirror added the bug label 2026-05-04 10:54:17 -05:00
GiteaMirror commented 2026-05-04 10:54:22 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 23, 2025):

Can you provide more information about your testing methodology?

I extracted the modelfile with ollama show --modelfile deepseek-r1:7b > Modelfile.orig and applied the following change:

 # Modelfile generated by "ollama show"
 # To build a new Modelfile based on this, replace FROM with:
-# FROM deepseek-r1:7b
+FROM deepseek-r1:7b
 
-FROM /root/.ollama/models/blobs/sha256-96c415656d377afbff962f6cdb2394ab092ccbcbaab4b82525bc4ca800fe8a49
+# FROM /root/.ollama/models/blobs/sha256-96c415656d377afbff962f6cdb2394ab092ccbcbaab4b82525bc4ca800fe8a49
 TEMPLATE """{{- if .System }}{{ .System }}{{ end }}
+
+{{- if .Tools }}
+# Tools
+{{- end }}
+
 {{- range $i, $_ := .Messages }}
 {{- $last := eq (len (slice $.Messages $i)) 1}}
 {{- if eq .Role "user" }}<｜User｜>{{ .Content }}

$ ollama create deepseek-r1-tools

Original model:

$ curl -s localhost:11434/api/chat -d '{"model":"deepseek-r1","messages":[{"role":"user","content":"2+2?"}],"stream":false}' | jq -r .message.content
<think>

</think>

2 + 2 = **4**.

New model:

$ curl -s localhost:11434/api/chat -d '{"model":"deepseek-r1-tools","messages":[{"role":"user","content":"2+2?"}],"stream":false}' | jq -r .message.content
<think>

</think>

2 + 2 = **4**.
<!-- gh-comment-id:2610537575 --> @rick-github commented on GitHub (Jan 23, 2025): Can you provide more information about your testing methodology? I extracted the modelfile with `ollama show --modelfile deepseek-r1:7b > Modelfile.orig` and applied the following change: ```diff # Modelfile generated by "ollama show" # To build a new Modelfile based on this, replace FROM with: -# FROM deepseek-r1:7b +FROM deepseek-r1:7b -FROM /root/.ollama/models/blobs/sha256-96c415656d377afbff962f6cdb2394ab092ccbcbaab4b82525bc4ca800fe8a49 +# FROM /root/.ollama/models/blobs/sha256-96c415656d377afbff962f6cdb2394ab092ccbcbaab4b82525bc4ca800fe8a49 TEMPLATE """{{- if .System }}{{ .System }}{{ end }} + +{{- if .Tools }} +# Tools +{{- end }} + {{- range $i, $_ := .Messages }} {{- $last := eq (len (slice $.Messages $i)) 1}} {{- if eq .Role "user" }}<｜User｜>{{ .Content }} ``` ```console $ ollama create deepseek-r1-tools ``` Original model: ```console $ curl -s localhost:11434/api/chat -d '{"model":"deepseek-r1","messages":[{"role":"user","content":"2+2?"}],"stream":false}' | jq -r .message.content <think> </think> 2 + 2 = **4**. ``` New model: ```console $ curl -s localhost:11434/api/chat -d '{"model":"deepseek-r1-tools","messages":[{"role":"user","content":"2+2?"}],"stream":false}' | jq -r .message.content <think> </think> 2 + 2 = **4**. ```
GiteaMirror commented 2026-05-04 10:54:23 -05:00
Author
Owner
Copy Link

@odrobnik commented on GitHub (Jan 23, 2025):

@nickgithub you also need to send tool definitions to the model. Only the {{- if .Tools }} by itself doesn't seem to trigger it. But if you send tools JSON then the error occurs.

Let me know if you cannot reproduce it, then I'll send you the full curl for it.

<!-- gh-comment-id:2610543670 --> @odrobnik commented on GitHub (Jan 23, 2025): @nickgithub you also need to send tool definitions to the model. Only the `{{- if .Tools }}` by itself doesn't seem to trigger it. But if you send tools JSON then the error occurs. Let me know if you cannot reproduce it, then I'll send you the full curl for it.
GiteaMirror commented 2026-05-04 10:54:24 -05:00
Author
Owner
Copy Link

@odrobnik commented on GitHub (Jan 23, 2025):

Ok, and - if that makes a difference - I am using the OpenAI API endpoints.

<!-- gh-comment-id:2610549544 --> @odrobnik commented on GitHub (Jan 23, 2025): Ok, and - if that makes a difference - I am using the OpenAI API endpoints.
GiteaMirror commented 2026-05-04 10:54:25 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 23, 2025):

$ ./tool-test.py --model deepseek-r1-tools --prompt '2+2=?'
<think>
First, I recognize that the problem is a simple addition of two numbers: 2 and 2.

To solve it, I add these two numbers together.

The result of adding 2 and 2 is 4.
</think>

Sure! Let's solve the equation step by step.

**Problem:**
\[ 2 + 2 = ? \]

**Solution:**

1. **Identify the Numbers to Add:**
   - We have two numbers: 2 and 2.

2. **Add the Numbers Together:**
   \[
   2 + 2 = 4
   \]

3. **Final Answer:**
   \[
   \boxed{4}
   \]

So, \( 2 + 2 = \boxed{4} \).
<!-- gh-comment-id:2610549831 --> @rick-github commented on GitHub (Jan 23, 2025): ```console $ ./tool-test.py --model deepseek-r1-tools --prompt '2+2=?' <think> First, I recognize that the problem is a simple addition of two numbers: 2 and 2. To solve it, I add these two numbers together. The result of adding 2 and 2 is 4. </think> Sure! Let's solve the equation step by step. **Problem:** \[ 2 + 2 = ? \] **Solution:** 1. **Identify the Numbers to Add:** - We have two numbers: 2 and 2. 2. **Add the Numbers Together:** \[ 2 + 2 = 4 \] 3. **Final Answer:** \[ \boxed{4} \] So, \( 2 + 2 = \boxed{4} \). ```
GiteaMirror commented 2026-05-04 10:54:26 -05:00
Author
Owner
Copy Link

@odrobnik commented on GitHub (Jan 23, 2025):

curl -X POST \
  http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {
        "content": "You are a helpful personal assistant.",
        "role": "system"
      },
      {
        "name": "Oliver",
        "role": "user",
        "content": "What is the current date"
      }
    ],
    "model": "deepseek-qwen:latest",
    "stream": false,
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_current_date",
          "description": "Get the current date and time. Use this when you need to determine the current timestamp for reference in answering user queries.",
          "parameters": {
            "type": "object",
            "properties": {},
            "required": []
          }
        }
      }
    ]
  }'

Results in

{"id":"chatcmpl-276","object":"chat.completion","created":1737654425,"model":"deepseek-qwen:latest","system_fingerprint":"fp_ollama","choices":[{"index":0,"message":{"role":"assistant","content":"The current date is \u003ccurrent_date\u003e.\n\n\u003c/think\u003e\n\nThe current date is Monday, November 20, 2023 at 12:45 PM."},"finish_reason":"stop"}],"usage":{"prompt_tokens":180,"completion_tokens":36,"total_tokens":216}}

And ahead of The current date there should be a <think> because there's also a </think>

<!-- gh-comment-id:2610559862 --> @odrobnik commented on GitHub (Jan 23, 2025): ``` curl -X POST \ http://localhost:11434/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "messages": [ { "content": "You are a helpful personal assistant.", "role": "system" }, { "name": "Oliver", "role": "user", "content": "What is the current date" } ], "model": "deepseek-qwen:latest", "stream": false, "tools": [ { "type": "function", "function": { "name": "get_current_date", "description": "Get the current date and time. Use this when you need to determine the current timestamp for reference in answering user queries.", "parameters": { "type": "object", "properties": {}, "required": [] } } } ] }' ``` Results in ``` {"id":"chatcmpl-276","object":"chat.completion","created":1737654425,"model":"deepseek-qwen:latest","system_fingerprint":"fp_ollama","choices":[{"index":0,"message":{"role":"assistant","content":"The current date is \u003ccurrent_date\u003e.\n\n\u003c/think\u003e\n\nThe current date is Monday, November 20, 2023 at 12:45 PM."},"finish_reason":"stop"}],"usage":{"prompt_tokens":180,"completion_tokens":36,"total_tokens":216}} ``` And ahead of `The current date` there should be a `<think>` because there's also a `</think>`
GiteaMirror commented 2026-05-04 10:54:26 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 23, 2025):

$ curl -s -X POST   http://localhost:11434/v1/chat/completions   -H "Content-Type: application/json"   -d '{
    "messages": [
      {
        "content": "You are a helpful personal assistant.",
        "role": "system"
      },
      {
        "name": "Oliver",
        "role": "user",
        "content": "What is the current date"
      }
    ],
    "model": "deepseek-r1-tools",
    "stream": false,
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_current_date",
          "description": "Get the current date and time. Use this when you need to determine the current timestamp for reference in answering user queries.",
          "parameters": {
            "type": "object",
            "properties": {},
            "required": []
          }
        }
      }
    ]
  }' | jq -r '.choices[0].message.content'
<think>

</think>

Hi there! I suggest getting online to get real-time information. If you have any other questions, please don't hesitate to let me know!

The model I'm using is deepseek-r1:7b-qwen-distill-q4_K_M

<!-- gh-comment-id:2610571702 --> @rick-github commented on GitHub (Jan 23, 2025): ```console $ curl -s -X POST http://localhost:11434/v1/chat/completions -H "Content-Type: application/json" -d '{ "messages": [ { "content": "You are a helpful personal assistant.", "role": "system" }, { "name": "Oliver", "role": "user", "content": "What is the current date" } ], "model": "deepseek-r1-tools", "stream": false, "tools": [ { "type": "function", "function": { "name": "get_current_date", "description": "Get the current date and time. Use this when you need to determine the current timestamp for reference in answering user queries.", "parameters": { "type": "object", "properties": {}, "required": [] } } } ] }' | jq -r '.choices[0].message.content' <think> </think> Hi there! I suggest getting online to get real-time information. If you have any other questions, please don't hesitate to let me know! ``` The model I'm using is deepseek-r1:7b-qwen-distill-q4_K_M
GiteaMirror commented 2026-05-04 10:54:27 -05:00
Author
Owner
Copy Link

@odrobnik commented on GitHub (Jan 23, 2025):

This is my model file, I modified the one for deepseek-r1:32b

FROM deepseek-r1:32b
TEMPLATE """
{{- if .Messages }}
{{- if or .System .Tools }}system
{{- if .System }}
{{ .System }}
{{- end }}
{{- if .Tools }}

# Tools

You may call one or more functions to assist with the user query.

You are provided with function signatures within <tools></tools> XML tags:
<tools>
{{- range .Tools }}
{"type": "function", "function": {{ .Function }}}
{{- end }}
</tools>

For each function call, return a JSON object with function name and arguments within <tool_call></tool_call> XML tags:
<tool_call>
{"name": <function-name>, "arguments": <args-json-object>}
</tool_call>
{{- end }}<｜end▁of▁sentence｜>
{{ end }}

{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1 -}}
{{- if eq .Role "user" }}<｜begin▁of▁sentence｜>user
{{ .Content }}<｜end▁of▁sentence｜>

{{- else if eq .Role "assistant" }}<｜begin▁of▁sentence｜>assistant
{{- if .Content }}
{{ .Content }}
{{- end }}
{{- if .ToolCalls }}
<tool_call>
{{- range .ToolCalls }}
{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{- end }}
</tool_call>
{{- end }}
{{- if not $last }}<｜end▁of▁sentence｜>{{ end }}

{{- else if eq .Role "tool" }}<｜begin▁of▁sentence｜>user
<tool_response>
{{ .Content }}
</tool_response><｜end▁of▁sentence｜>
{{- end }}

{{- if and (ne .Role "assistant") $last }}<｜begin▁of▁sentence｜>assistant
{{- end }}

{{- end }}

{{- else }}
{{- if .System }}<｜begin▁of▁sentence｜>system
{{ .System }}<｜end▁of▁sentence｜>
{{- end }}

{{- if .Prompt }}<｜begin▁of▁sentence｜>user
{{ .Prompt }}<｜end▁of▁sentence｜>
{{- end }}

<｜begin▁of▁sentence｜>assistant
{{ .Response }}
{{- if .Response }}<｜end▁of▁sentence｜>{{ end }}
{{- end }}
"""
PARAMETER stop <｜begin▁of▁sentence｜>
PARAMETER stop <｜end▁of▁sentence｜>
PARAMETER stop <｜User｜>
PARAMETER stop <｜Assistant｜>
LICENSE """MIT License

Copyright (c) 2023 DeepSeek

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
"""
<!-- gh-comment-id:2610581081 --> @odrobnik commented on GitHub (Jan 23, 2025): This is my model file, I modified the one for deepseek-r1:32b ``` FROM deepseek-r1:32b TEMPLATE """ {{- if .Messages }} {{- if or .System .Tools }}system {{- if .System }} {{ .System }} {{- end }} {{- if .Tools }} # Tools You may call one or more functions to assist with the user query. You are provided with function signatures within <tools></tools> XML tags: <tools> {{- range .Tools }} {"type": "function", "function": {{ .Function }}} {{- end }} </tools> For each function call, return a JSON object with function name and arguments within <tool_call></tool_call> XML tags: <tool_call> {"name": <function-name>, "arguments": <args-json-object>} </tool_call> {{- end }}<｜end▁of▁sentence｜> {{ end }} {{- range $i, $_ := .Messages }} {{- $last := eq (len (slice $.Messages $i)) 1 -}} {{- if eq .Role "user" }}<｜begin▁of▁sentence｜>user {{ .Content }}<｜end▁of▁sentence｜> {{- else if eq .Role "assistant" }}<｜begin▁of▁sentence｜>assistant {{- if .Content }} {{ .Content }} {{- end }} {{- if .ToolCalls }} <tool_call> {{- range .ToolCalls }} {"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}} {{- end }} </tool_call> {{- end }} {{- if not $last }}<｜end▁of▁sentence｜>{{ end }} {{- else if eq .Role "tool" }}<｜begin▁of▁sentence｜>user <tool_response> {{ .Content }} </tool_response><｜end▁of▁sentence｜> {{- end }} {{- if and (ne .Role "assistant") $last }}<｜begin▁of▁sentence｜>assistant {{- end }} {{- end }} {{- else }} {{- if .System }}<｜begin▁of▁sentence｜>system {{ .System }}<｜end▁of▁sentence｜> {{- end }} {{- if .Prompt }}<｜begin▁of▁sentence｜>user {{ .Prompt }}<｜end▁of▁sentence｜> {{- end }} <｜begin▁of▁sentence｜>assistant {{ .Response }} {{- if .Response }}<｜end▁of▁sentence｜>{{ end }} {{- end }} """ PARAMETER stop <｜begin▁of▁sentence｜> PARAMETER stop <｜end▁of▁sentence｜> PARAMETER stop <｜User｜> PARAMETER stop <｜Assistant｜> LICENSE """MIT License Copyright (c) 2023 DeepSeek Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. """ ```
GiteaMirror commented 2026-05-04 10:54:28 -05:00
Author
Owner
Copy Link

@odrobnik commented on GitHub (Jan 23, 2025):

Ok, I pulled deepseek-r1:7b and created deepseek-qwen-7b with the above modelfile, only changing the from to the 7b.

create deepseek-qwen-7b -f deepseek_r1.modelfile

Again, the tag is missing.

<!-- gh-comment-id:2610594274 --> @odrobnik commented on GitHub (Jan 23, 2025): Ok, I pulled `deepseek-r1:7b` and created `deepseek-qwen-7b` with the above modelfile, only changing the from to the 7b. create deepseek-qwen-7b -f deepseek_r1.modelfile Again, the tag is missing.
GiteaMirror commented 2026-05-04 10:54:29 -05:00
Author
Owner
Copy Link

@odrobnik commented on GitHub (Jan 23, 2025):

Ok, I did a fresh modelfile, from deepseek-r1:7b, made the minimal modification, and now the <think> is there.

FROM deepseek-r1:7b
TEMPLATE """{{- if .System }}{{ .System }}{{ end }}

{{- if .Tools }}
# Tools
{{- end }}

{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1}}
{{- if eq .Role "user" }}<｜User｜>{{ .Content }}
{{- else if eq .Role "assistant" }}<｜Assistant｜>{{ .Content }}{{- if not $last }}<｜end▁of▁sentence｜>{{- end }}
{{- end }}
{{- if and $last (ne .Role "assistant") }}<｜Assistant｜>{{- end }}
{{- end }}"""
PARAMETER stop <｜begin▁of▁sentence｜>
PARAMETER stop <｜end▁of▁sentence｜>
PARAMETER stop <｜User｜>
PARAMETER stop <｜Assistant｜>

output now

{"id":"chatcmpl-896","object":"chat.completion","created":1737655518,"model":"deepseek-qwen-7b2","system_fingerprint":"fp_ollama","choices":[{"index":0,"message":{"role":"assistant",
"content":"\u003cthink\u003e\n\n\u003c/think\u003e\n\nHi there! I suggest getting online to get real-time information. If you have any other questions, please don't hesitate to let me know!"},"finish_reason":"stop"}],"usage":{"prompt_tokens":17,"completion_tokens":34,"total_tokens":51}}

Next I'll try to re-add the tools things to see at what point the tag starts disappearing.

<!-- gh-comment-id:2610609685 --> @odrobnik commented on GitHub (Jan 23, 2025): Ok, I did a fresh modelfile, from deepseek-r1:7b, made the minimal modification, and now the `<think>` is there. ``` FROM deepseek-r1:7b TEMPLATE """{{- if .System }}{{ .System }}{{ end }} {{- if .Tools }} # Tools {{- end }} {{- range $i, $_ := .Messages }} {{- $last := eq (len (slice $.Messages $i)) 1}} {{- if eq .Role "user" }}<｜User｜>{{ .Content }} {{- else if eq .Role "assistant" }}<｜Assistant｜>{{ .Content }}{{- if not $last }}<｜end▁of▁sentence｜>{{- end }} {{- end }} {{- if and $last (ne .Role "assistant") }}<｜Assistant｜>{{- end }} {{- end }}""" PARAMETER stop <｜begin▁of▁sentence｜> PARAMETER stop <｜end▁of▁sentence｜> PARAMETER stop <｜User｜> PARAMETER stop <｜Assistant｜> ``` output now ``` {"id":"chatcmpl-896","object":"chat.completion","created":1737655518,"model":"deepseek-qwen-7b2","system_fingerprint":"fp_ollama","choices":[{"index":0,"message":{"role":"assistant", "content":"\u003cthink\u003e\n\n\u003c/think\u003e\n\nHi there! I suggest getting online to get real-time information. If you have any other questions, please don't hesitate to let me know!"},"finish_reason":"stop"}],"usage":{"prompt_tokens":17,"completion_tokens":34,"total_tokens":51}} ``` Next I'll try to re-add the tools things to see at what point the tag starts disappearing.
GiteaMirror commented 2026-05-04 10:54:30 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 23, 2025):

Replicated with your more complex change and :32b. Looking to see what's happening under the covers.

<!-- gh-comment-id:2610612542 --> @rick-github commented on GitHub (Jan 23, 2025): Replicated with your more complex change and :32b. Looking to see what's happening under the covers.
GiteaMirror commented 2026-05-04 10:54:31 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 23, 2025):

The raw output from the model doesn't contain the <think> tag.

Minimal replication using your modified template with :7b, note no tools:

$ curl -s localhost:11434/v1/chat/completions -d '{"messages":[{"role": "user","content": "hello"}],"model":"deepseek-r1-tools","stream":false}' | jq 
{
  "id": "chatcmpl-639",
  "object": "chat.completion",
  "created": 1737656456,
  "model": "deepseek-r1-tools",
  "system_fingerprint": "fp_ollama",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Alright, the user just said \"hi.\" That's pretty straightforward. They're probably looking to start a conversation or get some help.\n\nI should respond in a friendly and welcoming manner. Maybe ask how I can assist them today.\n\nKeeping it open-ended so they feel comfortable to share what's on their mind.\n</think>\n\nHello! How can I assist you today?"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 74,
    "total_tokens": 83
  }
}

This is the prompt sent to the runner with the modified template:

{
  "cache_prompt": true,
  "frequency_penalty": 0,
  "image_data": null,
  "main_gpu": 0,
  "min_p": 0,
  "mirostat": 0,
  "mirostat_eta": 0.1,
  "mirostat_tau": 5,
  "n_keep": 4,
  "n_predict": -1,
  "presence_penalty": 0,
  "prompt": "<...begin...of...sentence...>user\nhello<...end...of...sentence...><...begin...of...sentence...>assistant\n",
  "repeat_last_n": 64,
  "repeat_penalty": 1.1,
  "seed": -1,
  "stop": [
    "<...begin...of...sentence...>",
    "<...end...of...sentence...>",
    "<...User...>",
    "<...Assistant...>"
  ],
  "stream": true,
  "temperature": 1,
  "top_k": 40,
  "top_p": 1,
  "typical_p": 1
}

The response from the runner starts:

{"content":"Alright","stop":false,"timings":{"predicted_n":0,"predicted_ms":0,"prompt_n":0,"prompt_ms":0}}
{"content":",","stop":false,"timings":{"predicted_n":0,"predicted_ms":0,"prompt_n":0,"prompt_ms":0}}

This is the prompt sent to the runner using the default template:

{
  "cache_prompt": true,
  "frequency_penalty": 0,
  "image_data": null,
  "main_gpu": 0,
  "min_p": 0,
  "mirostat": 0,
  "mirostat_eta": 0.1,
  "mirostat_tau": 5,
  "n_keep": 4,
  "n_predict": -1,
  "presence_penalty": 0,
  "prompt": "<...User...>hello<...Assistant...>",
  "repeat_last_n": 64,
  "repeat_penalty": 1.1,
  "seed": -1,
  "stop": [
    "<...begin...of...sentence...>",
    "<...end...of...sentence...>",
    "<...User...>",
    "<...Assistant...>"
  ],
  "stream": true,
  "temperature": 1,
  "top_k": 40,
  "top_p": 1,
  "typical_p": 1
}

And the response from the runner:

{"content":"\u003cthink\u003e","stop":false,"timings":{"predicted_n":0,"predicted_ms":0,"prompt_n":0,"prompt_ms":0}}
{"content":"\n\n","stop":false,"timings":{"predicted_n":0,"predicted_ms":0,"prompt_n":0,"prompt_ms":0}}

So it looks like the template is constructing a sequence that breaks the CoT processing:

  • modified template: "<...begin...of...sentence...>user\nhello<...end...of...sentence...><...begin...of...sentence...>assistant\n"
  • default template: "<...User...>hello<...Assistant...>"
<!-- gh-comment-id:2610657918 --> @rick-github commented on GitHub (Jan 23, 2025): The raw output from the model doesn't contain the \<think\> tag. Minimal replication using your modified template with :7b, note no `tools`: ```console $ curl -s localhost:11434/v1/chat/completions -d '{"messages":[{"role": "user","content": "hello"}],"model":"deepseek-r1-tools","stream":false}' | jq { "id": "chatcmpl-639", "object": "chat.completion", "created": 1737656456, "model": "deepseek-r1-tools", "system_fingerprint": "fp_ollama", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "Alright, the user just said \"hi.\" That's pretty straightforward. They're probably looking to start a conversation or get some help.\n\nI should respond in a friendly and welcoming manner. Maybe ask how I can assist them today.\n\nKeeping it open-ended so they feel comfortable to share what's on their mind.\n</think>\n\nHello! How can I assist you today?" }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 9, "completion_tokens": 74, "total_tokens": 83 } } ``` This is the prompt sent to the runner with the modified template: ```json { "cache_prompt": true, "frequency_penalty": 0, "image_data": null, "main_gpu": 0, "min_p": 0, "mirostat": 0, "mirostat_eta": 0.1, "mirostat_tau": 5, "n_keep": 4, "n_predict": -1, "presence_penalty": 0, "prompt": "<...begin...of...sentence...>user\nhello<...end...of...sentence...><...begin...of...sentence...>assistant\n", "repeat_last_n": 64, "repeat_penalty": 1.1, "seed": -1, "stop": [ "<...begin...of...sentence...>", "<...end...of...sentence...>", "<...User...>", "<...Assistant...>" ], "stream": true, "temperature": 1, "top_k": 40, "top_p": 1, "typical_p": 1 } ``` The response from the runner starts: ```json {"content":"Alright","stop":false,"timings":{"predicted_n":0,"predicted_ms":0,"prompt_n":0,"prompt_ms":0}} {"content":",","stop":false,"timings":{"predicted_n":0,"predicted_ms":0,"prompt_n":0,"prompt_ms":0}} ``` This is the prompt sent to the runner using the default template: ```json { "cache_prompt": true, "frequency_penalty": 0, "image_data": null, "main_gpu": 0, "min_p": 0, "mirostat": 0, "mirostat_eta": 0.1, "mirostat_tau": 5, "n_keep": 4, "n_predict": -1, "presence_penalty": 0, "prompt": "<...User...>hello<...Assistant...>", "repeat_last_n": 64, "repeat_penalty": 1.1, "seed": -1, "stop": [ "<...begin...of...sentence...>", "<...end...of...sentence...>", "<...User...>", "<...Assistant...>" ], "stream": true, "temperature": 1, "top_k": 40, "top_p": 1, "typical_p": 1 } ``` And the response from the runner: ```json {"content":"\u003cthink\u003e","stop":false,"timings":{"predicted_n":0,"predicted_ms":0,"prompt_n":0,"prompt_ms":0}} {"content":"\n\n","stop":false,"timings":{"predicted_n":0,"predicted_ms":0,"prompt_n":0,"prompt_ms":0}} ``` So it looks like the template is constructing a sequence that breaks the CoT processing: - modified template: `"<...begin...of...sentence...>user\nhello<...end...of...sentence...><...begin...of...sentence...>assistant\n"` - default template: `"<...User...>hello<...Assistant...>"`
GiteaMirror commented 2026-05-04 10:54:31 -05:00
Author
Owner
Copy Link

@odrobnik commented on GitHub (Jan 24, 2025):

I am pretty sure that the BOS tokens are what causes the problem. The default template doesn't have those. So my issue was not caused by the reason I gave, so I withdraw it.... thanks for investigating though!

<!-- gh-comment-id:2613324079 --> @odrobnik commented on GitHub (Jan 24, 2025): I am pretty sure that the BOS tokens are what causes the problem. The default template doesn't have those. So my issue was not caused by the reason I gave, so I withdraw it.... thanks for investigating though!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#67575
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?