github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-08 00:51:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #9628] mistral-small:24B chat template #68339

New Issue
Open
opened 2026-05-04 13:15:46 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-04 13:15:46 -05:00
Owner
Copy Link

Originally created by @logkn on GitHub (Mar 10, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/9628

I was finding that mistral-small:24B had some trouble calling tools -- namely, adding or dropping tokens that rendered the tool call as message content rather than an actual tool call.

I tried to see if maybe there was a discrepancy in the chat template. The jinja template on the model's Huggingface page was actually not very helpful because it doesn't even include tool calling. I dug around a bit to find the Tekken V7 tokenizer on mistral_common, and sure enough the chat template for providing and calling tools didn't quite match up with Ollama's.

I had Claude whip up a version that matches up with the Tekken template with tools, and the template below is much more consistent with tools:

{{- range $index, $_ := .Messages }}
{{- if eq .Role "system" }}[SYSTEM_PROMPT]{{ .Content }}[/SYSTEM_PROMPT]
{{- else if eq .Role "user" }}
{{- if and (le (len (slice $.Messages $index)) 2) $.Tools }}[AVAILABLE_TOOLS]{{ $.Tools }}[/AVAILABLE_TOOLS]
{{- end }}[INST]{{ .Content }}[/INST]
{{- else if eq .Role "assistant" }}
{{- if .Content }}{{ .Content }}
{{- if not (eq (len (slice $.Messages $index)) 1) }}</s>
{{- end }}
{{- else if .ToolCalls }}[TOOL_CALLS] [
{{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}}
{{- end }}]</s>
{{- end }}
{{- else if eq .Role "tool" }}[TOOL_RESULTS] [TOOL_CONTENT] {{ .Content }}[/TOOL_RESULTS]
{{- end }}
{{- end }}

This might not be perfect, but it's definitely an improvement. The main thing I was finding was that Mistral liked to prefix its tool calls with [TOOL_CALLS], which makes sense because it's part of the chat template it was trained on (see https://github.com/mistralai/mistral-common/blob/main/tests/test_tokenizer_v7.py).

Originally created by @logkn on GitHub (Mar 10, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/9628 I was finding that mistral-small:24B had some trouble calling tools -- namely, adding or dropping tokens that rendered the tool call as message content rather than an actual tool call. I tried to see if maybe there was a discrepancy in the chat template. The jinja template on the model's Huggingface page was actually not very helpful because it doesn't even include tool calling. I dug around a bit to find the Tekken V7 tokenizer on mistral_common, and sure enough the chat template for providing and calling tools didn't quite match up with Ollama's. I had Claude whip up a version that matches up with the Tekken template with tools, and the template below is much more consistent with tools: ```go {{- range $index, $_ := .Messages }} {{- if eq .Role "system" }}[SYSTEM_PROMPT]{{ .Content }}[/SYSTEM_PROMPT] {{- else if eq .Role "user" }} {{- if and (le (len (slice $.Messages $index)) 2) $.Tools }}[AVAILABLE_TOOLS]{{ $.Tools }}[/AVAILABLE_TOOLS] {{- end }}[INST]{{ .Content }}[/INST] {{- else if eq .Role "assistant" }} {{- if .Content }}{{ .Content }} {{- if not (eq (len (slice $.Messages $index)) 1) }}</s> {{- end }} {{- else if .ToolCalls }}[TOOL_CALLS] [ {{- range .ToolCalls }}{"name": "{{ .Function.Name }}", "arguments": {{ .Function.Arguments }}} {{- end }}]</s> {{- end }} {{- else if eq .Role "tool" }}[TOOL_RESULTS] [TOOL_CONTENT] {{ .Content }}[/TOOL_RESULTS] {{- end }} {{- end }} ``` This might not be perfect, but it's definitely an improvement. The main thing I was finding was that Mistral liked to prefix its tool calls with `[TOOL_CALLS]`, which makes sense because it's part of the chat template it was trained on (see https://github.com/mistralai/mistral-common/blob/main/tests/test_tokenizer_v7.py).
GiteaMirror added the feature request label 2026-05-04 13:15:46 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#68339
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?