github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #5403] Codestral template prevents using it for FIM #65420

New Issue
Closed
opened 2026-05-03 21:15:01 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-05-03 21:15:01 -05:00
Owner
Copy Link

Originally created by @brnrc on GitHub (Jul 1, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/5403

Originally assigned to: @mxyng on GitHub.

What is the issue?

Hi 👋 . The current template of codestral:latest https://ollama.com/library/codestral:latest/blobs/36ee4ce5634b makes it harder to use the model for Fill in the Middle (FIM).

Codestral for FIM requires the following template:

<s>[SUFFIX] {{ suffix }} [PREFIX] {{ prefix }}

While its Instruct mode requires the template to be:

[INST] {{ .Prompt }} [/INST] {{ .Response }}

If a ollama user would like to use codestral for FIM, he/she will have to set raw=True (docs) in order to ignore the default template.

The main issue I'm facing is that I'm using ollama with the litellm openai compatible endpoints, and in this setting, the raw parameter is not supported. Thus I can't ignore the default template of codestral which results in a mix of Instruct/FIM model responses, since the FIM user template will always be wrapped with the Instruct template:

[INST] <s>[SUFFIX] {{ suffix }} [PREFIX] {{ prefix }} [/INST]

Proposal

Is it possible to change the default template to just be {{ .Prompt }} like many other models?

Examples:

bruno@mac ~> ollama show mistral --template
{{ .Prompt }}

bruno@mac ~> ollama show mixtral --template
{{ .Prompt }}

bruno@mac ~> ollama show codegemma:2b --template
{{ .Prompt }}

I'm aware I can modify or create a new Modefile for codestral, but I think it is wrong to have the default template to be the Instruct template for a model that supports both modes.

OS

No response

GPU

No response

CPU

No response

Ollama version

No response

Originally created by @brnrc on GitHub (Jul 1, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/5403 Originally assigned to: @mxyng on GitHub. ### What is the issue? Hi 👋 . The current template of `codestral:latest` https://ollama.com/library/codestral:latest/blobs/36ee4ce5634b makes it harder to use the model for Fill in the Middle (FIM). Codestral for FIM requires the following template: ``` <s>[SUFFIX] {{ suffix }} [PREFIX] {{ prefix }} ``` While its Instruct mode requires the template to be: ``` [INST] {{ .Prompt }} [/INST] {{ .Response }} ``` If a ollama user would like to use codestral for FIM, he/she will have to set `raw=True` ([docs](https://github.com/ollama/ollama/blob/main/docs/api.md#parameters)) in order to ignore the default template. The main issue I'm facing is that I'm using ollama with the litellm openai compatible endpoints, and in this setting, the `raw` parameter is not supported. Thus I can't ignore the default template of codestral which results in a mix of Instruct/FIM model responses, since the FIM user template will always be wrapped with the Instruct template: ``` [INST] <s>[SUFFIX] {{ suffix }} [PREFIX] {{ prefix }} [/INST] ``` ## Proposal Is it possible to change the default template to just be `{{ .Prompt }}` like many other models? Examples: ``` bruno@mac ~> ollama show mistral --template {{ .Prompt }} bruno@mac ~> ollama show mixtral --template {{ .Prompt }} bruno@mac ~> ollama show codegemma:2b --template {{ .Prompt }} ``` I'm aware I can modify or create a new Modefile for codestral, but I think it is wrong to have the default template to be the Instruct template for a model that supports both modes. ### OS _No response_ ### GPU _No response_ ### CPU _No response_ ### Ollama version _No response_
GiteaMirror added the feature request label 2026-05-03 21:15:01 -05:00
GiteaMirror commented 2026-05-03 21:15:04 -05:00
Author
Owner
Copy Link

@d3tk commented on GitHub (Jul 8, 2024):

I would like to use FIM with codestral as well. How would one edit the model file to do this?

<!-- gh-comment-id:2214405930 --> @d3tk commented on GitHub (Jul 8, 2024): I would like to use FIM with codestral as well. How would one edit the model file to do this?
GiteaMirror commented 2026-05-03 21:15:05 -05:00
Author
Owner
Copy Link

@mxyng commented on GitHub (Jul 8, 2024):

FIM is implemented in #5207 and will be, I'm hoping, part of the next release. This will allow users to call the API with an additional suffix field in addition to the existing prompt field in /api/generate to pass to the LLM the prefix and suffix. Existing models will require an template update in order to leverage the new feature.

Is it possible to change the default template to just be {{ .Prompt }} like many other models?

Using the API, it's easier to set template="{{ .Prompt }}" for FIM than it is for chat. Since codestral is dual chat and FIM model so the current template is more appropriate than {{ .Prompt }} since many user's interaction with the model isn't through the API but through a chat interface such as the CLI

<!-- gh-comment-id:2214694940 --> @mxyng commented on GitHub (Jul 8, 2024): FIM is implemented in #5207 and will be, I'm hoping, part of the next release. This will allow users to call the API with an additional `suffix` field in addition to the existing `prompt` field in `/api/generate` to pass to the LLM the prefix and suffix. Existing models will require an template update in order to leverage the new feature. > Is it possible to change the default template to just be `{{ .Prompt }}` like many other models? Using the API, it's easier to set `template="{{ .Prompt }}"` for FIM than it is for chat. Since codestral is dual chat and FIM [model](https://huggingface.co/mistralai/Codestral-22B-v0.1) so the current template is more appropriate than `{{ .Prompt }}` since many user's interaction with the model isn't through the API but through a chat interface such as the CLI
GiteaMirror commented 2026-05-03 21:15:06 -05:00
Author
Owner
Copy Link

@g0t4 commented on GitHub (Jun 7, 2025):

@brnrc how did you find/derive the FIM template for codestral?

I only found the tokens: https://huggingface.co/mistralai/Codestral-22B-v0.1/blob/main/tokenizer_config.json

But, can't find any examples of PSM/SPM/etc

<!-- gh-comment-id:2952280661 --> @g0t4 commented on GitHub (Jun 7, 2025): @brnrc how did you find/derive the FIM template for codestral? I only found the tokens: https://huggingface.co/mistralai/Codestral-22B-v0.1/blob/main/tokenizer_config.json But, can't find any examples of PSM/SPM/etc
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#65420
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?