github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #9929] Limit role collation to system, user, and assistant #32260

New Issue
Open
opened 2026-04-22 13:21:12 -05:00 by GiteaMirror · 1 comment
GiteaMirror commented 2026-04-22 13:21:12 -05:00
Owner
Copy Link

Originally created by @gabe-l-hart on GitHub (Mar 21, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/9929

Description

The current code here will automatically collapse multiple elements of the messages list where the role string is identical into a single message entry with the content strings concatenated. With the addition of new roles such as control and document that are designed to inform the chat template when creating the prompt string, this logic causes problems if multiple such messages are given sequentially. For example, with the granite3.2 template, consider the following request:

{
    "messages": [
        {"role": "document", "content": "My name is Gabe"},
        {"role": "document", "content": "I work for IBM"},
        {"role": "control", "content": "citations"},
        {"role": "control", "content": "hallucinations"},
        {"role": "control", "content": "length long"},
        {"role": "user", "content": "Who do I work for?"}
    ]
}

This should result in the following:

<|start_of_role|>system<|end_of_role|>Knowledge Cutoff Date: April 2024.
You are Granite, developed by IBM. Write the response to the user's input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data.

In your response, use the symbols <co> and </co> to indicate when a fact comes from a document in the search result, e.g <co>0</co> for a fact from document 0. Afterwards, list all the citations with their corresponding documents in an ordered list.

Finally, after the response is written, include a numbered list of sentences from the response that are potentially hallucinated and not based in the documents.<|end_of_text|>
<|start_of_role|>documents<|end_of_role|>
Document 0
My name is Gabe

Document 1
I work for IBM<|end_of_text|>
<|start_of_role|>user<|end_of_role|>Who do I work for?<|end_of_text|>
<|start_of_role|>assistant {"length": "long"}<|end_of_role|>

but with collation, it instead produces the following:

<|start_of_role|>system<|end_of_role|>Knowledge Cutoff Date: April 2024.
You are Granite, developed by IBM. Write the response to the user's input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data.<|end_of_text|>
<|start_of_role|>documents<|end_of_role|>
Document 0
My name is Gabe

I work for IBM<|end_of_text|>
<|start_of_role|>user<|end_of_role|>Who do I work for?<|end_of_text|>
<|start_of_role|>assistant<|end_of_role|>

Note that the document elements were both combined into a single Document 0 and the control elements were completely ignored.

Proposed Solutions

There are two candidate solutions here:

  1. Remove the collation entirely: This is simpler, but also less backwards-compatible
  2. Limit the collation to user, assistant, and system roles
Originally created by @gabe-l-hart on GitHub (Mar 21, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/9929 ## Description The current code [here](https://github.com/ollama/ollama/blob/main/template/template.go#L222) will automatically collapse multiple elements of the `messages` list where the `role` string is identical into a single message entry with the `content` strings concatenated. With the addition of new roles such as `control` and `document` that are designed to inform the chat template when creating the prompt string, this logic causes problems if multiple such messages are given sequentially. For example, with the [granite3.2 template](https://ollama.com/library/granite3.2/blobs/3e7ca51acd6e), consider the following request: ```json { "messages": [ {"role": "document", "content": "My name is Gabe"}, {"role": "document", "content": "I work for IBM"}, {"role": "control", "content": "citations"}, {"role": "control", "content": "hallucinations"}, {"role": "control", "content": "length long"}, {"role": "user", "content": "Who do I work for?"} ] } ``` This should result in the following: ``` <|start_of_role|>system<|end_of_role|>Knowledge Cutoff Date: April 2024. You are Granite, developed by IBM. Write the response to the user's input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data. In your response, use the symbols <co> and </co> to indicate when a fact comes from a document in the search result, e.g <co>0</co> for a fact from document 0. Afterwards, list all the citations with their corresponding documents in an ordered list. Finally, after the response is written, include a numbered list of sentences from the response that are potentially hallucinated and not based in the documents.<|end_of_text|> <|start_of_role|>documents<|end_of_role|> Document 0 My name is Gabe Document 1 I work for IBM<|end_of_text|> <|start_of_role|>user<|end_of_role|>Who do I work for?<|end_of_text|> <|start_of_role|>assistant {"length": "long"}<|end_of_role|> ``` but with collation, it instead produces the following: ``` <|start_of_role|>system<|end_of_role|>Knowledge Cutoff Date: April 2024. You are Granite, developed by IBM. Write the response to the user's input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data.<|end_of_text|> <|start_of_role|>documents<|end_of_role|> Document 0 My name is Gabe I work for IBM<|end_of_text|> <|start_of_role|>user<|end_of_role|>Who do I work for?<|end_of_text|> <|start_of_role|>assistant<|end_of_role|> ``` Note that the `document` elements were both combined into a single `Document 0` and the `control` elements were completely ignored. ## Proposed Solutions There are two candidate solutions here: 1. Remove the collation entirely: This is simpler, but also less backwards-compatible 2. Limit the collation to `user`, `assistant`, and `system` roles
GiteaMirror added the feature request label 2026-04-22 13:21:12 -05:00
GiteaMirror commented 2026-04-22 13:21:34 -05:00
Author
Owner
Copy Link

@gabe-l-hart commented on GitHub (Mar 21, 2025):

This relates to the recent changes to allow additional roles: https://github.com/ollama/ollama/pull/9874

<!-- gh-comment-id:2744146772 --> @gabe-l-hart commented on GitHub (Mar 21, 2025): This relates to the recent changes to allow additional roles: https://github.com/ollama/ollama/pull/9874
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#32260
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?