github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #8386] Return in a response a flag if the input request was truncated #67439

New Issue
Closed
opened 2026-05-04 10:20:42 -05:00 by GiteaMirror · 1 comment
GiteaMirror commented 2026-05-04 10:20:42 -05:00
Owner
Copy Link

Originally created by @MarkWard0110 on GitHub (Jan 11, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/8386

Add to the response a flag that indicates true if Ollama truncated the input request.

As a developer, I would like the Ollama response to have a flag indicating that it truncated the input prompt so that I can initiate client-side behavior based on this information.

In some situations, the accuracy of the chat response depends upon processing the complete input prompt. A client is not informed if the input was truncated.

A client may be unable to match Ollama's calculation for the context size required for a given input.

The truncation works well for chat conversations, but the user may experience a degradation in the quality of the conversation.
The truncation might negatively affect an agent. An agent not interacting with a user might receive a longer-than-expected response in a previous iteration. In the following chat request, Ollama truncates the input prompt. The agent would not be aware the request was truncated. The agent may try to estimate but may not be able to match Ollama's calculation.

It also appears that in certain situations, the routes.go chatPrompt call will not truncate the input prompt, but the runner will. I saw this when I had put the context request size one less than the reported prompt eval for the same request. For example, a chat request that the prompt eval is 13. I sent the same request but with a context size of 12. The eval is 55. So, the runner also hit the context limit.

Originally created by @MarkWard0110 on GitHub (Jan 11, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/8386 Add to the response a flag that indicates true if Ollama truncated the input request. As a developer, I would like the Ollama response to have a flag indicating that it truncated the input prompt so that I can initiate client-side behavior based on this information. In some situations, the accuracy of the chat response depends upon processing the complete input prompt. A client is not informed if the input was truncated. A client may be unable to match Ollama's calculation for the context size required for a given input. The truncation works well for chat conversations, but the user may experience a degradation in the quality of the conversation. The truncation might negatively affect an agent. An agent not interacting with a user might receive a longer-than-expected response in a previous iteration. In the following chat request, Ollama truncates the input prompt. The agent would not be aware the request was truncated. The agent may try to estimate but may not be able to match Ollama's calculation. It also appears that in certain situations, the routes.go chatPrompt call will not truncate the input prompt, but the runner will. I saw this when I had put the context request size one less than the reported prompt eval for the same request. For example, a chat request that the prompt eval is 13. I sent the same request but with a context size of 12. The eval is 55. So, the runner also hit the context limit.
GiteaMirror added the feature request label 2026-05-04 10:20:42 -05:00
GiteaMirror commented 2026-05-04 10:20:45 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 12, 2025):

#7043, #3839, #1005, #4967

<!-- gh-comment-id:2585535204 --> @rick-github commented on GitHub (Jan 12, 2025): #7043, #3839, #1005, #4967
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#67439
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?