github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #3839] Feature Request: Detect Truncation Due to Exceeding Context Size #48889

New Issue
Open
opened 2026-04-28 09:58:27 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-04-28 09:58:27 -05:00
Owner
Copy Link

Originally created by @guoxf on GitHub (Apr 23, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3839

Understanding whether model output has been truncated due to exceeding context size is crucial for trusting the model to provide complete and accurate information. Here are some specific examples that illustrate why it's necessary to know if the output has been truncated:

  1. Question-Answering Systems: When building a question-answering system, if the answer to a user's query is truncated due to exceeding context size, the user may receive incomplete or inaccurate answers, which can degrade user experience and system reliability.
  2. Text Summarization: When using a model to generate summaries of articles or reports, if the summary is truncated due to exceeding context size, it may omit key information, leading to incomplete understanding of the original content.
  3. Chatbots: In chatbot applications, if a response in a conversation is truncated due to exceeding context size, it may cause the dialogue flow to be interrupted, affecting the user's interaction experience with the bot.
  4. Content Generation: When using a model to generate articles, stories, or other creative content, if the generated text is truncated due to exceeding context size, it may disrupt the coherence and completeness of the content.
  5. Machine Translation: In machine translation scenarios, if the translated text is truncated due to exceeding context size, it may result in the loss of the latter part of the translation, affecting translation accuracy.
  6. Natural Language Processing Tasks: When dealing with long texts in any natural language processing task, such as sentiment analysis or topic classification, if the model's output is truncated due to exceeding context size, it may lead to incorrect processing results.
  7. Legal and Compliance: In applications with high legal or compliance requirements, the completeness and accuracy of information are crucial. If model output is truncated due to exceeding context size, it may violate these requirements and lead to serious consequences.
  8. Academic and Research: When using large language models in academic research, ensuring the completeness and accuracy of results is vital for validating research hypotheses and conclusions. If model output is truncated, it may negatively impact the effectiveness and reliability of the research.

Please consider implementing a mechanism within the API that clearly indicates whether the output has been truncated. This could be a boolean flag in the response payload or an error message that provides insight into the truncation.
Thank you for your attention to this matter. I believe this feature would greatly enhance the usability and trustworthiness of ollama for all users.

Originally created by @guoxf on GitHub (Apr 23, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3839 Understanding whether model output has been truncated due to exceeding context size is crucial for trusting the model to provide complete and accurate information. Here are some specific examples that illustrate why it's necessary to know if the output has been truncated: 1. **Question-Answering Systems**: When building a question-answering system, if the answer to a user's query is truncated due to exceeding context size, the user may receive incomplete or inaccurate answers, which can degrade user experience and system reliability. 2. **Text Summarization**: When using a model to generate summaries of articles or reports, if the summary is truncated due to exceeding context size, it may omit key information, leading to incomplete understanding of the original content. 3. **Chatbots**: In chatbot applications, if a response in a conversation is truncated due to exceeding context size, it may cause the dialogue flow to be interrupted, affecting the user's interaction experience with the bot. 4. **Content Generation**: When using a model to generate articles, stories, or other creative content, if the generated text is truncated due to exceeding context size, it may disrupt the coherence and completeness of the content. 5. **Machine Translation**: In machine translation scenarios, if the translated text is truncated due to exceeding context size, it may result in the loss of the latter part of the translation, affecting translation accuracy. 6. **Natural Language Processing Tasks**: When dealing with long texts in any natural language processing task, such as sentiment analysis or topic classification, if the model's output is truncated due to exceeding context size, it may lead to incorrect processing results. 7. **Legal and Compliance**: In applications with high legal or compliance requirements, the completeness and accuracy of information are crucial. If model output is truncated due to exceeding context size, it may violate these requirements and lead to serious consequences. 8. **Academic and Research**: When using large language models in academic research, ensuring the completeness and accuracy of results is vital for validating research hypotheses and conclusions. If model output is truncated, it may negatively impact the effectiveness and reliability of the research. Please consider implementing a mechanism within the API that clearly indicates whether the output has been truncated. This could be a boolean flag in the response payload or an error message that provides insight into the truncation. Thank you for your attention to this matter. I believe this feature would greatly enhance the usability and trustworthiness of ollama for all users.
GiteaMirror added the feature request label 2026-04-28 09:58:27 -05:00
GiteaMirror commented 2026-04-28 09:58:29 -05:00
Author
Owner
Copy Link

@dimitribellini commented on GitHub (Apr 23, 2024):

Good suggestion, I'm faced the same issue in my simple python scripts using the Ollama RestAPI...
In the beginning of my debuging session, I can not image a problem related to the "context" size.
Please add something to prevents such kind of issues.
Thanks so much
PS: Great project!!!

<!-- gh-comment-id:2072332410 --> @dimitribellini commented on GitHub (Apr 23, 2024): Good suggestion, I'm faced the same issue in my simple python scripts using the Ollama RestAPI... In the beginning of my debuging session, I can not image a problem related to the "context" size. Please add something to prevents such kind of issues. Thanks so much PS: Great project!!!
GiteaMirror commented 2026-04-28 09:58:29 -05:00
Author
Owner
Copy Link

@eamag commented on GitHub (Jul 1, 2024):

+1, and it's already logged on the cpp server side, just has to be propagated into the API response: 1963c00201/llm/ext_server/server.cpp (L1682)

<!-- gh-comment-id:2200658855 --> @eamag commented on GitHub (Jul 1, 2024): +1, and it's already logged on the cpp server side, just has to be propagated into the API response: https://github.com/ollama/ollama/blob/1963c00201958da7165a40f9d2f22b28e11be718/llm/ext_server/server.cpp#L1682
GiteaMirror commented 2026-04-28 09:58:30 -05:00
Author
Owner
Copy Link

@simplerick commented on GitHub (Apr 15, 2026):

any progress?

<!-- gh-comment-id:4253701354 --> @simplerick commented on GitHub (Apr 15, 2026): any progress?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#48889
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?