github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 05:05:37 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #7222] How to check the "maxResponse" that the Ollama model supports? #4583

New Issue
Closed
opened 2026-04-12 15:30:38 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-04-12 15:30:38 -05:00
Owner
Copy Link

Originally created by @goactiongo on GitHub (Oct 16, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/7222

What is the issue?

I wanna know How to check the "maxResponse" that the Ollama model supports? and how to set this parameter.

I can obtain the context length information using the following command:

(base) [root@gpu ~]# ollama show glm4:9b
  Model
    architecture        chatglm
    parameters          9.4B
    context length      131072
    embedding length    4096
    quantization        Q4_0

  Parameters
    stop    "<|system|>"
    stop    "<|user|>"
    stop    "<|assistant|>"

  License
    The glm-4-9b License

The following output content is incomplete, apparently due to the content being truncated by the maxResponse setting. Could you please tell me how to set the maxResponse parameter?

````````````glm4:9b``````````````````
Based on the provided document, here is a summary of China World Trade Center Co., Ltd.'s (CWT) 2024 half-year report:

Company Overview: CWT is primarily engaged in property investment and management, including office buildings, shopping malls, apartments, and hotel operations. It manages its business through its own efforts, except for the hotel business which is managed by Shangri-La International Hotel Management Limited.

Financial Highlights:

Revenue: Increased by 1.4% to RMB 19.7 billion.
Net Profit: Increased by 5.18% to RMB 0.68 per share.
Net Profit attributable to shareholders: Increased by 4.19% to RMB 674.6 million.
Operating Cash Flow: Decreased by 5.06% to RMB 9.42 billion.
Market Analysis:

Office market in Beijing faced challenges with low demand

OS

No response

GPU

No response

CPU

No response

Ollama version

No response

Originally created by @goactiongo on GitHub (Oct 16, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/7222 ### What is the issue? I wanna know How to check the "maxResponse" that the Ollama model supports? and how to set this parameter. I can obtain the context length information using the following command: ```shell (base) [root@gpu ~]# ollama show glm4:9b Model architecture chatglm parameters 9.4B context length 131072 embedding length 4096 quantization Q4_0 Parameters stop "<|system|>" stop "<|user|>" stop "<|assistant|>" License The glm-4-9b License ``` The following output content is incomplete, apparently due to the content being truncated by the maxResponse setting. Could you please tell me how to set the maxResponse parameter? ```shell ````````````glm4:9b`````````````````` Based on the provided document, here is a summary of China World Trade Center Co., Ltd.'s (CWT) 2024 half-year report: Company Overview: CWT is primarily engaged in property investment and management, including office buildings, shopping malls, apartments, and hotel operations. It manages its business through its own efforts, except for the hotel business which is managed by Shangri-La International Hotel Management Limited. Financial Highlights: Revenue: Increased by 1.4% to RMB 19.7 billion. Net Profit: Increased by 5.18% to RMB 0.68 per share. Net Profit attributable to shareholders: Increased by 4.19% to RMB 674.6 million. Operating Cash Flow: Decreased by 5.06% to RMB 9.42 billion. Market Analysis: Office market in Beijing faced challenges with low demand ``` ### OS _No response_ ### GPU _No response_ ### CPU _No response_ ### Ollama version _No response_
GiteaMirror added the bug label 2026-04-12 15:30:38 -05:00
GiteaMirror commented 2026-04-12 15:30:39 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Oct 16, 2024):

The maximum response is essentially the size of the context window minus the length of the input. You can lower this by setting num_predict in the API call.

<!-- gh-comment-id:2415986900 --> @rick-github commented on GitHub (Oct 16, 2024): The maximum response is essentially the size of the context window minus the length of the input. You can lower this by setting `num_predict` in the API call.
GiteaMirror commented 2026-04-12 15:30:39 -05:00
Author
Owner
Copy Link

@goactiongo commented on GitHub (Oct 16, 2024):

Thank you for your reply. I don't quite understand this part --- "lower this by setting num_predict". I hope to get as much output information as possible. Should I increase the num_predict parameter?

Or, is there another approach? Can I achieve more output information by reducing the length of the input content without adjusting the num_predict parameter?

<!-- gh-comment-id:2416042936 --> @goactiongo commented on GitHub (Oct 16, 2024): Thank you for your reply. I don't quite understand this part --- "lower this by setting num_predict". I hope to get as much output information as possible. Should I increase the num_predict parameter? Or, is there another approach? Can I achieve more output information by reducing the length of the input content without adjusting the num_predict parameter?
GiteaMirror commented 2026-04-12 15:30:40 -05:00
Author
Owner
Copy Link

@goactiongo commented on GitHub (Oct 16, 2024):

hi bro, I found these two pieces of information in the log. Please don't respond for now, as I've discovered a new issue: when the input context is relatively large, the response tokens sent to Ollama default to updating to 200 tokens (if the context is smaller, this generation token is the actual value I pass to Ollama). This is why the model's responses are incomplete. I will look into this 200-token issue and update the information. I'll come back to you for advice then

prompt eval time = 469.81 ms / 52919 tokens
generation eval time = 11332.11 ms / 200 runs

10月 16 20:50:49 gpu ollama[24746]: DEBUG [print_timings] prompt eval time     =     469.81 ms / 52919 tokens (    0.01 ms per token, 112638.67 tokens per second) | n_prompt_tokens_processed=52919 n_tokens_second=112638.6724902727 slot_id=0 t_prompt_processing=469.812 t_token=0.008877945539409286 task_id=209 tid="140372587372544" timestamp=1729083049
10月 16 20:50:49 gpu ollama[24746]: DEBUG [print_timings] generation eval time =   11332.11 ms /   200 runs   (   56.66 ms per token,    17.65 tokens per second) | n_decoded=200 n_tokens_second=17.6489638734534 slot_id=0 t_token=56.66055 t_token_generation=11332.11 task_id=209 tid="140372587372544" timestamp=1729083049
10月 16 20:50:49 gpu ollama[24746]: DEBUG [print_timings]           total time =   11801.92 ms | slot_id=0 t_prompt_processing=469.812 t_token_generation=11332.11 t_total=11801.922 task_id=209 tid="140372587372544" timestamp=1729083049
10月 16 20:50:49 gpu ollama[24746]: DEBUG [update_slots] slot released | n_cache_tokens=53119 n_ctx=128032 n_past=53118 n_system_tokens=0 slot_id=0 task_id=209 tid="140372587372544" timestamp=1729083049 truncated=false
<!-- gh-comment-id:2416783854 --> @goactiongo commented on GitHub (Oct 16, 2024): hi bro, I found these two pieces of information in the log. Please don't respond for now, as I've discovered a new issue: when the input context is relatively large, the response tokens sent to Ollama default to updating to 200 tokens (if the context is smaller, this generation token is the actual value I pass to Ollama). This is why the model's responses are incomplete. I will look into this 200-token issue and update the information. I'll come back to you for advice then prompt eval time = 469.81 ms / 52919 tokens generation eval time = 11332.11 ms / 200 runs ```shell 10月 16 20:50:49 gpu ollama[24746]: DEBUG [print_timings] prompt eval time = 469.81 ms / 52919 tokens ( 0.01 ms per token, 112638.67 tokens per second) | n_prompt_tokens_processed=52919 n_tokens_second=112638.6724902727 slot_id=0 t_prompt_processing=469.812 t_token=0.008877945539409286 task_id=209 tid="140372587372544" timestamp=1729083049 10月 16 20:50:49 gpu ollama[24746]: DEBUG [print_timings] generation eval time = 11332.11 ms / 200 runs ( 56.66 ms per token, 17.65 tokens per second) | n_decoded=200 n_tokens_second=17.6489638734534 slot_id=0 t_token=56.66055 t_token_generation=11332.11 task_id=209 tid="140372587372544" timestamp=1729083049 10月 16 20:50:49 gpu ollama[24746]: DEBUG [print_timings] total time = 11801.92 ms | slot_id=0 t_prompt_processing=469.812 t_token_generation=11332.11 t_total=11801.922 task_id=209 tid="140372587372544" timestamp=1729083049 10月 16 20:50:49 gpu ollama[24746]: DEBUG [update_slots] slot released | n_cache_tokens=53119 n_ctx=128032 n_past=53118 n_system_tokens=0 slot_id=0 task_id=209 tid="140372587372544" timestamp=1729083049 truncated=false ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#4583
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?