github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-08 00:51:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #5297] How to get same length of response from CLI and API? #3322

New Issue
Closed
opened 2026-04-12 13:53:34 -05:00 by GiteaMirror · 10 comments
GiteaMirror commented 2026-04-12 13:53:34 -05:00
Owner
Copy Link

Originally created by @dsbyprateekg on GitHub (Jun 26, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/5297

What is the issue?

Hi,

I checked the prompt 'why is the sky blue?' with CLI and with the API through postman.
The response generated in CLI is longer than the response generated with the API-

api_response_gemma2b

cli_response_gemma2b

Is there any parameter in API, we can set to get the longer response as we are getting in CLI?

I am running the ollama server in my Windows 11 laptop with AMD Ryzen CPU.

OS

Windows

GPU

AMD

CPU

AMD

Ollama version

0.1.46

Originally created by @dsbyprateekg on GitHub (Jun 26, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/5297 ### What is the issue? Hi, I checked the prompt 'why is the sky blue?' with CLI and with the API through postman. The response generated in CLI is longer than the response generated with the API- ![api_response_gemma2b](https://github.com/ollama/ollama/assets/30830541/a142983d-9c57-466f-a920-e3b4966870a1) ![cli_response_gemma2b](https://github.com/ollama/ollama/assets/30830541/127a6bf9-ede2-4544-8a31-a4dccd5e24d9) Is there any parameter in API, we can set to get the longer response as we are getting in CLI? I am running the ollama server in my Windows 11 laptop with AMD Ryzen CPU. ### OS Windows ### GPU AMD ### CPU AMD ### Ollama version 0.1.46
GiteaMirror added the bug label 2026-04-12 13:53:34 -05:00
GiteaMirror commented 2026-04-12 13:53:36 -05:00
Author
Owner
Copy Link

@d-kleine commented on GitHub (Jun 27, 2024):

Try to set a fixed context length with

"options": {"num_ctx": 2048}

You can try a different value as long it does not exceed the context length of your model.

If you want your output to be fully reproducible, your need to make your model deterministic:

"options": {  
            "seed": 42,
            "temperature": 0,
            "num_ctx": 2048
        }

See Ollama API docs and model params docs.

Does this solve your problem?

<!-- gh-comment-id:2194121924 --> @d-kleine commented on GitHub (Jun 27, 2024): Try to set a fixed context length with ```json "options": {"num_ctx": 2048} ``` You can try a different value as long it does not exceed the context length of your model. If you want your output to be fully reproducible, your need to make your model deterministic: ```json "options": { "seed": 42, "temperature": 0, "num_ctx": 2048 } ``` See [Ollama API docs](https://github.com/ollama/ollama/blob/main/docs/api.md#chat-request-reproducible-outputs) and [model params docs](https://github.com/ollama/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values). Does this solve your problem?
GiteaMirror commented 2026-04-12 13:53:37 -05:00
Author
Owner
Copy Link

@dsbyprateekg commented on GitHub (Jun 27, 2024):

@d-kleine Thanks for your reply.
I tried the "options": {"num_ctx": 8000} , but still it's answer is shorter than the response in the CLI.
image

And if I use seed and temperature then it's gives shorter answer-
image

<!-- gh-comment-id:2194148440 --> @dsbyprateekg commented on GitHub (Jun 27, 2024): @d-kleine Thanks for your reply. I tried the "options": {"num_ctx": 8000} , but still it's answer is shorter than the response in the CLI. ![image](https://github.com/ollama/ollama/assets/30830541/5c5c3b00-8d8e-4d4d-b81e-a024bcc54799) And if I use seed and temperature then it's gives shorter answer- ![image](https://github.com/ollama/ollama/assets/30830541/516052fc-cada-437f-a942-c4f9e76c079b)
GiteaMirror commented 2026-04-12 13:53:38 -05:00
Author
Owner
Copy Link

@d-kleine commented on GitHub (Jun 27, 2024):

Can you show me the input and output of the CLI?

The input should contain the same "options" parameters you have used in the API request.

<!-- gh-comment-id:2194191047 --> @d-kleine commented on GitHub (Jun 27, 2024): Can you show me the input and output of the CLI? The input should contain the same "options" parameters you have used in the API request.
GiteaMirror commented 2026-04-12 13:53:39 -05:00
Author
Owner
Copy Link

@dsbyprateekg commented on GitHub (Jun 27, 2024):

Sure, here is the cli response-
image

My question was to get same response in API as we got in cli. Please help me to achieve that.

<!-- gh-comment-id:2194207418 --> @dsbyprateekg commented on GitHub (Jun 27, 2024): Sure, here is the cli response- ![image](https://github.com/ollama/ollama/assets/30830541/40a8c500-2759-466b-94be-1e739f3ab295) My question was to get same response in API as we got in cli. Please help me to achieve that.
GiteaMirror commented 2026-04-12 13:53:40 -05:00
Author
Owner
Copy Link

@d-kleine commented on GitHub (Jun 27, 2024):

I believe this won't work like you did it as you need to steer the params, which is not possible with the CLI (at least as far as I see). With curl, you could do something like that:

curl -X POST http://localhost:11434/api/generate -d '{
  "model": "gemma:2b",
  "prompt": "Why is the sky blue?",
  "options": {
    "seed": 0,
    "temperature": 0,
    "num_ctx": 2048
  }
}'

See the API docs

<!-- gh-comment-id:2194249612 --> @d-kleine commented on GitHub (Jun 27, 2024): I believe this won't work like you did it as you need to steer the params, which is not possible with the CLI (at least as far as I see). With curl, you could do something like that: ```bash curl -X POST http://localhost:11434/api/generate -d '{ "model": "gemma:2b", "prompt": "Why is the sky blue?", "options": { "seed": 0, "temperature": 0, "num_ctx": 2048 } }' ``` See the [API docs](https://github.com/ollama/ollama/blob/main/docs/api.md)
GiteaMirror commented 2026-04-12 13:53:42 -05:00
Author
Owner
Copy Link

@dsbyprateekg commented on GitHub (Jun 27, 2024):

@d-kleine Issue is not with the cli.
Infact it is generating a very good response.

I was asking how to get the same length response in API?

<!-- gh-comment-id:2194281813 --> @dsbyprateekg commented on GitHub (Jun 27, 2024): @d-kleine Issue is not with the cli. Infact it is generating a very good response. I was asking how to get the same length response in API?
GiteaMirror commented 2026-04-12 13:53:43 -05:00
Author
Owner
Copy Link

@d-kleine commented on GitHub (Jun 27, 2024):

You could try to use the same params like here with the API put in the "options" (with seed, etc.), it might produce the same results. But I am not very optimistic about that because it seems like you cannot steer the generated output via the CLI. Therefore, your generated tokens will slightly vary and there generated different (and with different length) outputs.

<!-- gh-comment-id:2194298833 --> @d-kleine commented on GitHub (Jun 27, 2024): You could try to use the same params like [here](https://ollama.com/library/gemma:2b/blobs/22a838ceb7fb) with the API put in the "options" (with seed, etc.), it might produce the same results. But I am not very optimistic about that because it seems like you cannot steer the generated output via the CLI. Therefore, your generated tokens will slightly vary and there generated different (and with different length) outputs.
GiteaMirror commented 2026-04-12 13:53:44 -05:00
Author
Owner
Copy Link

@dsbyprateekg commented on GitHub (Jun 27, 2024):

I tried but no success.
Still response length generated in CLI is quite different to the response in API.
I don't know the reason but API should also generate the same length response as cli.

<!-- gh-comment-id:2194315348 --> @dsbyprateekg commented on GitHub (Jun 27, 2024): I tried but no success. Still response length generated in CLI is quite different to the response in API. I don't know the reason but API should also generate the same length response as cli.
GiteaMirror commented 2026-04-12 13:53:45 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jun 27, 2024):

Your role should be user, not assistant.

Compare:

$ curl -s http://ollama:11434/api/chat -d '{"model":"gemma:2b","messages":[{"role":"assistant","content":"why is the sky blue?"}],"format":"","options":{},"stream": false}'|jq -r .message.content


The sky is blue because of a phenomenon called Rayleigh scattering. 

Rayleigh scattering is the scattering of electromagnetic radiation by particles smaller than the wavelength of the radiation. In the case of the sky, the particles that scatter sunlight are molecules in the atmosphere. 

These molecules are smaller than the wavelength of visible light, which means that they scatter blue light more effectively than other colors of light. 

This is why the sky appears blue during the day and red at night

$ curl -s http://ollama:11434/api/chat -d '{"model":"gemma:2b","messages":[{"role":"user","content":"why is the sky blue?"}],"format":"","options":{},"stream": false}'|jq -r .message.content
The sky appears blue due to a phenomenon called Rayleigh scattering.

**Rayleigh Scattering:**
- When sunlight enters the atmosphere, it is scattered by molecules in the air.
- The blue light, which has a shorter wavelength, is scattered more intensely than other colors.
- This scattering effect is responsible for the blue color we perceive in the sky at sunrise and sunset.

**Explanation:**
- The blue wavelengths of sunlight are scattered more frequently by molecules in the Earth's atmosphere compared to other colors.
- This scattering is caused by the interaction of sunlight with the molecules, where blue light is scattered more effectively than other colors.
- As sunlight enters our eyes from the sky, the scattered blue light reaches our retina, giving the sky its blue appearance.

**Additional Factors:**
- The density of the atmosphere and the angle of sunlight also influence the intensity of blue scattering.
- Clouds, pollution, and atmospheric conditions can also affect the color of the sky.

But the content will vary run to run, as seed and temperature have a randomizing effect.

<!-- gh-comment-id:2195821958 --> @rick-github commented on GitHub (Jun 27, 2024): Your `role` should be `user`, not `assistant`. Compare: ``` $ curl -s http://ollama:11434/api/chat -d '{"model":"gemma:2b","messages":[{"role":"assistant","content":"why is the sky blue?"}],"format":"","options":{},"stream": false}'|jq -r .message.content The sky is blue because of a phenomenon called Rayleigh scattering. Rayleigh scattering is the scattering of electromagnetic radiation by particles smaller than the wavelength of the radiation. In the case of the sky, the particles that scatter sunlight are molecules in the atmosphere. These molecules are smaller than the wavelength of visible light, which means that they scatter blue light more effectively than other colors of light. This is why the sky appears blue during the day and red at night ``` ``` $ curl -s http://ollama:11434/api/chat -d '{"model":"gemma:2b","messages":[{"role":"user","content":"why is the sky blue?"}],"format":"","options":{},"stream": false}'|jq -r .message.content The sky appears blue due to a phenomenon called Rayleigh scattering. **Rayleigh Scattering:** - When sunlight enters the atmosphere, it is scattered by molecules in the air. - The blue light, which has a shorter wavelength, is scattered more intensely than other colors. - This scattering effect is responsible for the blue color we perceive in the sky at sunrise and sunset. **Explanation:** - The blue wavelengths of sunlight are scattered more frequently by molecules in the Earth's atmosphere compared to other colors. - This scattering is caused by the interaction of sunlight with the molecules, where blue light is scattered more effectively than other colors. - As sunlight enters our eyes from the sky, the scattered blue light reaches our retina, giving the sky its blue appearance. **Additional Factors:** - The density of the atmosphere and the angle of sunlight also influence the intensity of blue scattering. - Clouds, pollution, and atmospheric conditions can also affect the color of the sky. ``` But the content will vary run to run, as `seed` and `temperature` have a randomizing effect.
GiteaMirror commented 2026-04-12 13:53:46 -05:00
Author
Owner
Copy Link

@dsbyprateekg commented on GitHub (Jun 28, 2024):

@rick-github thanks, I changed assistant to user and it gave me longer response as cli.

<!-- gh-comment-id:2196092530 --> @dsbyprateekg commented on GitHub (Jun 28, 2024): @rick-github thanks, I changed assistant to user and it gave me longer response as cli.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#3322
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?