github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #4660] Changing seed does not change response #49440

New Issue
Closed
opened 2026-04-28 11:50:09 -05:00 by GiteaMirror · 1 comment
GiteaMirror commented 2026-04-28 11:50:09 -05:00
Owner
Copy Link

Originally created by @ccreutzi on GitHub (May 27, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/4660

Originally assigned to: @jmorganca on GitHub.

What is the issue?

According to the documentation, getting reproducible outputs requires setting the seed and setting temperature to 0.

As far as I can tell, the part of these that works is setting the temperature to 0. But changing the seed does not change the response I get.

$ curl -s http://localhost:11434/api/generate -d '{
  "model": "mistral",
  "prompt": "Why is the sky blue?",
  "options": {
    "seed": 1234,
    "temperature": 0
  }, "stream": false
}' | jq .response
" The sky appears blue due to a process called Rayleigh scattering. As sunlight reaches Earth, it is made up of different wavelengths or colors. Short-wavelength light (blue and violet) is scattered more easily than longer-wavelength light (red, orange, yellow, etc.) because it interacts more with the molecules in the Earth's atmosphere.\n\nThe blue light gets scattered in all directions, making the sky appear blue to us. However, you might wonder why we don't see a violet sky since violet light is scattered even more than blue light. This is because our eyes are more sensitive to blue light and because sunlight reaches us with less violet light due to the ozone layer absorbing some of it.\n\nAt sunrise and sunset, you may observe different colors in the sky, such as red or orange. This happens because at those times, the sun is low on the horizon, and its light has to pass through more atmosphere. The shorter blue and violet wavelengths are scattered out of the line of sight, leaving longer wavelengths like red, orange, and yellow to reach our eyes."
$ curl -s http://localhost:11434/api/generate -d '{
  "model": "mistral",
  "prompt": "Why is the sky blue?",
  "options": {
    "seed": 123,
    "temperature": 0
  }, "stream": false
}' | jq .response
" The sky appears blue due to a process called Rayleigh scattering. As sunlight reaches Earth, it is made up of different wavelengths or colors. Short-wavelength light (blue and violet) is scattered more easily than longer-wavelength light (red, orange, yellow, etc.) because it interacts more with the molecules in the Earth's atmosphere.\n\nThe blue light gets scattered in all directions, making the sky appear blue to us. However, you might wonder why we don't see a violet sky since violet light is scattered even more than blue light. This is because our eyes are more sensitive to blue light and because sunlight reaches us with less violet light due to the ozone layer absorbing some of it.\n\nAt sunrise and sunset, you may observe different colors in the sky, such as red or orange. This happens because at those times, the sun is low on the horizon, and its light has to pass through more atmosphere. The shorter blue and violet wavelengths are scattered out of the line of sight, leaving longer wavelengths like red, orange, and yellow to reach our eyes."

(I did test other seeds as well, with no change.)

My expectation would have been that seed is used for all (pseudo-)random choices in the response generation, and setting a fixed seed would result in a deterministic response, including with a positive temperature.

OS

macOS

GPU

Apple

CPU

Apple

Ollama version

0.1.38, installed with Homebrew

Originally created by @ccreutzi on GitHub (May 27, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/4660 Originally assigned to: @jmorganca on GitHub. ### What is the issue? According to [the documentation](https://github.com/ollama/ollama/blob/main/docs/api.md#request-reproducible-outputs), getting reproducible outputs requires setting the seed and setting temperature to 0. As far as I can tell, the part of these that works is setting the temperature to 0. But changing the seed does not change the response I get. ``` $ curl -s http://localhost:11434/api/generate -d '{ "model": "mistral", "prompt": "Why is the sky blue?", "options": { "seed": 1234, "temperature": 0 }, "stream": false }' | jq .response " The sky appears blue due to a process called Rayleigh scattering. As sunlight reaches Earth, it is made up of different wavelengths or colors. Short-wavelength light (blue and violet) is scattered more easily than longer-wavelength light (red, orange, yellow, etc.) because it interacts more with the molecules in the Earth's atmosphere.\n\nThe blue light gets scattered in all directions, making the sky appear blue to us. However, you might wonder why we don't see a violet sky since violet light is scattered even more than blue light. This is because our eyes are more sensitive to blue light and because sunlight reaches us with less violet light due to the ozone layer absorbing some of it.\n\nAt sunrise and sunset, you may observe different colors in the sky, such as red or orange. This happens because at those times, the sun is low on the horizon, and its light has to pass through more atmosphere. The shorter blue and violet wavelengths are scattered out of the line of sight, leaving longer wavelengths like red, orange, and yellow to reach our eyes." $ curl -s http://localhost:11434/api/generate -d '{ "model": "mistral", "prompt": "Why is the sky blue?", "options": { "seed": 123, "temperature": 0 }, "stream": false }' | jq .response " The sky appears blue due to a process called Rayleigh scattering. As sunlight reaches Earth, it is made up of different wavelengths or colors. Short-wavelength light (blue and violet) is scattered more easily than longer-wavelength light (red, orange, yellow, etc.) because it interacts more with the molecules in the Earth's atmosphere.\n\nThe blue light gets scattered in all directions, making the sky appear blue to us. However, you might wonder why we don't see a violet sky since violet light is scattered even more than blue light. This is because our eyes are more sensitive to blue light and because sunlight reaches us with less violet light due to the ozone layer absorbing some of it.\n\nAt sunrise and sunset, you may observe different colors in the sky, such as red or orange. This happens because at those times, the sun is low on the horizon, and its light has to pass through more atmosphere. The shorter blue and violet wavelengths are scattered out of the line of sight, leaving longer wavelengths like red, orange, and yellow to reach our eyes." ``` (I did test other seeds as well, with no change.) ---- My expectation would have been that `seed` is used for all (pseudo-)random choices in the response generation, and setting a fixed seed would result in a deterministic response, including with a positive temperature. ### OS macOS ### GPU Apple ### CPU Apple ### Ollama version 0.1.38, installed with Homebrew
GiteaMirror added the bug label 2026-04-28 11:50:09 -05:00
GiteaMirror commented 2026-04-28 11:50:11 -05:00
Author
Owner
Copy Link

@trebor commented on GitHub (May 29, 2024):

i'm seeing the same thing.

<!-- gh-comment-id:2138148882 --> @trebor commented on GitHub (May 29, 2024): i'm seeing the same thing.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#49440
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?