github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-08 17:49:24 -05:00
Code Issues 14.6k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #3775] Achieving Deterministic Output with Ollama #64368

New Issue
Closed
opened 2026-05-03 17:23:54 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-05-03 17:23:54 -05:00
Owner
Copy Link

Originally created by @antonkratz on GitHub (Apr 20, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3775

For a research project, I am interested in exploring the effect of different prompts. The problem is, when I change the prompt even slightly, and I get a different result, I am unable to say how much is because I changed the prompt input and how much is because of the random and pseudo-random effects because of concepts such as top-k, top-n and temperature.

Is it possible, in principle, to get a deterministic output? Is it technically possible to get a deterministic output in practice with ollama?

I am also thinking about things like multi-threading and things like RDRAND in Intel CPUs... but I do not know if RDRAND is actually used or if ollama uses a deterministic random number generating function where the seed could be fixed?!

Basically, I want to use ollama in a way that the same prompt generates the same output, at any temperature. There can and should be pseudo-randomness but it must be necessary for me to fix the seed. I want only changes that are caused by the prompt. Is that possible with ollama?

Originally created by @antonkratz on GitHub (Apr 20, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3775 For a research project, I am interested in exploring the effect of different prompts. The problem is, when I change the prompt even slightly, and I get a different result, I am unable to say how much is because I changed the prompt input and how much is because of the random and pseudo-random effects because of concepts such as top-k, top-n and temperature. Is it possible, in principle, to get a deterministic output? Is it technically possible to get a deterministic output in practice with ollama? I am also thinking about things like multi-threading and things like RDRAND in Intel CPUs... but I do not know if RDRAND is actually used or if ollama uses a deterministic random number generating function where the seed could be fixed?! Basically, I want to use ollama in a way that the same prompt generates the same output, at any temperature. There can and should be pseudo-randomness but it must be necessary for me to fix the seed. I want only changes that are caused by the prompt. Is that possible with ollama?
GiteaMirror commented 2026-05-03 17:23:57 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (May 14, 2024):

Hey @antonkratz sorry for the slow response. There's really only two ways I know of to get deterministic output from any given model in an LLM. Either set temperature to 0, or set top_k to 1. In both cases you're essentially just picking the next token with the highest probability. When you set the temperature higher you're injecting more randomness into the system. When you set top_k to 1 you constrain the next token to the most probable next token.

I hope that helps.

<!-- gh-comment-id:2111320453 --> @pdevine commented on GitHub (May 14, 2024): Hey @antonkratz sorry for the slow response. There's really only two ways I know of to get deterministic output from any given model in an LLM. *Either* set `temperature` to 0, *or* set `top_k` to 1. In both cases you're essentially just picking the next token with the highest probability. When you set the temperature higher you're injecting more randomness into the system. When you set top_k to 1 you constrain the next token to the most probable next token. I hope that helps.
GiteaMirror commented 2026-05-03 17:23:58 -05:00
Author
Owner
Copy Link

@cirosantilli commented on GitHub (Mar 18, 2025):

Related about fixing a seed: https://github.com/ollama/ollama/issues/2773 That seems to work?

OK it seems you can't set parameters from CLI yet https://github.com/ollama/ollama/issues/2505#issuecomment-2733425744 but we can expect it for now:

ollama-expect

#!/usr/bin/expect -f
set prompt ">>> "
log_user 0
spawn ollama run [lindex $argv 0]
expect $prompt
send "/set parameter seed 0\r"
expect $prompt
send "/set parameter temperature 0\r"
expect $prompt
send "/set parameter num_predict 100\r"
expect $prompt
send "[lindex $argv 1]\r"
expect -re "\n(.*?)$prompt"
puts -nonewline $expect_out(1,string)
send -- "/bye"

then:

./ollama-expect llama3.2 'What is quantum field theory?'

Another idea is to use llama-cli from llama.cpp, which is the backend for ollama. It has a --temperature flag and does not produce stray characters.

<!-- gh-comment-id:2732867990 --> @cirosantilli commented on GitHub (Mar 18, 2025): Related about fixing a seed: https://github.com/ollama/ollama/issues/2773 That seems to work? OK it seems you can't set parameters from CLI yet https://github.com/ollama/ollama/issues/2505#issuecomment-2733425744 but we can `expect` it for now: ollama-expect ``` #!/usr/bin/expect -f set prompt ">>> " log_user 0 spawn ollama run [lindex $argv 0] expect $prompt send "/set parameter seed 0\r" expect $prompt send "/set parameter temperature 0\r" expect $prompt send "/set parameter num_predict 100\r" expect $prompt send "[lindex $argv 1]\r" expect -re "\n(.*?)$prompt" puts -nonewline $expect_out(1,string) send -- "/bye" ``` then: ``` ./ollama-expect llama3.2 'What is quantum field theory?' ``` Another idea is to use llama-cli from llama.cpp, which is the backend for ollama. It has a `--temperature` flag and does not produce stray characters.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#64368
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?