github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #2014] How to make output consistent #63203

New Issue
Closed
opened 2026-05-03 12:33:59 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-05-03 12:33:59 -05:00
Owner
Copy Link

Originally created by @Fei-Wang on GitHub (Jan 16, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/2014

Setting seed and temperature cannot make the output consistent.
image

Originally created by @Fei-Wang on GitHub (Jan 16, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/2014 Setting seed and temperature cannot make the output consistent. <img width="1087" alt="image" src="https://github.com/jmorganca/ollama/assets/11441526/9a00ac1f-c120-4211-9b2e-fcec627f69e1">
GiteaMirror commented 2026-05-03 12:34:01 -05:00
Author
Owner
Copy Link

@mxyng commented on GitHub (Jan 16, 2024):

I'm not able to reproduce this using llama2 and mistral with setting seed and temperature through both the API and the Modelfile.

What version of ollama (ollama -v) are you using? Can you also provide your Modelfile?

<!-- gh-comment-id:1894376882 --> @mxyng commented on GitHub (Jan 16, 2024): I'm not able to reproduce this using `llama2` and `mistral` with setting `seed` and `temperature` through both the API and the Modelfile. What version of ollama (`ollama -v`) are you using? Can you also provide your Modelfile?
GiteaMirror commented 2026-05-03 12:34:02 -05:00
Author
Owner
Copy Link

@Fei-Wang commented on GitHub (Jan 17, 2024):

I'm not able to reproduce this using llama2 and mistral with setting seed and temperature through both the API and the Modelfile.

What version of ollama (ollama -v) are you using? Can you also provide your Modelfile?

ollama -v
ollama version is 0.1.20

cat Modelfile

FROM ./q4_0.bin

TEMPLATE """{{ if .First }}{{ .System }}{{ end }}{{ .Prompt }} [/INST]{{ .Response }} </s><s>[INST] """

SYSTEM "[INST] "
PARAMETER stop "[INST]"
PARAMETER stop "[/INST]"
PARAMETER stop "<<SYS>>"
PARAMETER stop "<</SYS>>"

PARAMETER temperature 0
PARAMETER seed 37
PARAMETER num_ctx 4096
<!-- gh-comment-id:1894769439 --> @Fei-Wang commented on GitHub (Jan 17, 2024): > I'm not able to reproduce this using `llama2` and `mistral` with setting `seed` and `temperature` through both the API and the Modelfile. > > What version of ollama (`ollama -v`) are you using? Can you also provide your Modelfile? `ollama -v` ollama version is 0.1.20 `cat Modelfile` ``` FROM ./q4_0.bin TEMPLATE """{{ if .First }}{{ .System }}{{ end }}{{ .Prompt }} [/INST]{{ .Response }} </s><s>[INST] """ SYSTEM "[INST] " PARAMETER stop "[INST]" PARAMETER stop "[/INST]" PARAMETER stop "<<SYS>>" PARAMETER stop "<</SYS>>" PARAMETER temperature 0 PARAMETER seed 37 PARAMETER num_ctx 4096 ```
GiteaMirror commented 2026-05-03 12:34:03 -05:00
Author
Owner
Copy Link

@Fei-Wang commented on GitHub (Jan 22, 2024):

Hi @mxyng, could you please take a look at the Modelfile config I provided when you get a chance?
Thanks!

<!-- gh-comment-id:1903447644 --> @Fei-Wang commented on GitHub (Jan 22, 2024): Hi @mxyng, could you please take a look at the Modelfile config I provided when you get a chance? Thanks!
GiteaMirror commented 2026-05-03 12:34:04 -05:00
Author
Owner
Copy Link

@mxyng commented on GitHub (Jan 22, 2024):

@Fei-Wang what kind of model is q4_0.bin? The template may be incorrect. It should probably be something like this:

[INST] {{ .System }} {{ .Prompt }} [/INST]

<s> and </s> shouldn't be necessary and {{ .Response }} is (currently) ignored.

<!-- gh-comment-id:1904499547 --> @mxyng commented on GitHub (Jan 22, 2024): @Fei-Wang what kind of model is `q4_0.bin`? The template may be incorrect. It should probably be something like this: ``` [INST] {{ .System }} {{ .Prompt }} [/INST] ``` `<s>` and `</s>` shouldn't be necessary and `{{ .Response }}` is (currently) ignored.
GiteaMirror commented 2026-05-03 12:34:05 -05:00
Author
Owner
Copy Link

@Fei-Wang commented on GitHub (Jan 23, 2024):

Hey @mxyng,

I'm working with q4_0.bin, a finetuned llama2 model, and I've hit two snags:

  1. I'm using <s> and </s> as per the guide on Hugging Face (https://huggingface.co/blog/codellama#conversational-instructions). Did I get something wrong?

  2. Changing the ModelFile to llama2 hasn't fixed inconsistent outputs. See the screenshot below.

Any ideas?

Screenshot 2024-01-23 at 2 03 40 PM
<!-- gh-comment-id:1905362210 --> @Fei-Wang commented on GitHub (Jan 23, 2024): Hey @mxyng, I'm working with `q4_0.bin`, a finetuned llama2 model, and I've hit two snags: 1. I'm using `<s>` and `</s>` as per the guide on Hugging Face (https://huggingface.co/blog/codellama#conversational-instructions). Did I get something wrong? 2. Changing the ModelFile to `llama2` hasn't fixed inconsistent outputs. See the screenshot below. Any ideas? <img width="1323" alt="Screenshot 2024-01-23 at 2 03 40 PM" src="https://github.com/jmorganca/ollama/assets/11441526/e5cd6da1-7481-46de-add0-ac70bc1f8362">
GiteaMirror commented 2026-05-03 12:34:05 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Jan 27, 2024):

Closing this as a dupe of #1749

<!-- gh-comment-id:1912888885 --> @pdevine commented on GitHub (Jan 27, 2024): Closing this as a dupe of #1749
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#63203
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?