github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-11-12 03:46:14 -06:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #3803] Does not produce same results via curl API for the same model. #48861

New Issue
Closed
opened 2026-04-28 09:55:24 -05:00 by GiteaMirror · 4 comments
GiteaMirror commented 2026-04-28 09:55:24 -05:00
Owner
Copy Link
Copy Source

Originally created by @MathematicianOnGithub on GitHub (Apr 21, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3803

What is the issue?

export OLLAMA_HOST="127.0.0.1:3336"; ollama create tagger -f tagger_Modelfile && ollama run tagger

Running the below does NOT produce the same result as in the terminal cli in the above command.

curl http://localhost:3336/api/generate -d '{
  "model": "tagger",
  "prompt": "hello"
}'

with or without streaming.
It seems like it is using a different model somehow, it just is not giving the same responses at all.

tagger_Modelfile:

FROM gemma:7b
PARAMETER temperature 0.5
PARAMETER num_ctx 8192

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>User
{{ .Prompt }}<|im_end|>
<|im_start|>Assistant
"""

SYSTEM """
Categorize user text using hierarchical tags.
You are to respond in one sentence. Example responses:
- #mathematics #linear-algebra #matrices #vectors
- #philosophy #epistemology
- #computer-science #algorithms #memoization
- #statistics #probability-theory
- #language #grammar
- #statistics #sample-theory #confidence-intervals
Especially important to get down whether it is calculus, analysis, combinatorics, etc.
"""
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"

MESSAGE user """
Matrix, a set of numbers arranged in rows and columns so as to form a rectangular array. The numbers are called the elements, or entries, of the matrix. Matrices have wide applications in engineering, physics, economics, and statistics as well as in various branches of mathematics.
"""
MESSAGE assistant #mathematics #linear-algebra #matrices
MESSAGE user """
the binomial coefficients are the positive integers that occur as coefficients in the binomial theorem. Commonly, a binomial coefficient is indexed by a pair of integers n ≥ k ≥ 0
"""
MESSAGE assistant #mathematics #combinatorics #binomial-coefficient

Help much appreciated!

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.1.32

Originally created by @MathematicianOnGithub on GitHub (Apr 21, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3803 ### What is the issue? `export OLLAMA_HOST="127.0.0.1:3336"; ollama create tagger -f tagger_Modelfile && ollama run tagger` Running the below does NOT produce the same result as in the terminal cli in the above command. ``` curl http://localhost:3336/api/generate -d '{ "model": "tagger", "prompt": "hello" }' ``` with or without streaming. It seems like it is using a different model somehow, it just is not giving the same responses at all. tagger_Modelfile: ``` FROM gemma:7b PARAMETER temperature 0.5 PARAMETER num_ctx 8192 TEMPLATE """<|im_start|>system {{ .System }}<|im_end|> <|im_start|>User {{ .Prompt }}<|im_end|> <|im_start|>Assistant """ SYSTEM """ Categorize user text using hierarchical tags. You are to respond in one sentence. Example responses: - #mathematics #linear-algebra #matrices #vectors - #philosophy #epistemology - #computer-science #algorithms #memoization - #statistics #probability-theory - #language #grammar - #statistics #sample-theory #confidence-intervals Especially important to get down whether it is calculus, analysis, combinatorics, etc. """ PARAMETER stop "<|im_start|>" PARAMETER stop "<|im_end|>" MESSAGE user """ Matrix, a set of numbers arranged in rows and columns so as to form a rectangular array. The numbers are called the elements, or entries, of the matrix. Matrices have wide applications in engineering, physics, economics, and statistics as well as in various branches of mathematics. """ MESSAGE assistant #mathematics #linear-algebra #matrices MESSAGE user """ the binomial coefficients are the positive integers that occur as coefficients in the binomial theorem. Commonly, a binomial coefficient is indexed by a pair of integers n ≥ k ≥ 0 """ MESSAGE assistant #mathematics #combinatorics #binomial-coefficient ``` Help much appreciated! ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.1.32
GiteaMirror added the bug label 2026-04-28 09:55:24 -05:00
GiteaMirror commented 2026-04-28 09:55:26 -05:00
Author
Owner
Copy Link
Copy Source

@igorschlum commented on GitHub (Apr 22, 2024):

Hi @MathematicianOnGithub Models don't give the same answer each time, specially for an open prompt like "Hello".
If you want answers to look very close the can set the temperature to 0.
Do you notice a bigger difference as if you use twice the same prompt without API?

<!-- gh-comment-id:2068398716 --> @igorschlum commented on GitHub (Apr 22, 2024): Hi @MathematicianOnGithub Models don't give the same answer each time, specially for an open prompt like "Hello". If you want answers to look very close the can set the temperature to 0. Do you notice a bigger difference as if you use twice the same prompt without API?
GiteaMirror commented 2026-04-28 09:55:28 -05:00
Author
Owner
Copy Link
Copy Source

@MathematicianOnGithub commented on GitHub (Apr 22, 2024):

Hi @igorschlum
Yes I tried using the same prompt 20 times out of 20 in the CLI REPL, I get expected response even after restarting.
Through the API 20 out of 20 observations I don't get expected response. Same when I changed the temperature.
So there is indeed a statistical significance here. I was up testing and trying stuff for 4 hours yesterday night, but to no avail.
I also listed the model through the API endpoint to verify, I also tried using "tagger:latest" as the model name which seemed more consistent with its name, but still to no avail.
I also tried different prompts of course, still to no avail.

<!-- gh-comment-id:2068581498 --> @MathematicianOnGithub commented on GitHub (Apr 22, 2024): Hi @igorschlum Yes I tried using the same prompt 20 times out of 20 in the CLI REPL, I get expected response even after restarting. Through the API 20 out of 20 observations I don't get expected response. Same when I changed the temperature. So there is indeed a statistical significance here. I was up testing and trying stuff for 4 hours yesterday night, but to no avail. I also listed the model through the API endpoint to verify, I also tried using "tagger:latest" as the model name which seemed more consistent with its name, but still to no avail. I also tried different prompts of course, still to no avail.
GiteaMirror commented 2026-04-28 09:55:29 -05:00
Author
Owner
Copy Link
Copy Source

@chigkim commented on GitHub (Apr 22, 2024):

Make sure you specify a seed, so random generator use the same seed every time.

<!-- gh-comment-id:2069310999 --> @chigkim commented on GitHub (Apr 22, 2024): Make sure you specify a seed, so random generator use the same seed every time.
GiteaMirror commented 2026-04-28 09:55:31 -05:00
Author
Owner
Copy Link
Copy Source

@MathematicianOnGithub commented on GitHub (Apr 25, 2024):

I made an assumption that their cli uses http as well, so I used tcpdump, and then it is starting to make sense:

E.....@.@..J............f....<.............
..#-.-..POST /api/chat HTTP/1.1
Host: 127.0.0.1:3336
User-Agent: ollama/0.1.32 (amd64 linux) Go/go1.22.2
Content-Length: 776
Accept: application/x-ndjson
Content-Type: application/json
Accept-Encoding: gzip

{"model":"tagger","messages":[{"role":"user","content":"\nMatrix, a set of numbers arranged in rows and columns so as to form a rectangular array. The numbers are called the elements, or entries, of the matrix. Matrices have wide applications in engineering, physics, economics, and statistics as well as in various branches of mathematics.\n"},{"role":"assistant","content":"#mathematics #linear-algebra #matrices"},{"role":"user","content":"\nthe binomial coefficients are the positive integers that occur as coefficients in the binomial theorem. Commonly, a binomial coefficient is indexed by a pair of integers n ... k ... 0\n"},{"role":"assistant","content":"#mathematics #combinatorics #binomial-coefficient"},{"role":"user","content":"hello"}],"format":"","options":{}}

It seems to be sending multiple requests along with the "messages", so basically the CLI is not 1:1 with the curl API.
but at least now I understand a little bit more how it works.

<!-- gh-comment-id:2077684521 --> @MathematicianOnGithub commented on GitHub (Apr 25, 2024): I made an assumption that their cli uses http as well, so I used tcpdump, and then it is starting to make sense: E.....@.@..J............f....<............. ..#-.-..POST /api/chat HTTP/1.1 Host: 127.0.0.1:3336 User-Agent: ollama/0.1.32 (amd64 linux) Go/go1.22.2 Content-Length: 776 Accept: application/x-ndjson Content-Type: application/json Accept-Encoding: gzip {"model":"tagger","messages":[{"role":"user","content":"\nMatrix, a set of numbers arranged in rows and columns so as to form a rectangular array. The numbers are called the elements, or entries, of the matrix. Matrices have wide applications in engineering, physics, economics, and statistics as well as in various branches of mathematics.\n"},{"role":"assistant","content":"#mathematics #linear-algebra #matrices"},{"role":"user","content":"\nthe binomial coefficients are the positive integers that occur as coefficients in the binomial theorem. Commonly, a binomial coefficient is indexed by a pair of integers n ... k ... 0\n"},{"role":"assistant","content":"#mathematics #combinatorics #binomial-coefficient"},{"role":"user","content":"hello"}],"format":"","options":{}} It seems to be sending multiple requests along with the "messages", so basically the CLI is not 1:1 with the curl API. but at least now I understand a little bit more how it works.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No labels bug
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#48861