github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-11-13 17:34:40 -06:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #2352] API streaming and non streaming mode produces garbage output after the first query #63402

New Issue
Closed
opened 2026-05-03 13:21:06 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-05-03 13:21:06 -05:00
Owner
Copy Link
Copy Source

Originally created by @nextdimension on GitHub (Feb 4, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/2352

Originally assigned to: @bmizerany on GitHub.

When I use the ollama API the first response works fine, then without changing anything, subsequent requests give a response as if its ignoring the system prompt and spits out garbage. Restarting the ollama service works, until the second query.

Latest version and one version previous had the same issue.
My setup:
I installed it using the install bash script on linux. Its running as a system service. Its running under my own user account and group as thats the only way I could get the OLLAMA_MODEL dir ENV VAR to work. GPU acceleration. NVIDIA 2X GPUS. 3090 && A6000. CPU - AMD Ryzen 9 5900X 12-Core. I can see the VRAM gets almost filled on the A6000, and the 3090 is half FULL. Then ollama hangs and GPU (A6000) is at 100%. The hang is caused by having Ollama-webui running under docker and then trying to use the API.
In the tests below I stopped the ollama-webui docker container and restarted the ollama-service before testing.

Example Query:

curl http://localhost:11434/api/generate -d '{
"model": "llama2",
"prompt": "C programming langugage",
"system": "You are poet write poems about topics given in the prompt",
"stream": false
}'

curl http://localhost:11434/api/generate -d '{
"model": "llama2",
"prompt": "C programming langugage",
"system": "You are poet write poems about topics given in the prompt",
"stream": true
}'

Originally created by @nextdimension on GitHub (Feb 4, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/2352 Originally assigned to: @bmizerany on GitHub. When I use the ollama API the first response works fine, then without changing anything, subsequent requests give a response as if its ignoring the system prompt and spits out garbage. Restarting the ollama service works, until the second query. Latest version and one version previous had the same issue. My setup: I installed it using the install bash script on linux. Its running as a system service. Its running under my own user account and group as thats the only way I could get the OLLAMA_MODEL dir ENV VAR to work. GPU acceleration. NVIDIA 2X GPUS. 3090 && A6000. CPU - AMD Ryzen 9 5900X 12-Core. I can see the VRAM gets almost filled on the A6000, and the 3090 is half FULL. Then ollama hangs and GPU (A6000) is at 100%. The hang is caused by having Ollama-webui running under docker and then trying to use the API. In the tests below I stopped the ollama-webui docker container and restarted the ollama-service before testing. Example Query: curl http://localhost:11434/api/generate -d '{ "model": "llama2", "prompt": "C programming langugage", "system": "You are poet write poems about topics given in the prompt", "stream": false }' curl http://localhost:11434/api/generate -d '{ "model": "llama2", "prompt": "C programming langugage", "system": "You are poet write poems about topics given in the prompt", "stream": true }'
GiteaMirror commented 2026-05-03 13:21:12 -05:00
Author
Owner
Copy Link
Copy Source

@bmizerany commented on GitHub (Mar 12, 2024):

Could you please provide the model your using?

Will you also please try to reproduce with the ollama cli and see if you see the same problem?

<!-- gh-comment-id:1989684683 --> @bmizerany commented on GitHub (Mar 12, 2024): Could you please provide the model your using? Will you also please try to reproduce with the ollama cli and see if you see the same problem?
GiteaMirror commented 2026-05-03 13:21:15 -05:00
Author
Owner
Copy Link
Copy Source

@nextdimension commented on GitHub (Mar 12, 2024):

wow, i forgot about this.
if you read the original post it has all the answers you need…. thanks but i gave up using ollama after i spoke to the devs and community on discord and no one bothered taking it seriously, goodbye.

<!-- gh-comment-id:1990517055 --> @nextdimension commented on GitHub (Mar 12, 2024): wow, i forgot about this. if you read the original post it has all the answers you need…. thanks but i gave up using ollama after i spoke to the devs and community on discord and no one bothered taking it seriously, goodbye.
GiteaMirror commented 2026-05-03 13:21:17 -05:00
Author
Owner
Copy Link
Copy Source

@bmizerany commented on GitHub (Mar 12, 2024):

@nextdimension Thank you for the ticket. If you can reproduce with the latest version of Ollama, please feel free to reopen, but I'll close this for now.

FWIW: It sounds like you're running into resource limits and the restart frees up enough to start another round. Can you confirm that the next attempt after a restart exhibits the same behavior?

You are right, the model was in your ticket. That was a miss on my part.

<!-- gh-comment-id:1992126692 --> @bmizerany commented on GitHub (Mar 12, 2024): @nextdimension Thank you for the ticket. If you can reproduce with the latest version of Ollama, please feel free to reopen, but I'll close this for now. FWIW: It sounds like you're running into resource limits and the restart frees up enough to start another round. Can you confirm that the next attempt after a restart exhibits the same behavior? You are right, the model was in your ticket. That was a miss on my part.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#63402