github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 13:19:40 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #2325] Hallucination on successive generation #1343

New Issue
Closed
opened 2026-04-12 11:10:42 -05:00 by GiteaMirror · 9 comments
GiteaMirror commented 2026-04-12 11:10:42 -05:00
Owner
Copy Link

Originally created by @pltrdy on GitHub (Feb 2, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/2325

Originally assigned to: @bmizerany on GitHub.

Hi,

I'm probably missing something about how ollama works or am I facing a kind of side effect.

I use a super simple code to get 3 outputs given a simple input Why is the sky blue? (in order to check for reproducibility and stuff)

from ollama import Client

client = Client(HOST)

outputs = [client.generate("llama2", "Why is the sky blue?")["response"] for o in range(3)]

At some point the outputs becomes really messy:

  • outputs[0]:

The sky appears blue because of a phenomenon called Rayleigh scattering, which occurs when sunlight passes through the Earth's atmosphere. The shortest (blue) wavelengths of light are scattered in all directions by the tiny molecules of gases in the air, while the longer (red) wavelengths pass straight through the atmosphere with little scattering. This is why the sky appears blue during the daytime, as the blue light is scattered in all directions and reaches our eyes from every part of the sky.\n\nThe reason for this scattering effect has to do with the size of the molecules in the air and their wavelengths. The shorter wavelengths of light (such as blue and violet) have shorter wave lengths, which means that they have a smaller wavelength than the longer wavelengths (such as red and orange). As a result, these shorter wavelengths are more easily scattered by the tiny molecules in the air, while the longer wavelengths pass straight through with less scattering.\n\nIn addition to Rayleigh scattering, the sky can also appear blue due to the way that light interacts with the atmosphere at sunrise and sunset. During these times, the light from the sun has to travel through more of the Earth's atmosphere, which scatters the light in all directions and gives the sky a more reddish hue. This is known as Mie scattering and is responsible for the orange and red colors that we see during sunrise and sunset.\n\nSo, to summarize, the sky appears blue because of Rayleigh scattering, which occurs when light passes through the Earth's atmosphere and the shorter wavelengths of light are scattered in all directions by the tiny molecules of gases in the air. The longer wavelengths pass straight through with less scattering, which is why the sky appears blue during the daytime.

  • outputs[1]

ϊ (K) = 10\n Unterscheidung zwischen einer fremden und eigenen Währung (z. B.) 30,98 EUR = 250,00 USD\n\nIn der Regel ist es jedoch nicht ratsam, die Währung in einer Anzeige zu nennen, wenn dies nicht notwendig ist, um das Geräusch zu verdeutlichen. Stattdessen kann man sich an die Verwendung der fremden Währung anpassen und lediglich den Wert in der eigenen Währung anführen. Hier ist ein Beispiel für eine Anzeige, die nur den Wert in der eigenen Währung enthält:\n\nϊ (K) = 10\n\nDisziplinär kann man auch die Verwendung der fremden Währung vermeiden und stattdessen den Wert in der eigenen Währung immer anführen. Hier ist ein Beispiel für eine Anzeige, die nur den Wert in der eigenen Währung enthält:\n\n10 (K) = 30,98 EUR\n\nInsgesamt ist es jedoch wichtig, dass man sich an die Verwendung der fremden Währung anpassen kann, wenn dies notwendig ist, um den Leser zu verstehen, wie viel Geld er oder sie für eine bestimmte Angelegenheit benötigt.

  • outputs[2]:

Unterscheidung between the two models can be done using various statistical techniques, such as hypothesis testing or confidence intervals.его models are widely used in various fields such as finance, marketing, and economics.\n\nIn conclusion, both linear regression and logistic regression are powerful statistical tools that have numerous applications in various fields. While linear regression is used for predicting continuous outcomes, logistic regression is used for predicting categorical outcomes. Understanding the differences between these two models can help researchers choose the appropriate model for their data and research questions, ultimately leading to more accurate and informative results.

Any idea about how to fix that?

My objective is to get 3 times the same generation, I had in mind to set options like temperature and seed but this troubles me.

Originally created by @pltrdy on GitHub (Feb 2, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/2325 Originally assigned to: @bmizerany on GitHub. Hi, I'm probably missing something about how ollama works or am I facing a kind of side effect. I use a super simple code to get 3 outputs given a simple input `Why is the sky blue?` (in order to check for reproducibility and stuff) ``` from ollama import Client client = Client(HOST) outputs = [client.generate("llama2", "Why is the sky blue?")["response"] for o in range(3)] ``` At some point the outputs becomes really messy: - `outputs[0]`: > The sky appears blue because of a phenomenon called Rayleigh scattering, which occurs when sunlight passes through the Earth's atmosphere. The shortest (blue) wavelengths of light are scattered in all directions by the tiny molecules of gases in the air, while the longer (red) wavelengths pass straight through the atmosphere with little scattering. This is why the sky appears blue during the daytime, as the blue light is scattered in all directions and reaches our eyes from every part of the sky.\n\nThe reason for this scattering effect has to do with the size of the molecules in the air and their wavelengths. The shorter wavelengths of light (such as blue and violet) have shorter wave lengths, which means that they have a smaller wavelength than the longer wavelengths (such as red and orange). As a result, these shorter wavelengths are more easily scattered by the tiny molecules in the air, while the longer wavelengths pass straight through with less scattering.\n\nIn addition to Rayleigh scattering, the sky can also appear blue due to the way that light interacts with the atmosphere at sunrise and sunset. During these times, the light from the sun has to travel through more of the Earth's atmosphere, which scatters the light in all directions and gives the sky a more reddish hue. This is known as Mie scattering and is responsible for the orange and red colors that we see during sunrise and sunset.\n\nSo, to summarize, the sky appears blue because of Rayleigh scattering, which occurs when light passes through the Earth's atmosphere and the shorter wavelengths of light are scattered in all directions by the tiny molecules of gases in the air. The longer wavelengths pass straight through with less scattering, which is why the sky appears blue during the daytime. - outputs[1] > ϊ (K) = 10\n Unterscheidung zwischen einer fremden und eigenen Währung (z. B.) 30,98 EUR = 250,00 USD\n\nIn der Regel ist es jedoch nicht ratsam, die Währung in einer Anzeige zu nennen, wenn dies nicht notwendig ist, um das Geräusch zu verdeutlichen. Stattdessen kann man sich an die Verwendung der fremden Währung anpassen und lediglich den Wert in der eigenen Währung anführen. Hier ist ein Beispiel für eine Anzeige, die nur den Wert in der eigenen Währung enthält:\n\nϊ (K) = 10\n\nDisziplinär kann man auch die Verwendung der fremden Währung vermeiden und stattdessen den Wert in der eigenen Währung immer anführen. Hier ist ein Beispiel für eine Anzeige, die nur den Wert in der eigenen Währung enthält:\n\n10 (K) = 30,98 EUR\n\nInsgesamt ist es jedoch wichtig, dass man sich an die Verwendung der fremden Währung anpassen kann, wenn dies notwendig ist, um den Leser zu verstehen, wie viel Geld er oder sie für eine bestimmte Angelegenheit benötigt. - `outputs[2]`: > Unterscheidung between the two models can be done using various statistical techniques, such as hypothesis testing or confidence intervals.его models are widely used in various fields such as finance, marketing, and economics.\n\nIn conclusion, both linear regression and logistic regression are powerful statistical tools that have numerous applications in various fields. While linear regression is used for predicting continuous outcomes, logistic regression is used for predicting categorical outcomes. Understanding the differences between these two models can help researchers choose the appropriate model for their data and research questions, ultimately leading to more accurate and informative results. Any idea about how to fix that? My objective is to get 3 times the same generation, I had in mind to set options like `temperature` and `seed` but this troubles me.
GiteaMirror added the bug label 2026-04-12 11:10:42 -05:00
GiteaMirror commented 2026-04-12 11:10:43 -05:00
Author
Owner
Copy Link

@MichaelFomenko commented on GitHub (Feb 2, 2024):

Please provide more Information about your Hardware and Software Versions. And which Model version are you using.

<!-- gh-comment-id:1923836726 --> @MichaelFomenko commented on GitHub (Feb 2, 2024): Please provide more Information about your Hardware and Software Versions. And which Model version are you using.
GiteaMirror commented 2026-04-12 11:10:43 -05:00
Author
Owner
Copy Link

@pltrdy commented on GitHub (Feb 2, 2024):

It's running llama2 model on Colab with 1x V100 GPU following 09a6f76f4c/examples/jupyter-notebook/ollama.ipynb

<!-- gh-comment-id:1923840807 --> @pltrdy commented on GitHub (Feb 2, 2024): It's running llama2 model on Colab with 1x V100 GPU following https://github.com/ollama/ollama/blob/09a6f76f4c30fb8a9708680c519d08feeb504197/examples/jupyter-notebook/ollama.ipynb
GiteaMirror commented 2026-04-12 11:10:44 -05:00
Author
Owner
Copy Link

@easp commented on GitHub (Feb 3, 2024):

CUDA isn't deterministic unless the code is specifically designed for that, which generally comes at significant performance cost. Because of this some projects don't even support a deterministic mode.

That said, for troubleshooting purposes, I wonder how it would behave if the Ollama server was restarted between each successive request. By my reading of the code, the client doesn't carry any context, so successive calls for generate should be "fresh," but I wonder if the server is keeping some state (whether by design or accident).

<!-- gh-comment-id:1925436548 --> @easp commented on GitHub (Feb 3, 2024): CUDA isn't deterministic unless the code is specifically designed for that, which generally comes at significant performance cost. Because of this some projects don't even support a deterministic mode. That said, for troubleshooting purposes, I wonder how it would behave if the Ollama server was restarted between each successive request. By my reading of the code, the client doesn't carry any context, so successive calls for generate should be "fresh," but I wonder if the server is keeping some state (whether by design or accident).
GiteaMirror commented 2026-04-12 11:10:44 -05:00
Author
Owner
Copy Link

@pltrdy commented on GitHub (Feb 4, 2024):

I understand that CUDA should not be considered deterministic by default, therefore I would not bother to find small discrepancies from one run to another.

On the other hand, it seems to me that CUDA alone does not explain the huge gap that I found between generations, switching from perfectly useful answer to total nonsense in another language really quick.

<!-- gh-comment-id:1925758595 --> @pltrdy commented on GitHub (Feb 4, 2024): I understand that CUDA should not be considered deterministic by default, therefore I would not bother to find small discrepancies from one run to another. On the other hand, it seems to me that CUDA alone does not explain the huge gap that I found between generations, switching from perfectly useful answer to total nonsense in another language really quick.
GiteaMirror commented 2026-04-12 11:10:45 -05:00
Author
Owner
Copy Link

@MichaelFomenko commented on GitHub (Feb 4, 2024):

CUDA isn't deterministic unl

You need to provide more Information, like Model, the Temperature rate, the Quantization type and so on. In my Opinion the Quantization is the Problem.

<!-- gh-comment-id:1925810920 --> @MichaelFomenko commented on GitHub (Feb 4, 2024): > CUDA isn't deterministic unl You need to provide more Information, like Model, the Temperature rate, the Quantization type and so on. In my Opinion the Quantization is the Problem.
GiteaMirror commented 2026-04-12 11:10:45 -05:00
Author
Owner
Copy Link

@pltrdy commented on GitHub (Feb 4, 2024):

@MichaelFomenko the server uses model llama2 and runs on colab following this example 09a6f76f4c/examples/jupyter-notebook/ollama.ipynb

The call itself to generate uses defaults values outputs = [client.generate("llama2", "Why is the sky blue?")["response"] for o in range(3)]

<!-- gh-comment-id:1925875560 --> @pltrdy commented on GitHub (Feb 4, 2024): @MichaelFomenko the server uses model `llama2` and runs on colab following this example https://github.com/ollama/ollama/blob/09a6f76f4c30fb8a9708680c519d08feeb504197/examples/jupyter-notebook/ollama.ipynb The call itself to generate uses defaults values `outputs = [client.generate("llama2", "Why is the sky blue?")["response"] for o in range(3)]`
GiteaMirror commented 2026-04-12 11:10:45 -05:00
Author
Owner
Copy Link

@WeirdCarrotMonster commented on GitHub (Mar 10, 2024):

I've encountered same problem, although not immediately, but after some number of successful generations, and on multiple models - llama2 and llama2-uncensored.

My setup:
GPU: GeForce RTX 3060
Driver Version: 525.147.05
Ollama version: 0.1.28

The problem persists until ollama restart. I can't reliably reproduce it but i think i haven't encountered this behavior while running ollama on CPU (Ryzen 7 5700G).

I'm using n8n integration with default settings, so i'm not sure what temperature and quantization values are, but i can dig around if needed.

<!-- gh-comment-id:1987023847 --> @WeirdCarrotMonster commented on GitHub (Mar 10, 2024): I've encountered same problem, although not immediately, but after some number of successful generations, and on multiple models - llama2 and llama2-uncensored. My setup: GPU: GeForce RTX 3060 Driver Version: 525.147.05 Ollama version: 0.1.28 The problem persists until ollama restart. I can't reliably reproduce it but i think i haven't encountered this behavior while running ollama on CPU (Ryzen 7 5700G). I'm using n8n integration with default settings, so i'm not sure what temperature and quantization values are, but i can dig around if needed.
GiteaMirror commented 2026-04-12 11:10:46 -05:00
Author
Owner
Copy Link

@bmizerany commented on GitHub (Mar 11, 2024):

This seems to be specific to the model, not Ollama. I'm going to close this issue out, but please reopen and update if you believe it is Ollama specific.

<!-- gh-comment-id:1989652265 --> @bmizerany commented on GitHub (Mar 11, 2024): This seems to be specific to the model, not Ollama. I'm going to close this issue out, but please reopen and update if you believe it is Ollama specific.
GiteaMirror commented 2026-04-12 11:10:46 -05:00
Author
Owner
Copy Link

@piratos commented on GitHub (May 13, 2024):

I can't see how this is linked to the model itself if the first generation works but not the subsequent, which should not be related.

I used both moondream and llava-llama3. first generation is on point with prompt/picture, subsequent generation are not at all. (jibberish, unrelated chinese, etc)

unloading the model

curl http://localhost:11434/api/generate -d '{"model": "llava-llama3", "keep_alive": 0}'

or restarting ollama fixes the issue for the next generation but same behavior.

Running ollama on debian with rtx3090

<!-- gh-comment-id:2108947596 --> @piratos commented on GitHub (May 13, 2024): I can't see how this is linked to the model itself if the first generation works but not the subsequent, which should not be related. I used both `moondream` and `llava-llama3`. first generation is on point with prompt/picture, subsequent generation are not at all. (jibberish, unrelated chinese, etc) unloading the model ``` curl http://localhost:11434/api/generate -d '{"model": "llava-llama3", "keep_alive": 0}' ``` or restarting ollama fixes the issue for the next generation but same behavior. Running ollama on debian with rtx3090
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#1343
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?