github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #8495] Why do I keep getting "@@@@" as responses? #67529

New Issue
Open
opened 2026-05-04 10:40:14 -05:00 by GiteaMirror · 28 comments
GiteaMirror commented 2026-05-04 10:40:14 -05:00
Owner
Copy Link

Originally created by @Jetbuzz on GitHub (Jan 20, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/8495

What is the issue?

I have attached the screenshot to what is happening. I have an Nvidia 980m 4gb. Running latest version of Windows 10 and ollama.

Image

OS

Windows

GPU

Nvidia

CPU

Intel

Ollama version

0.5.7

Originally created by @Jetbuzz on GitHub (Jan 20, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/8495 ### What is the issue? I have attached the screenshot to what is happening. I have an Nvidia 980m 4gb. Running latest version of Windows 10 and ollama. ![Image](https://github.com/user-attachments/assets/e670af15-7220-4a98-ab74-9cc398ffb52e) ### OS Windows ### GPU Nvidia ### CPU Intel ### Ollama version 0.5.7
GiteaMirror added the bug label 2026-05-04 10:40:14 -05:00
GiteaMirror commented 2026-05-04 10:40:16 -05:00
Author
Owner
Copy Link

@Jetbuzz commented on GitHub (Jan 20, 2025):

Image

I tried forcing ollama to use my graphics card but still facing the same issue

<!-- gh-comment-id:2601368504 --> @Jetbuzz commented on GitHub (Jan 20, 2025): ![Image](https://github.com/user-attachments/assets/399bb02f-2f39-4537-9f8e-481fc58c3c9f) I tried forcing ollama to use my graphics card but still facing the same issue
GiteaMirror commented 2026-05-04 10:40:18 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 20, 2025):

Server logs may aid in debugging.

<!-- gh-comment-id:2601846675 --> @rick-github commented on GitHub (Jan 20, 2025): [Server logs](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#how-to-troubleshoot-issues) may aid in debugging.
GiteaMirror commented 2026-05-04 10:40:19 -05:00
Author
Owner
Copy Link

@Jetbuzz commented on GitHub (Jan 20, 2025):

Server logs may aid in debugging.

Thank you for your response. Here's the server log

server.log

<!-- gh-comment-id:2602264502 --> @Jetbuzz commented on GitHub (Jan 20, 2025): > [Server logs](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#how-to-troubleshoot-issues) may aid in debugging. Thank you for your response. Here's the server log [server.log](https://github.com/user-attachments/files/18477164/server.log)
GiteaMirror commented 2026-05-04 10:40:24 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 20, 2025):

If you could add the full log that would help, there are lines earlier with information about device detection, environment, etc. It may also help if you add OLLAMA_DEBUG=1 to the server environment for additional debugging.

<!-- gh-comment-id:2602278548 --> @rick-github commented on GitHub (Jan 20, 2025): If you could add the full log that would help, there are lines earlier with information about device detection, environment, etc. It may also help if you add `OLLAMA_DEBUG=1` to the [server environment](https://github.com/ollama/ollama/blob/main/docs/faq.md#setting-environment-variables-on-windows) for additional debugging.
GiteaMirror commented 2026-05-04 10:40:27 -05:00
Author
Owner
Copy Link

@TungstenWolframite commented on GitHub (Jan 24, 2025):

I am getting the same error. It seems to be triggered when I have a long conversation with a large context size.
Interestingly, even if you start a new conversation by going "/bye" and rerunning the same model, the error persists.
The only way I have found to solve this issue is to stop the model entirely by using "ollama stop" and then rerunning the model.

<!-- gh-comment-id:2612738360 --> @TungstenWolframite commented on GitHub (Jan 24, 2025): I am getting the same error. It seems to be triggered when I have a long conversation with a large context size. Interestingly, even if you start a new conversation by going "/bye" and rerunning the same model, the error persists. The only way I have found to solve this issue is to stop the model entirely by using "ollama stop" and then rerunning the model.
GiteaMirror commented 2026-05-04 10:40:28 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 24, 2025):

Server logs may aid in debugging.

<!-- gh-comment-id:2612741459 --> @rick-github commented on GitHub (Jan 24, 2025): [Server logs](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#how-to-troubleshoot-issues) may aid in debugging.
GiteaMirror commented 2026-05-04 10:40:29 -05:00
Author
Owner
Copy Link

@TungstenWolframite commented on GitHub (Feb 3, 2025):

Here is my server log: server.log

The last entry had the repeated "@"

Thanks for your help!

<!-- gh-comment-id:2631211051 --> @TungstenWolframite commented on GitHub (Feb 3, 2025): Here is my server log: [server.log](https://github.com/user-attachments/files/18643815/server.log) The last entry had the repeated "@" Thanks for your help!
GiteaMirror commented 2026-05-04 10:40:29 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 3, 2025):

What model?

<!-- gh-comment-id:2631222249 --> @rick-github commented on GitHub (Feb 3, 2025): What model?
GiteaMirror commented 2026-05-04 10:40:30 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 3, 2025):

Found it, https://ollama.com/library/qwen2.5:72b-instruct-q4_K_S

<!-- gh-comment-id:2631229531 --> @rick-github commented on GitHub (Feb 3, 2025): Found it, https://ollama.com/library/qwen2.5:72b-instruct-q4_K_S
GiteaMirror commented 2026-05-04 10:40:31 -05:00
Author
Owner
Copy Link

@TungstenWolframite commented on GitHub (Feb 3, 2025):

This happens with multiple other models as well.
E.g., llama3.3:70b-instruct-q8_0, llama3.2:3b-instruct-q4_K_M, deepseek-r1:70b-llama-distill-q8_0, deepseek-r1:70b-llama-distill-q4_K_M, etc.

<!-- gh-comment-id:2631238386 --> @TungstenWolframite commented on GitHub (Feb 3, 2025): This happens with multiple other models as well. E.g., llama3.3:70b-instruct-q8_0, llama3.2:3b-instruct-q4_K_M, deepseek-r1:70b-llama-distill-q8_0, deepseek-r1:70b-llama-distill-q4_K_M, etc.
GiteaMirror commented 2026-05-04 10:40:32 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 3, 2025):

If it's affecting multiple models of different architectures and different sizes, it may be a hardware issue. #8235 had a similar problem with repeating '@'s, running a GPU VRAM tester revealed a faulty card.

<!-- gh-comment-id:2631272623 --> @rick-github commented on GitHub (Feb 3, 2025): If it's affecting multiple models of different architectures and different sizes, it may be a hardware issue. #8235 had a similar problem with repeating '@'s, running a [GPU VRAM tester]( https://www.programming4beginners.com/gpumemtest) revealed a faulty card.
GiteaMirror commented 2026-05-04 10:40:34 -05:00
Author
Owner
Copy Link

@TungstenWolframite commented on GitHub (Feb 3, 2025):

Hmm, the issue looks the same as #8235, but I just ran the GPU VRAM test and it showed "All tests OK." for both my GPUs.

<!-- gh-comment-id:2631301294 --> @TungstenWolframite commented on GitHub (Feb 3, 2025): Hmm, the issue looks the same as #8235, but I just ran the GPU VRAM test and it showed "All tests OK." for both my GPUs.
GiteaMirror commented 2026-05-04 10:40:35 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 3, 2025):

It's a limited tester, perhaps OCCT might reveal more.

<!-- gh-comment-id:2631364657 --> @rick-github commented on GitHub (Feb 3, 2025): It's a limited tester, perhaps [`OCCT`](https://www.ocbase.com/occt/personal`) might reveal more.
GiteaMirror commented 2026-05-04 10:40:36 -05:00
Author
Owner
Copy Link

@TungstenWolframite commented on GitHub (Feb 3, 2025):

Using ollama through CLI does not seem to trigger the issue.
It only occurs when using the server. E.g., Open WebUI, Python's ollama Client

<!-- gh-comment-id:2631365477 --> @TungstenWolframite commented on GitHub (Feb 3, 2025): Using ollama through CLI does not seem to trigger the issue. It only occurs when using the server. E.g., Open WebUI, Python's ollama Client
GiteaMirror commented 2026-05-04 10:40:37 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 3, 2025):

Using ollama through CLI does not seem to trigger the issue.

Whihch way do use the CLI:

C:\> olllama run qwen2.5:72b-instruct-q4_K_S Hello
Hello @@@

or

C:\> olllama run qwen2.5:72b-instruct-q4_K_S
>>> Hello
Hello @@@
<!-- gh-comment-id:2631385079 --> @rick-github commented on GitHub (Feb 3, 2025): > Using ollama through CLI does not seem to trigger the issue. Whihch way do use the CLI: ```console C:\> olllama run qwen2.5:72b-instruct-q4_K_S Hello Hello @@@ ``` or ```console C:\> olllama run qwen2.5:72b-instruct-q4_K_S >>> Hello Hello @@@ ```
GiteaMirror commented 2026-05-04 10:40:38 -05:00
Author
Owner
Copy Link

@TungstenWolframite commented on GitHub (Feb 3, 2025):

The latter

<!-- gh-comment-id:2631386900 --> @TungstenWolframite commented on GitHub (Feb 3, 2025): The latter
GiteaMirror commented 2026-05-04 10:40:40 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 3, 2025):

The latter uses /api/chat, the log shows the client used /api/generate. What was the client at that time?

Open-webui uses /api/chat so if you've seen this in open-webui then the endpoint isn't relevant but it's an interesting data point.

<!-- gh-comment-id:2631405981 --> @rick-github commented on GitHub (Feb 3, 2025): The latter uses `/api/chat`, the log shows the client used `/api/generate`. What was the client at that time? Open-webui uses `/api/chat` so if you've seen this in open-webui then the endpoint isn't relevant but it's an interesting data point.
GiteaMirror commented 2026-05-04 10:40:41 -05:00
Author
Owner
Copy Link

@TungstenWolframite commented on GitHub (Feb 3, 2025):

I am currently spamming CLI using the first method, and no errors can be reproduced.

For the log, I was using python client.generate

<!-- gh-comment-id:2631411336 --> @TungstenWolframite commented on GitHub (Feb 3, 2025): I am currently spamming CLI using the first method, and no errors can be reproduced. For the log, I was using python client.generate
GiteaMirror commented 2026-05-04 10:40:41 -05:00
Author
Owner
Copy Link

@TungstenWolframite commented on GitHub (Feb 3, 2025):

Here is an image of one of the conversations in Open WebUI that had the issues very quickly. This was using llama3.3:70b-instruct-q8_0:

Image

<!-- gh-comment-id:2631420292 --> @TungstenWolframite commented on GitHub (Feb 3, 2025): Here is an image of one of the conversations in Open WebUI that had the issues very quickly. This was using llama3.3:70b-instruct-q8_0: ![Image](https://github.com/user-attachments/assets/d7ba46d8-40f3-4944-8a77-74bf465c782c)
GiteaMirror commented 2026-05-04 10:40:43 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 19, 2025):

Some other are reporting the same problem.

#9190, #9225

No resolution as yet.

<!-- gh-comment-id:2669041127 --> @rick-github commented on GitHub (Feb 19, 2025): Some other are reporting the same problem. #9190, #9225 No resolution as yet.
GiteaMirror commented 2026-05-04 10:40:45 -05:00
Author
Owner
Copy Link

@TungstenWolframite commented on GitHub (Feb 19, 2025):

Thank you for the update.

I’m not sure if it’s helpful, but all my models are working with vLLM.

<!-- gh-comment-id:2669713952 --> @TungstenWolframite commented on GitHub (Feb 19, 2025): Thank you for the update. I’m not sure if it’s helpful, but all my models are working with vLLM.
GiteaMirror commented 2026-05-04 10:40:47 -05:00
Author
Owner
Copy Link

@TungstenWolframite commented on GitHub (Feb 27, 2025):

Good news! @rick-github
The issue resolved with the new NVIDIA graphics driver update.
Ollama is back in action for me!

<!-- gh-comment-id:2689011974 --> @TungstenWolframite commented on GitHub (Feb 27, 2025): Good news! @rick-github The issue resolved with the new NVIDIA graphics driver update. Ollama is back in action for me!
GiteaMirror commented 2026-05-04 10:40:49 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 27, 2025):

Great! What version of the driver did you upgrade to?

<!-- gh-comment-id:2689018459 --> @rick-github commented on GitHub (Feb 27, 2025): Great! What version of the driver did you upgrade to?
GiteaMirror commented 2026-05-04 10:40:50 -05:00
Author
Owner
Copy Link

@TungstenWolframite commented on GitHub (Feb 27, 2025):

This is what is stated in Nvidia App:
Driver Version: 572.60
Released: Thu Feb 27, 2025

<!-- gh-comment-id:2689023820 --> @TungstenWolframite commented on GitHub (Feb 27, 2025): This is what is stated in Nvidia App: Driver Version: 572.60 Released: Thu Feb 27, 2025
GiteaMirror commented 2026-05-04 10:40:51 -05:00
Author
Owner
Copy Link

@TungstenWolframite commented on GitHub (Feb 27, 2025):

My previous driver version was from 19/11/2024.

<!-- gh-comment-id:2689025399 --> @TungstenWolframite commented on GitHub (Feb 27, 2025): My previous driver version was from 19/11/2024.
GiteaMirror commented 2026-05-04 10:40:52 -05:00
Author
Owner
Copy Link

@Jetbuzz commented on GitHub (Feb 28, 2025):

Just want to update: I tried updating my drivers and it seems that I still
face the @@@@ problem :C
I have given up on running LLMs offline.

On Fri, Feb 28, 2025 at 1:48 AM Jordan @.***> wrote:

My previous driver version was from 19/11/2024.

—
Reply to this email directly, view it on GitHub
https://github.com/ollama/ollama/issues/8495#issuecomment-2689025399,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AMVIQKC2HVYKVRC4VHOGNX32R5XKVAVCNFSM6AAAAABVPPZ2QCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMOBZGAZDKMZZHE
.
You are receiving this because you authored the thread.Message ID:
@.***>
[image: TungstenWolframite]TungstenWolframite left a comment
(ollama/ollama#8495)
https://github.com/ollama/ollama/issues/8495#issuecomment-2689025399

My previous driver version was from 19/11/2024.

—
Reply to this email directly, view it on GitHub
https://github.com/ollama/ollama/issues/8495#issuecomment-2689025399,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AMVIQKC2HVYKVRC4VHOGNX32R5XKVAVCNFSM6AAAAABVPPZ2QCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMOBZGAZDKMZZHE
.
You are receiving this because you authored the thread.Message ID:
@.***>

<!-- gh-comment-id:2690282179 --> @Jetbuzz commented on GitHub (Feb 28, 2025): Just want to update: I tried updating my drivers and it seems that I still face the @@@@ problem :C I have given up on running LLMs offline. On Fri, Feb 28, 2025 at 1:48 AM Jordan ***@***.***> wrote: > My previous driver version was from 19/11/2024. > > — > Reply to this email directly, view it on GitHub > <https://github.com/ollama/ollama/issues/8495#issuecomment-2689025399>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AMVIQKC2HVYKVRC4VHOGNX32R5XKVAVCNFSM6AAAAABVPPZ2QCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMOBZGAZDKMZZHE> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> > [image: TungstenWolframite]*TungstenWolframite* left a comment > (ollama/ollama#8495) > <https://github.com/ollama/ollama/issues/8495#issuecomment-2689025399> > > My previous driver version was from 19/11/2024. > > — > Reply to this email directly, view it on GitHub > <https://github.com/ollama/ollama/issues/8495#issuecomment-2689025399>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AMVIQKC2HVYKVRC4VHOGNX32R5XKVAVCNFSM6AAAAABVPPZ2QCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMOBZGAZDKMZZHE> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >
GiteaMirror commented 2026-05-04 10:40:53 -05:00
Author
Owner
Copy Link

@TungstenWolframite commented on GitHub (Mar 3, 2025):

@Jetbuzz Do you have any overclocking or undervolting for your GPU?

<!-- gh-comment-id:2693947831 --> @TungstenWolframite commented on GitHub (Mar 3, 2025): @Jetbuzz Do you have any overclocking or undervolting for your GPU?
GiteaMirror commented 2026-05-04 10:40:54 -05:00
Author
Owner
Copy Link

@Jetbuzz commented on GitHub (Mar 4, 2025):

@Jetbuzz Do you have any overclocking or undervolting for your GPU?

Nope, unless my laptop (Alienware 15 R2, GTX 980M) is doing it. I haven't messed with the settings though.

<!-- gh-comment-id:2696345857 --> @Jetbuzz commented on GitHub (Mar 4, 2025): > [@Jetbuzz](https://github.com/Jetbuzz) Do you have any overclocking or undervolting for your GPU? Nope, unless my laptop (Alienware 15 R2, GTX 980M) is doing it. I haven't messed with the settings though.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#67529
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?