github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 20:54:54 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #7166] Qwen 2.5 72B missing stop parameter #30308

New Issue
Open
opened 2026-04-22 09:52:43 -05:00 by GiteaMirror · 9 comments
GiteaMirror commented 2026-04-22 09:52:43 -05:00
Owner
Copy Link

Originally created by @bold84 on GitHub (Oct 10, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/7166

What is the issue?

Often doesn't stop generating...

telegram-cloud-photo-size-5-6071353675453939699-x

PARAMETER stop <|endoftext|>

seems to be missing in the model configuration. Adding it solved the problem.

OS

Ubuntu 22.04.5 LTS (but ollama runs in official docker container)

GPU

2 x RTX 4090

CPU

AMD Ryzen 9 7950X

Ollama version

0.3.12

Originally created by @bold84 on GitHub (Oct 10, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/7166 ### What is the issue? Often doesn't stop generating... ![telegram-cloud-photo-size-5-6071353675453939699-x](https://github.com/user-attachments/assets/0e5bb916-068c-44d8-88b8-797fa1e77b0e) PARAMETER stop <|endoftext|> seems to be missing in the model configuration. Adding it solved the problem. ### OS Ubuntu 22.04.5 LTS (but ollama runs in official docker container) ### GPU 2 x RTX 4090 ### CPU AMD Ryzen 9 7950X ### Ollama version 0.3.12
GiteaMirror added the bug label 2026-04-22 09:52:43 -05:00
GiteaMirror commented 2026-04-22 09:52:45 -05:00
Author
Owner
Copy Link

@Mushoz commented on GitHub (Oct 10, 2024):

Experiencing the same with the 32b model

<!-- gh-comment-id:2406137842 --> @Mushoz commented on GitHub (Oct 10, 2024): Experiencing the same with the 32b model
GiteaMirror commented 2026-04-22 09:52:45 -05:00
Author
Owner
Copy Link

@bold84 commented on GitHub (Oct 10, 2024):

Experiencing the same with the 32b model

Probably the same reason.

<!-- gh-comment-id:2406139294 --> @bold84 commented on GitHub (Oct 10, 2024): > Experiencing the same with the 32b model Probably the same reason.
GiteaMirror commented 2026-04-22 09:52:46 -05:00
Author
Owner
Copy Link

@juangon commented on GitHub (Oct 11, 2024):

Using Qwen2.5 32B without problems, Ollama 0.3.12

<!-- gh-comment-id:2406619197 --> @juangon commented on GitHub (Oct 11, 2024): Using Qwen2.5 32B without problems, Ollama 0.3.12
GiteaMirror commented 2026-04-22 09:52:46 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Oct 16, 2024):

I'm trying to duplicate this on macos and am not seeing it, although I am getting it to print the replacement character for some of its output. I've tried both the 32b and 72b models.

@bold84 and @Mushoz which version of ollama are you using?

<!-- gh-comment-id:2415509905 --> @pdevine commented on GitHub (Oct 16, 2024): I'm trying to duplicate this on macos and am not seeing it, although I am getting it to print the replacement character for some of its output. I've tried both the 32b and 72b models. @bold84 and @Mushoz which version of ollama are you using?
GiteaMirror commented 2026-04-22 09:52:47 -05:00
Author
Owner
Copy Link

@bold84 commented on GitHub (Oct 16, 2024):

I'm trying to duplicate this on macos and am not seeing it, although I am getting it to print the replacement character for some of its output. I've tried both the 32b and 72b models.

@bold84 and @Mushoz which version of ollama are you using?

I updated the information in the first comment.

It seems to me that the <|endoftext|> only appears with models that are quantized with less than 5 bit.

qwen2.5:72b-instruct-q4_K_M has the problem
qwen2.5:32b-instruct-q5_K_M does not have the problem

<!-- gh-comment-id:2415968147 --> @bold84 commented on GitHub (Oct 16, 2024): > I'm trying to duplicate this on macos and am not seeing it, although I am getting it to print the replacement character for some of its output. I've tried both the 32b and 72b models. > > @bold84 and @Mushoz which version of ollama are you using? I updated the information in the first comment. It seems to me that the <|endoftext|> only appears with models that are quantized with less than 5 bit. qwen2.5:72b-instruct-q4_K_M has the problem qwen2.5:32b-instruct-q5_K_M does not have the problem
GiteaMirror commented 2026-04-22 09:52:47 -05:00
Author
Owner
Copy Link

@difro commented on GitHub (Oct 16, 2024):

I've experienced same issue with qwen2.5:72b-instruct-q8_0

Adding PARAMETER stop <|endoftext|> and PARAMETER num_ctx 8192(as suggested by this issue https://github.com/ollama/ollama/issues/6915#issuecomment-2378325223) to the Modelfile seems to have solved it for me.

<!-- gh-comment-id:2416059783 --> @difro commented on GitHub (Oct 16, 2024): I've experienced same issue with qwen2.5:72b-instruct-q8_0 Adding `PARAMETER stop <|endoftext|>` and `PARAMETER num_ctx 8192`(as suggested by this issue https://github.com/ollama/ollama/issues/6915#issuecomment-2378325223) to the Modelfile seems to have solved it for me.
GiteaMirror commented 2026-04-22 09:52:48 -05:00
Author
Owner
Copy Link

@Mushoz commented on GitHub (Oct 16, 2024):

I was able to solve the issue by increasing the num_ctx parameter. Apparently when the context size is exceeded, the model starts spitting out stuff that looks like training data, which includes the <|endoftext|> tag. Increasing the context size so that no longer happens seems to solve the issue. The maximum you can increase the context length to is 32768, as Ollama does NOT support YARN to multiply that context size by 4 up to 128k.

<!-- gh-comment-id:2416374504 --> @Mushoz commented on GitHub (Oct 16, 2024): I was able to solve the issue by increasing the `num_ctx` parameter. Apparently when the context size is exceeded, the model starts spitting out stuff that looks like training data, which includes the <|endoftext|> tag. Increasing the context size so that no longer happens seems to solve the issue. The maximum you can increase the context length to is `32768`, as Ollama does NOT support YARN to multiply that context size by 4 up to 128k.
GiteaMirror commented 2026-04-22 09:52:49 -05:00
Author
Owner
Copy Link

@bold84 commented on GitHub (Oct 16, 2024):

I have num_ctx at 32768. It still happened.

Only PARAMETER stop <|endoftext|> solved the problem reliably for me.

<!-- gh-comment-id:2416379835 --> @bold84 commented on GitHub (Oct 16, 2024): I have num_ctx at 32768. It still happened. Only `PARAMETER stop <|endoftext|>` solved the problem reliably for me.
GiteaMirror commented 2026-04-22 09:52:50 -05:00
Author
Owner
Copy Link

@Mushoz commented on GitHub (Oct 21, 2024):

Ollama pre-release 0.4.0 is available here: https://github.com/ollama/ollama/releases/tag/v0.4.0-rc3

The thing that caught my eyes was the following statement: which includes improved vision model caching, model reliability, caching and stop token detection among other fixes

<!-- gh-comment-id:2427422196 --> @Mushoz commented on GitHub (Oct 21, 2024): Ollama pre-release 0.4.0 is available here: https://github.com/ollama/ollama/releases/tag/v0.4.0-rc3 The thing that caught my eyes was the following statement: which includes improved vision model caching, model reliability, caching and **stop token detection** among other fixes
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#30308
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?