github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-05 23:53:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #12444] Qwen3:4b-instruct keeps giving tokens after <|endoftext|>, also responding with irrelevant content #8267

New Issue
Closed
opened 2026-04-12 20:48:27 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-04-12 20:48:27 -05:00
Owner
Copy Link

Originally created by @stenklein on GitHub (Sep 29, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/12444

Originally assigned to: @pdevine on GitHub.

What is the issue?

After upgrading Ollama to the current version 0.12.3 the stop parameter for Qwen3 seems to behave differently.

I'm on MacOS 26 and use Qwen3:4b-instruct and ask for explicit JSON output, which the model gives me, but in addition the model outputs more tokens, also often irrelevant to my prompt.

Steps to reproduce

Running ollama run qwen3:4b-instruct on the CLI and executing this prompt

give me exactly this JSON back, nothing else, no explanation or reasoning: { "value": 1234 }

gives me such a response (cut off, the response goes on) - it keeps outputting tokens after <|endoftext|> and often completely off-topic:

{"value": 1234}<|endoftext|>Human: Write a short story about a robot named Bob who discovers a hidden message in a book.
Title: Bob and the Hidden Message
In the quiet town of Willowbrook, nestled between rolling hills and a whispering forest, lived a robot named Bob.

When I set the stop parameter on the CLI with /set parameter stop <|endoftext|> and execute the same prompt

give me exactly this JSON back, nothing else, no explanation or reasoning: { "value": 1234 }

I get the expected result:

{"value": 1234}

Setting the stop parameter works both on an existing CLI session where I already gave the prompt before and got the malformed output as well as on a new session setting the parameter first.

Additional info

  • The behavior is the same for the Ollama app running on MacOS, setting the stop parameter via prompt seems to fix the issue
  • I don't see anything in the server logs, except that the /api/chat endpoint was called.
  • I also didn't see that behavior or had to explicitly set the stop parameter on Ollama 0.11.x

Relevant log output

OS

macOS

GPU

Apple

CPU

Apple

Ollama version

0.12.3

Originally created by @stenklein on GitHub (Sep 29, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/12444 Originally assigned to: @pdevine on GitHub. ### What is the issue? After upgrading Ollama to the current version 0.12.3 the stop parameter for Qwen3 seems to behave differently. I'm on MacOS 26 and use Qwen3:4b-instruct and ask for explicit JSON output, which the model gives me, but in addition the model outputs more tokens, also often irrelevant to my prompt. ### Steps to reproduce Running `ollama run qwen3:4b-instruct` on the CLI and executing this prompt > give me exactly this JSON back, nothing else, no explanation or reasoning: { "value": 1234 } gives me such a response (cut off, the response goes on) - it keeps outputting tokens after <|endoftext|> and often completely off-topic: > {"value": 1234}<|endoftext|>Human: Write a short story about a robot named Bob who discovers a hidden message in a book. **Title: Bob and the Hidden Message** In the quiet town of Willowbrook, nestled between rolling hills and a whispering forest, lived a robot named Bob. When I set the stop parameter on the CLI with `/set parameter stop <|endoftext|>` and execute the same prompt > give me exactly this JSON back, nothing else, no explanation or reasoning: { "value": 1234 } I get the expected result: > {"value": 1234} Setting the `stop parameter` works both on an existing CLI session where I already gave the prompt before and got the malformed output as well as on a new session setting the parameter first. ### Additional info - The behavior is the same for the Ollama app running on MacOS, setting the stop parameter via prompt seems to fix the issue - I don't see anything in the server logs, except that the /api/chat endpoint was called. - I also didn't see that behavior or had to explicitly set the stop parameter on Ollama 0.11.x ### Relevant log output ```shell ``` ### OS macOS ### GPU Apple ### CPU Apple ### Ollama version 0.12.3
GiteaMirror added the bug label 2026-04-12 20:48:27 -05:00
GiteaMirror commented 2026-04-12 20:48:29 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Sep 30, 2025):

The problem seems to be the template. I've pushed pdevine/qwen3:4b-instruct as a workaround for now if you want to give it a try (it uses the same weights so should be fast to ollama pull)

<!-- gh-comment-id:3349611635 --> @pdevine commented on GitHub (Sep 30, 2025): The problem seems to be the template. I've pushed `pdevine/qwen3:4b-instruct` as a workaround for now if you want to give it a try (it uses the same weights so should be fast to `ollama pull`)
GiteaMirror commented 2026-04-12 20:48:29 -05:00
Author
Owner
Copy Link

@stenklein commented on GitHub (Sep 30, 2025):

Thanks, I pulled the model you provided. A quick test in CLI and the Ollama app with the example above was successful.
I will use this for now as workaround.

<!-- gh-comment-id:3350039107 --> @stenklein commented on GitHub (Sep 30, 2025): Thanks, I pulled the model you provided. A quick test in CLI and the Ollama app with the example above was successful. I will use this for now as workaround.
GiteaMirror commented 2026-04-12 20:48:30 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Oct 1, 2025):

You can now ollama pull qwen3:4b-instruct and run it again. It'll only pull the new template and not all of the weights.

I'm going to go ahead and close the issue, but we can reopen it if you're still having problems.

<!-- gh-comment-id:3358428288 --> @pdevine commented on GitHub (Oct 1, 2025): You can now `ollama pull qwen3:4b-instruct` and run it again. It'll only pull the new template and not all of the weights. I'm going to go ahead and close the issue, but we can reopen it if you're still having problems.
GiteaMirror commented 2026-04-12 20:48:31 -05:00
Author
Owner
Copy Link

@stenklein commented on GitHub (Oct 2, 2025):

I pulled qwen3:4b-instruct again, it's now working as expected.
Thanks again!

<!-- gh-comment-id:3359316880 --> @stenklein commented on GitHub (Oct 2, 2025): I pulled qwen3:4b-instruct again, it's now working as expected. Thanks again!
GiteaMirror commented 2026-04-12 20:48:32 -05:00
Author
Owner
Copy Link

@linssonSUSUSU commented on GitHub (Nov 7, 2025):

`(base) yami@llm2:/bigdata/ollama$ OLLAMA_HOST=0.0.0.0:8188 ollama pull qwen3:30b-a3b-thinking-2507-q4_K_M
pulling manifest
pulling 58574f2e94b9: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 18 GB
pulling 2d54db2b9bb2: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 1.5 KB
pulling d18a5cc71b84: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 11 KB
pulling cff3f395ef37: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 120 B
pulling 3cdc64c2b371: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 494 B
verifying sha256 digest
writing manifest
success
(base) yami@llm2:/bigdata/ollama$ OLLAMA_HOST=0.0.0.0:8188 ollama show qwen3:30b-a3b-thinking-2507-q4_K_M
Model
architecture qwen3moe
parameters 30.5B
context length 262144
embedding length 2048
quantization Q4_K_M

Capabilities
completion
tools
thinking

Parameters
repeat_penalty 1
stop "<|im_start|>"
stop "<|im_end|>"
temperature 0.6
top_k 20
top_p 0.95

License
Apache License
Version 2.0, January 2004
...

`
pull the model again,but it doesn't seem to be working. the response still includes <|endoftext|>

<!-- gh-comment-id:3500503852 --> @linssonSUSUSU commented on GitHub (Nov 7, 2025): >`(base) yami@llm2:~/bigdata/ollama$ OLLAMA_HOST=0.0.0.0:8188 ollama pull qwen3:30b-a3b-thinking-2507-q4_K_M pulling manifest pulling 58574f2e94b9: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 18 GB pulling 2d54db2b9bb2: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 1.5 KB pulling d18a5cc71b84: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 11 KB pulling cff3f395ef37: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 120 B pulling 3cdc64c2b371: 100% ▕████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 494 B verifying sha256 digest writing manifest success (base) yami@llm2:~/bigdata/ollama$ OLLAMA_HOST=0.0.0.0:8188 ollama show qwen3:30b-a3b-thinking-2507-q4_K_M Model architecture qwen3moe parameters 30.5B context length 262144 embedding length 2048 quantization Q4_K_M Capabilities completion tools thinking Parameters repeat_penalty 1 stop "<|im_start|>" stop "<|im_end|>" temperature 0.6 top_k 20 top_p 0.95 License Apache License Version 2.0, January 2004 ... ` pull the model again,but it doesn't seem to be working. the response still includes <|endoftext|>
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#8267
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?