github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #9530] [Website] [Bug] Incorrect Sampling Parameters for QwQ 32B #68273

New Issue
Open
opened 2026-05-04 13:04:29 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-05-04 13:04:29 -05:00
Owner
Copy Link

Originally created by @vYLQs6 on GitHub (Mar 6, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/9530

The ollama repo for QwQ 32B didn't set any Sampling Parameters, which could cause significant performance degradation:

Image

From QwQ HF page:

Usage Guidelines
To achieve optimal performance, we recommend the following settings:
....
Sampling Parameters:
Use Temperature=0.6 and TopP=0.95 instead of Greedy decoding to avoid endless repetitions.
Use TopK between 20 and 40 to filter out rare token occurrences while maintaining the diversity of the generated output.

official generation config from qwq hf page:

{
  "bos_token_id": 151643,
  "do_sample": true,
  "eos_token_id": [
    151645,
    151643
  ],
  "pad_token_id": 151643,
  "repetition_penalty": 1.0,
  "temperature": 0.6,
  "top_k": 40,
  "top_p": 0.95,
  "transformers_version": "4.45.2"
}

I would recommend fix this, since it's really easy to fix

Plus the system prompt also needs an update, here is the new official system prompt from qwq-32b demo by qwen team:

You are a helpful and harmless assistant.

def format_history(history):
    messages = [{
        "role": "system",
        "content": "You are a helpful and harmless assistant.",
    }]
    for item in history:
        if item["role"] == "user":
            messages.append({"role": "user", "content": item["content"]})
        elif item["role"] == "assistant":
            messages.append({"role": "assistant", "content": item["content"]})
    return messages

https://huggingface.co/spaces/Qwen/QwQ-32B-Demo/blob/main/app.py

Originally created by @vYLQs6 on GitHub (Mar 6, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/9530 The ollama repo for QwQ 32B didn't set any Sampling Parameters, which could cause significant performance degradation: ![Image](https://github.com/user-attachments/assets/6bd6f966-94d8-447a-aab0-f64ba8e3235d) From QwQ HF page: > Usage Guidelines > To achieve optimal performance, we recommend the following settings: > .... > Sampling Parameters: > Use Temperature=0.6 and TopP=0.95 instead of Greedy decoding to avoid endless repetitions. > Use TopK between 20 and 40 to filter out rare token occurrences while maintaining the diversity of the generated output. official generation config from qwq hf page: ``` { "bos_token_id": 151643, "do_sample": true, "eos_token_id": [ 151645, 151643 ], "pad_token_id": 151643, "repetition_penalty": 1.0, "temperature": 0.6, "top_k": 40, "top_p": 0.95, "transformers_version": "4.45.2" } ``` I would recommend fix this, since it's really easy to fix --- Plus the system prompt also needs an update, here is the new official system prompt from qwq-32b demo by qwen team: ### You are a helpful and harmless assistant. ``` def format_history(history): messages = [{ "role": "system", "content": "You are a helpful and harmless assistant.", }] for item in history: if item["role"] == "user": messages.append({"role": "user", "content": item["content"]}) elif item["role"] == "assistant": messages.append({"role": "assistant", "content": item["content"]}) return messages ``` https://huggingface.co/spaces/Qwen/QwQ-32B-Demo/blob/main/app.py
GiteaMirror commented 2026-05-04 13:04:30 -05:00
Author
Owner
Copy Link

@vYLQs6 commented on GitHub (Mar 6, 2025):

Source:

https://huggingface.co/Qwen/QwQ-32B/blob/main/generation_config.json

https://huggingface.co/Qwen/QwQ-32B/blob/main/README.md

<!-- gh-comment-id:2702695945 --> @vYLQs6 commented on GitHub (Mar 6, 2025): Source: https://huggingface.co/Qwen/QwQ-32B/blob/main/generation_config.json https://huggingface.co/Qwen/QwQ-32B/blob/main/README.md
GiteaMirror commented 2026-05-04 13:04:31 -05:00
Author
Owner
Copy Link

@yorktownting commented on GitHub (Mar 6, 2025):

My QwQ-32B-Q5_K_M will ask and answer for herself, delivering an endless output. Is that what's causing this?

<!-- gh-comment-id:2702718721 --> @yorktownting commented on GitHub (Mar 6, 2025): My QwQ-32B-Q5_K_M will ask and answer for herself, delivering an endless output. Is that what's causing this?
GiteaMirror commented 2026-05-04 13:04:32 -05:00
Author
Owner
Copy Link

@WR-CREATOR commented on GitHub (Mar 11, 2025):

I tested qwq-fp16 deployed using both ollama and other methods for the same difficult task, and found that the output results of ollama deployment were far inferior to those of other deployment methods. Is this the reason？

<!-- gh-comment-id:2712259227 --> @WR-CREATOR commented on GitHub (Mar 11, 2025): I tested qwq-fp16 deployed using both ollama and other methods for the same difficult task, and found that the output results of ollama deployment were far inferior to those of other deployment methods. Is this the reason？
GiteaMirror commented 2026-05-04 13:04:33 -05:00
Author
Owner
Copy Link

@yorktownting commented on GitHub (Mar 11, 2025):

I tested qwq-fp16 deployed using both ollama and other methods for the same difficult task, and found that the output results of ollama deployment were far inferior to those of other deployment methods. Is this the reason？

Did you try this? They have bug-fixed param in their repo
https://docs.unsloth.ai/basics/tutorial-how-to-run-qwq-32b-effectively

<!-- gh-comment-id:2712265536 --> @yorktownting commented on GitHub (Mar 11, 2025): > I tested qwq-fp16 deployed using both ollama and other methods for the same difficult task, and found that the output results of ollama deployment were far inferior to those of other deployment methods. Is this the reason？ Did you try this? They have bug-fixed param in their repo https://docs.unsloth.ai/basics/tutorial-how-to-run-qwq-32b-effectively
GiteaMirror commented 2026-05-04 13:04:33 -05:00
Author
Owner
Copy Link

@WR-CREATOR commented on GitHub (Mar 11, 2025):

I tested qwq-fp16 deployed using both ollama and other methods for the same difficult task, and found that the output results of ollama deployment were far inferior to those of other deployment methods. Is this the reason？

Did you try this? They have bug-fixed param in their repo https://docs.unsloth.ai/basics/tutorial-how-to-run-qwq-32b-effectively

No, what I deployed was just that the generated results were relatively poor, not without endless generations.

<!-- gh-comment-id:2713312090 --> @WR-CREATOR commented on GitHub (Mar 11, 2025): > > I tested qwq-fp16 deployed using both ollama and other methods for the same difficult task, and found that the output results of ollama deployment were far inferior to those of other deployment methods. Is this the reason？ > > Did you try this? They have bug-fixed param in their repo https://docs.unsloth.ai/basics/tutorial-how-to-run-qwq-32b-effectively No, what I deployed was just that the generated results were relatively poor, not without endless generations.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#68273
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?