github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-17 05:36:16 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #9523] support qwq 32B #31970

New Issue
Closed
opened 2026-04-22 12:49:01 -05:00 by GiteaMirror · 14 comments
GiteaMirror commented 2026-04-22 12:49:01 -05:00
Owner
Copy Link

Originally created by @olumolu on GitHub (Mar 5, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/9523

https://huggingface.co/Qwen/QwQ-32B
Performance is as per with deepseek r1

Image

Originally created by @olumolu on GitHub (Mar 5, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/9523 https://huggingface.co/Qwen/QwQ-32B Performance is as per with deepseek r1 ![Image](https://github.com/user-attachments/assets/5b5ca282-fbb5-402c-9212-8d48d30d1f41)
GiteaMirror added the model label 2026-04-22 12:49:01 -05:00
GiteaMirror commented 2026-04-22 12:49:04 -05:00
Author
Owner
Copy Link

@coder543 commented on GitHub (Mar 5, 2025):

This appears to already be complete: https://ollama.com/library/qwq

Although it may not be following the recommended temperature, top_p, and top_k settings.

<!-- gh-comment-id:2702100919 --> @coder543 commented on GitHub (Mar 5, 2025): This appears to already be complete: https://ollama.com/library/qwq Although it may not be following the recommended temperature, top_p, and top_k settings.
GiteaMirror commented 2026-04-22 12:49:05 -05:00
Author
Owner
Copy Link

@Lr-2002 commented on GitHub (Mar 6, 2025):

is there a minimum requirement of qwq ?
I'm now using M3 with 10 gpu cores, 16G Mem

<!-- gh-comment-id:2702721034 --> @Lr-2002 commented on GitHub (Mar 6, 2025): is there a minimum requirement of qwq ? I'm now using M3 with 10 gpu cores, 16G Mem
GiteaMirror commented 2026-04-22 12:49:06 -05:00
Author
Owner
Copy Link

@coder543 commented on GitHub (Mar 6, 2025):

@Lr-2002 The main requirement is having enough RAM. The default size of the model is already quantized to around 4-bit precision, and it is still 20GB in size. I don’t think it’s really going to work on a laptop with 16GB of RAM.

(With some effort, you could theoretically run it directly from the SSD (including using swap memory to use the SSD), but that would be extremely slow. You could also potentially find a very heavily quantized version somewhere that would fit into 16GB of RAM, but there’s no telling if that model would be worth anything at that point. I don’t recommend either of these things.)

<!-- gh-comment-id:2702744941 --> @coder543 commented on GitHub (Mar 6, 2025): @Lr-2002 The main requirement is having enough RAM. The default size of the model is already quantized to around 4-bit precision, and it is still 20GB in size. I don’t think it’s really going to work on a laptop with 16GB of RAM. (With some effort, you could theoretically run it directly from the SSD (including using swap memory to use the SSD), but that would be extremely slow. You could also potentially find a _very_ heavily quantized version somewhere that would fit into 16GB of RAM, but there’s no telling if that model would be worth anything at that point. I don’t recommend either of these things.)
GiteaMirror commented 2026-04-22 12:49:06 -05:00
Author
Owner
Copy Link

@olumolu commented on GitHub (Mar 6, 2025):

You can run this but with swap memory.

<!-- gh-comment-id:2702780307 --> @olumolu commented on GitHub (Mar 6, 2025): You can run this but with swap memory.
GiteaMirror commented 2026-04-22 12:49:07 -05:00
Author
Owner
Copy Link

@Lr-2002 commented on GitHub (Mar 6, 2025):

yeah! thanks you all ,maybe i'll deploy it on another server

<!-- gh-comment-id:2702890986 --> @Lr-2002 commented on GitHub (Mar 6, 2025): yeah! thanks you all ,maybe i'll deploy it on another server
GiteaMirror commented 2026-04-22 12:49:07 -05:00
Author
Owner
Copy Link

@wohenbuguai commented on GitHub (Mar 6, 2025):

work on M1_PRO 32G ,Runs relatively slow, generation speed is close to deepseek-R1:32b

<!-- gh-comment-id:2703206620 --> @wohenbuguai commented on GitHub (Mar 6, 2025): work on M1_PRO 32G ,Runs relatively slow, generation speed is close to deepseek-R1:32b
GiteaMirror commented 2026-04-22 12:49:09 -05:00
Author
Owner
Copy Link

@akshaal commented on GitHub (Mar 6, 2025):

The HF page has the following in the Usage Guidelines:

Enforce Thoughtful Output: Ensure the model starts with "<think>\n" to prevent generating empty thinking content ...

And the think token is also present in the template GGUF template (tokenizer.chat_template) published on the HF: https://huggingface.co/Qwen/QwQ-32B-GGUF?show_file_info=qwq-32b-q4_k_m-00001-of-00005.gguf

But the template on the ollama.com looks very different and lacks the token: https://ollama.com/library/qwq:32b-q4_K_M/blobs/41190096a061

<!-- gh-comment-id:2703880818 --> @akshaal commented on GitHub (Mar 6, 2025): The HF page has the following in the `Usage Guidelines`: > Enforce Thoughtful Output: Ensure the model starts with "&lt;think&gt;\n" to prevent generating empty thinking content ... And the `think` token is also present in the template GGUF template (`tokenizer.chat_template`) published on the HF: https://huggingface.co/Qwen/QwQ-32B-GGUF?show_file_info=qwq-32b-q4_k_m-00001-of-00005.gguf But the template on the ollama.com looks very different and lacks the token: https://ollama.com/library/qwq:32b-q4_K_M/blobs/41190096a061
GiteaMirror commented 2026-04-22 12:49:09 -05:00
Author
Owner
Copy Link

@AlbertoSinigaglia commented on GitHub (Mar 6, 2025):

The HF page has the following in the Usage Guidelines:

Enforce Thoughtful Output: Ensure the model starts with "\n" to prevent generating empty thinking content ...

And the think token is also present in the template GGUF template (tokenizer.chat_template) published on the HF: https://huggingface.co/Qwen/QwQ-32B-GGUF?show_file_info=qwq-32b-q4_k_m-00001-of-00005.gguf

But the template on the ollama.com looks very different and lacks the token: https://ollama.com/library/qwq:32b-q4_K_M/blobs/41190096a061

On this point, most of the times the model never gets out of the thinking phase, even with all the sampling parameters set to the one suggested by their blogpost

<!-- gh-comment-id:2705190486 --> @AlbertoSinigaglia commented on GitHub (Mar 6, 2025): > The HF page has the following in the `Usage Guidelines`: > > > Enforce Thoughtful Output: Ensure the model starts with "<think>\n" to prevent generating empty thinking content ... > > And the `think` token is also present in the template GGUF template (`tokenizer.chat_template`) published on the HF: https://huggingface.co/Qwen/QwQ-32B-GGUF?show_file_info=qwq-32b-q4_k_m-00001-of-00005.gguf > > But the template on the ollama.com looks very different and lacks the token: https://ollama.com/library/qwq:32b-q4_K_M/blobs/41190096a061 On this point, most of the times the model never gets out of the thinking phase, even with all the sampling parameters set to the one suggested by their blogpost
GiteaMirror commented 2026-04-22 12:49:10 -05:00
Author
Owner
Copy Link

@gjd898 commented on GitHub (Mar 7, 2025):

Why do I use ollama run qwq to run correctly but without any output?

<!-- gh-comment-id:2705416657 --> @gjd898 commented on GitHub (Mar 7, 2025): Why do I use ollama run qwq to run correctly but without any output?
GiteaMirror commented 2026-04-22 12:49:11 -05:00
Author
Owner
Copy Link

@olumolu commented on GitHub (Mar 7, 2025):

Why do I use ollama run qwq to run correctly but without any output?

Can you explain what you want

<!-- gh-comment-id:2705545320 --> @olumolu commented on GitHub (Mar 7, 2025): > Why do I use ollama run qwq to run correctly but without any output? Can you explain what you want
GiteaMirror commented 2026-04-22 12:49:11 -05:00
Author
Owner
Copy Link

@gjd898 commented on GitHub (Mar 7, 2025):

Why do I use ollama run qwq to run correctly but without any output?

Can you explain what you want

For example, after I entered the command line with ollama run qwq, any questions I entered will return empty lines and no results.

<!-- gh-comment-id:2705606467 --> @gjd898 commented on GitHub (Mar 7, 2025): > > Why do I use ollama run qwq to run correctly but without any output? > > Can you explain what you want For example, after I entered the command line with ollama run qwq, any questions I entered will return empty lines and no results.
GiteaMirror commented 2026-04-22 12:49:12 -05:00
Author
Owner
Copy Link

@dempsey-wen commented on GitHub (Mar 9, 2025):

is there a minimum requirement of qwq ? I'm now using M3 with 10 gpu cores, 16G Mem

You can try the 3-bit quantized version of QWQ-32b on Modelscope: https://modelscope.cn/models/Qwen/QwQ-32B-GGUF/files

<!-- gh-comment-id:2708897428 --> @dempsey-wen commented on GitHub (Mar 9, 2025): > is there a minimum requirement of qwq ? I'm now using M3 with 10 gpu cores, 16G Mem You can try the 3-bit quantized version of QWQ-32b on Modelscope: https://modelscope.cn/models/Qwen/QwQ-32B-GGUF/files
GiteaMirror commented 2026-04-22 12:49:12 -05:00
Author
Owner
Copy Link

@cyclinbox commented on GitHub (Mar 11, 2025):

You can run this but with swap memory.

Thank you, but I still wonder how to run qwq-32b with swap memory? My desktop (win10) has 20GB (generally ~10GB is available) memory and 16GB pagefile, but when I ran qwq-32b I still got a error message "Error: model requires more system memory (22.3 GiB) than is available (9.5 GiB)".

<!-- gh-comment-id:2713134956 --> @cyclinbox commented on GitHub (Mar 11, 2025): > You can run this but with swap memory. Thank you, but I still wonder how to run qwq-32b with swap memory? My desktop (win10) has 20GB (generally ~10GB is available) memory and 16GB pagefile, but when I ran qwq-32b I still got a error message "Error: model requires more system memory (22.3 GiB) than is available (9.5 GiB)".
GiteaMirror commented 2026-04-22 12:49:13 -05:00
Author
Owner
Copy Link

@olumolu commented on GitHub (Mar 11, 2025):

You can run this but with swap memory.

Thank you, but I still wonder how to run qwq-32b with swap memory? My desktop (win10) has 20GB (generally ~10GB is available) memory and 16GB pagefile, but when I ran qwq-32b I still got a error message "Error: model requires more system memory (22.3 GiB) than is available (9.5 GiB)".

Install fedora ubuntu or any linux distro.
https://fedoraproject.org/
https://universal-blue.org/
So on...
At the time of installation choose zram swap with 20gb (make it a note that ram is way too fast than zram.)
Now install ollama and run qwq.
If you want to run qwq in windows you need 64 gb ram

<!-- gh-comment-id:2713460228 --> @olumolu commented on GitHub (Mar 11, 2025): > > You can run this but with swap memory. > > Thank you, but I still wonder how to run qwq-32b with swap memory? My desktop (win10) has 20GB (generally ~10GB is available) memory and 16GB pagefile, but when I ran qwq-32b I still got a error message "Error: model requires more system memory (22.3 GiB) than is available (9.5 GiB)". Install fedora ubuntu or any linux distro. https://fedoraproject.org/ https://universal-blue.org/ So on... At the time of installation choose zram swap with 20gb (make it a note that ram is way too fast than zram.) Now install ollama and run qwq. If you want to run qwq in windows you need 64 gb ram
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label model
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#31970
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?