github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-22 05:31:43 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #9267] Allow Option to Use Original (Unquantized) Models in Model Registry #52552

New Issue
Closed
opened 2026-04-28 23:39:34 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-04-28 23:39:34 -05:00
Owner
Copy Link

Originally created by @kushteppalwar on GitHub (Feb 21, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/9267

What is the issue?

When viewing models in the Ollama model registry, their sizes are significantly smaller than their original open-source versions. This indicates that, by default, all models are quantized using Q4_K_M, which results in a performance reduction compared to running the original versions.

We have observed a noticeable degradation in performance, particularly in LLaMA 3.2 text and vision models. The quantization affects output quality and model capabilities, making Ollama-hosted models less effective than their counterparts outside Ollama.

Feature Request:
Would it be possible to introduce support for adding original (unquantized) models to the model registry? This would allow users to choose whether to use a quantized version for efficiency or the full model for maximum performance.

Expected Behavior:

Users can select between original (unquantized) models and quantized models in the registry.
The performance of LLaMA 3.2 and other models should be on par with their open-source versions when using the unquantized option.
Additional Context:
We primarily tested LLaMA 3.2 text and vision models and noticed the performance drop due to default quantization. Allowing users to opt for the original models would enhance flexibility and usability.

Originally created by @kushteppalwar on GitHub (Feb 21, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/9267 ### What is the issue? When viewing models in the Ollama model registry, their sizes are significantly smaller than their original open-source versions. This indicates that, by default, all models are quantized using Q4_K_M, which results in a performance reduction compared to running the original versions. We have observed a noticeable degradation in performance, particularly in LLaMA 3.2 text and vision models. The quantization affects output quality and model capabilities, making Ollama-hosted models less effective than their counterparts outside Ollama. Feature Request: Would it be possible to introduce support for adding original (unquantized) models to the model registry? This would allow users to choose whether to use a quantized version for efficiency or the full model for maximum performance. Expected Behavior: Users can select between original (unquantized) models and quantized models in the registry. The performance of LLaMA 3.2 and other models should be on par with their open-source versions when using the unquantized option. Additional Context: We primarily tested LLaMA 3.2 text and vision models and noticed the performance drop due to default quantization. Allowing users to opt for the original models would enhance flexibility and usability.
GiteaMirror added the bug label 2026-04-28 23:39:34 -05:00
GiteaMirror commented 2026-04-28 23:39:48 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 21, 2025):

This indicates that, by default, all models are quantized using Q4_K_M

Have you tried a non-default quantization, eg fp16? The current inference backend only supports GGUF files, so some transformation is required, but fp16 should be close to the original quality.

<!-- gh-comment-id:2673856346 --> @rick-github commented on GitHub (Feb 21, 2025): > This indicates that, by default, all models are quantized using Q4_K_M Have you tried a non-default quantization, eg fp16? The current inference backend only supports GGUF files, so some transformation is required, but fp16 should be close to the original quality.
GiteaMirror commented 2026-04-28 23:39:54 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Feb 21, 2025):

Hi @kushteppalwar for now fp16 models are available (e.g. ollama run llama3.2:3b-instruct-fp16)

We are working on supporting the original bf16 weights in ollama and ollama.com - stay tuned!

<!-- gh-comment-id:2675148944 --> @jmorganca commented on GitHub (Feb 21, 2025): Hi @kushteppalwar for now fp16 models are available (e.g. `ollama run llama3.2:3b-instruct-fp16`) We are working on supporting the original bf16 weights in ollama and ollama.com - stay tuned!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#52552
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?