github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #8548] ollama create --quantize q4_K_M not working #5516

New Issue
Closed
opened 2026-04-12 16:46:05 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-04-12 16:46:05 -05:00
Owner
Copy Link

Originally created by @hgKang02 on GitHub (Jan 23, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/8548

What is the issue?

Dear ollama authors,

I hope this issue finds you well. First of all, thank you for such a wonderful project. It helped me to play around with different agentic AI models and tasks.

However, the quantize command is not working for me when I try to quantize the llava-next model. I have created a Modelfile on the same folder where the safetensors are placed. So for instance, if the model's safetensors are placed at /user/agent-models/llava-next, inside this folder I have created the Modelfile with context
FROM .

Now when I run the ollama create --quantize q4_K_M mymodel I get

gathering model components
Error: no Modelfile or safetensors files found

Would this be because ollama does not support Llava architecture for quantize function? Or is there anything that I am doing wrong?

Thank you and it will be much appreciated if I can hear back about this problem.

Sincerely,

OS

Linux

GPU

Nvidia

CPU

No response

Ollama version

0.5.7

Originally created by @hgKang02 on GitHub (Jan 23, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/8548 ### What is the issue? Dear ollama authors, I hope this issue finds you well. First of all, thank you for such a wonderful project. It helped me to play around with different agentic AI models and tasks. However, the quantize command is not working for me when I try to quantize the llava-next model. I have created a Modelfile on the same folder where the safetensors are placed. So for instance, if the model's safetensors are placed at /user/agent-models/llava-next, inside this folder I have created the Modelfile with context `FROM .` Now when I run the `ollama create --quantize q4_K_M mymodel` I get **gathering model components Error: no Modelfile or safetensors files found** Would this be because ollama does not support Llava architecture for quantize function? Or is there anything that I am doing wrong? Thank you and it will be much appreciated if I can hear back about this problem. Sincerely, ### OS Linux ### GPU Nvidia ### CPU _No response_ ### Ollama version 0.5.7
GiteaMirror added the bug label 2026-04-12 16:46:05 -05:00
GiteaMirror commented 2026-04-12 16:46:06 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 23, 2025):

I don't know if ollama will quantize a llava model, but I don't think that's the problem you have here. The model create feature underwent some surgery in 0.5.5 and still has some rough edges. Try rolling back to 0.5.4. You can create a model in 0.5.4 and it will work in 0.5.5+.

<!-- gh-comment-id:2610113059 --> @rick-github commented on GitHub (Jan 23, 2025): I don't know if ollama will quantize a llava model, but I don't think that's the problem you have here. The model create feature underwent some surgery in 0.5.5 and still has some rough edges. Try rolling back to 0.5.4. You can create a model in 0.5.4 and it will work in 0.5.5+.
GiteaMirror commented 2026-04-12 16:46:07 -05:00
Author
Owner
Copy Link

@hgKang02 commented on GitHub (Jan 24, 2025):

It works when I use 0.5.4. Thank you!

<!-- gh-comment-id:2613084249 --> @hgKang02 commented on GitHub (Jan 24, 2025): It works when I use 0.5.4. Thank you!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#5516
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?