github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #9343] BF16 gguf file model import regression #31863

New Issue
Closed
opened 2026-04-22 12:37:46 -05:00 by GiteaMirror · 4 comments
GiteaMirror commented 2026-04-22 12:37:46 -05:00
Owner
Copy Link

Originally created by @rjmalagon on GitHub (Feb 25, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/9343

What is the issue?

Current Ollama lost BF16 gguf file model import, v0.5.7 can import this kind of models just fine.

This problem is not present running already imported BF16 models, neither pulling BF16 models from repos.

Relevant log output

gathering model components 
copying file sha256:4c9bc8d88421a7b250927f251a40f6e79ff23f00fb4021cfef82be25c2675259 100% 
parsing GGUF 
Error: invalid file magic

OS

Docker

GPU

AMD

CPU

AMD

Ollama version

~0.5.8-rc

Originally created by @rjmalagon on GitHub (Feb 25, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/9343 ### What is the issue? Current Ollama lost BF16 gguf file model import, v0.5.7 can import this kind of models just fine. This problem is not present running already imported BF16 models, neither pulling BF16 models from repos. ### Relevant log output ```shell gathering model components copying file sha256:4c9bc8d88421a7b250927f251a40f6e79ff23f00fb4021cfef82be25c2675259 100% parsing GGUF Error: invalid file magic ``` ### OS Docker ### GPU AMD ### CPU AMD ### Ollama version ~0.5.8-rc
GiteaMirror added the bug label 2026-04-22 12:37:46 -05:00
GiteaMirror commented 2026-04-22 12:37:48 -05:00
Author
Owner
Copy Link

@rjmalagon commented on GitHub (Feb 25, 2025):

On Ollama v0.5.7

gathering model components 
copying file sha256:4c9bc8d88421a7b250927f251a40f6e79ff23f00fb4021cfef82be25c2675259 100% 
parsing GGUF 
using existing layer sha256:4c9bc8d88421a7b250927f251a40f6e79ff23f00fb4021cfef82be25c2675259 
creating new layer sha256:eb4402837c7829a690fa845de4d7f3fd842c2adee476d5341da8a46ea9255175 
writing manifest 
success
<!-- gh-comment-id:2682367829 --> @rjmalagon commented on GitHub (Feb 25, 2025): On Ollama v0.5.7 ``` gathering model components copying file sha256:4c9bc8d88421a7b250927f251a40f6e79ff23f00fb4021cfef82be25c2675259 100% parsing GGUF using existing layer sha256:4c9bc8d88421a7b250927f251a40f6e79ff23f00fb4021cfef82be25c2675259 creating new layer sha256:eb4402837c7829a690fa845de4d7f3fd842c2adee476d5341da8a46ea9255175 writing manifest success ```
GiteaMirror commented 2026-04-22 12:37:48 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 25, 2025):

Which model are you importing?

<!-- gh-comment-id:2682490497 --> @rick-github commented on GitHub (Feb 25, 2025): Which model are you importing?
GiteaMirror commented 2026-04-22 12:37:49 -05:00
Author
Owner
Copy Link

@rjmalagon commented on GitHub (Feb 25, 2025):

Which model are you importing?

I bake my own gguf files from HuggingFace repos, with the llama.cpp python script (hf-to-gguf).

convert_hf_to_gguf.py  --outtype bf16 --outfile model-file.gguf --model-name model-name ./

Produces mixed F32/BF16 models, and is better resource wise than pure F32 when importing BF16 models, and better precision than mixed F32/F16 models.

I upload some of my tests on https://ollama.com/rjmalagon

Example https://ollama.com/rjmalagon/lamarckvergence rjmalagon/lamarckvergence:14b-bf16

Quality over speed implementation is my own personal research.

<!-- gh-comment-id:2682951187 --> @rjmalagon commented on GitHub (Feb 25, 2025): > Which model are you importing? I bake my own gguf files from HuggingFace repos, with the llama.cpp python script (hf-to-gguf). ``` convert_hf_to_gguf.py --outtype bf16 --outfile model-file.gguf --model-name model-name ./ ``` Produces mixed F32/BF16 models, and is better resource wise than pure F32 when importing BF16 models, and better precision than mixed F32/F16 models. I upload some of my tests on https://ollama.com/rjmalagon Example https://ollama.com/rjmalagon/lamarckvergence rjmalagon/lamarckvergence:14b-bf16 Quality over speed implementation is my own personal research.
GiteaMirror commented 2026-04-22 12:37:49 -05:00
Author
Owner
Copy Link

@rjmalagon commented on GitHub (Feb 25, 2025):

I confirm on my builds that https://github.com/ollama/ollama/pull/9344 restores the BF16 support.
Thanks, @mxyng

<!-- gh-comment-id:2683478852 --> @rjmalagon commented on GitHub (Feb 25, 2025): I confirm on my builds that https://github.com/ollama/ollama/pull/9344 restores the BF16 support. Thanks, @mxyng
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#31863
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?