github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-08 08:59:13 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #8740] Model Support: FP8 Full Precision for Deepseek R1 671B #5670

New Issue
Closed
opened 2026-04-12 16:58:15 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-04-12 16:58:15 -05:00
Owner
Copy Link

Originally created by @ghostplant on GitHub (Jan 31, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/8740

Hi, I'm wondering if ollama supports FP8 Full Precision without quantization for Deepseek R1 671B?

F8 is the original precision provided by DeepSeek, and I don't want to have anything else quantized since it would introduce some other PPL and reproducing issues.

For now, I found the model by ollama run deepseek-r1:671b is after FP4 quat, which is not the full precision version I prefer.

Originally created by @ghostplant on GitHub (Jan 31, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/8740 Hi, I'm wondering if ollama supports FP8 Full Precision without quantization for Deepseek R1 671B? F8 is the original precision provided by DeepSeek, and I don't want to have anything else quantized since it would introduce some other PPL and reproducing issues. For now, I found the model by `ollama run deepseek-r1:671b` is after FP4 quat, which is not the full precision version I prefer.
GiteaMirror added the model label 2026-04-12 16:58:15 -05:00
GiteaMirror commented 2026-04-12 16:58:16 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 31, 2025):

llama.cpp, the current ollama backend, doesn't support FP8. There is a draft PR to add it. deepseek-r1-fp16 has FP8 cast to FP16 by way of BF16 and is closest to the original model.

<!-- gh-comment-id:2628576080 --> @rick-github commented on GitHub (Jan 31, 2025): llama.cpp, the current ollama backend, doesn't support FP8. There is a [draft PR](https://github.com/ggerganov/llama.cpp/pull/10055) to add it. [deepseek-r1-fp16](https://ollama.com/library/deepseek-r1:671b-fp16) has FP8 cast to FP16 by way of BF16 and is closest to the original model.
GiteaMirror commented 2026-04-12 16:58:16 -05:00
Author
Owner
Copy Link

@ghostplant commented on GitHub (Feb 1, 2025):

@rick-github What about this one 671b-q8_0? Is it supported only after the draft PR is issued?

<!-- gh-comment-id:2628877203 --> @ghostplant commented on GitHub (Feb 1, 2025): @rick-github What about this one [671b-q8_0](https://ollama.com/library/deepseek-r1:671b-q8_0)? Is it supported only after the draft PR is issued?
GiteaMirror commented 2026-04-12 16:58:17 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 1, 2025):

No, 671b-q8_0 will work now, but it is quantized, FP8 -> Q8. It is very light quantization so probably not noticeable, depends on your requirements.

<!-- gh-comment-id:2628908416 --> @rick-github commented on GitHub (Feb 1, 2025): No, 671b-q8_0 will work now, but it is quantized, FP8 -> Q8. It is very light quantization so probably not noticeable, depends on your requirements.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label model
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#5670
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?