github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #8436] kindly make f32 tensor type available in ollama #51935

New Issue
Closed
opened 2026-04-28 21:17:23 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-04-28 21:17:23 -05:00
Owner
Copy Link

Originally created by @olumolu on GitHub (Jan 15, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/8436

What is the issue?

There are many models available in hugging face with f32 b32 but in ollama the highest avail is f16 if this is possible having higher tensor could result in better performance and results as newer hardware can actually perform good.

OS

Linux

GPU

AMD, Nvidia, Intel

CPU

Intel, AMD, Apple

Ollama version

No response

Originally created by @olumolu on GitHub (Jan 15, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/8436 ### What is the issue? There are many models available in hugging face with f32 b32 but in ollama the highest avail is f16 if this is possible having higher tensor could result in better performance and results as newer hardware can actually perform good. ### OS Linux ### GPU AMD, Nvidia, Intel ### CPU Intel, AMD, Apple ### Ollama version _No response_
GiteaMirror added the bug label 2026-04-28 21:17:23 -05:00
GiteaMirror commented 2026-04-28 21:17:27 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 15, 2025):

Do you have a particular model in mind? Most of the base models I've looked at in HF are BF16. F32 is supported in ollama,
just convert the safetensors to f32 with llama.cpp and import to ollama. The quality difference is apparently quite minimal, you just end up with a bigger, slower model.

$ ollama run --verbose qwen2.5:0.5b-f32 'why is the sky blue?'
The sky is blue because of a combination of factors including sunlight, atmospheric scattering, and the red pigment that makes
it colorless. The blue part of the sky, also known as the "blue" or "green", is formed by the combination of light reflected off of
the Earth from clouds, oceans, and other water bodies. This light creates a reflective surface that reflects wavelengths in the
visible spectrum (visible light) and creates a reddish hue. 

The red pigment, while not visible to the naked eye, also reflects sunlight from stars and planets to some extent. When light
passes through the red pigment, it creates an intense red glow that can be seen by the naked eye but is less noticeable to the
eye of other humans or animals.

total duration:       6.141259489s
load duration:        38.831258ms
prompt eval count:    35 token(s)
prompt eval duration: 68ms
prompt eval rate:     514.71 tokens/s
eval count:           150 token(s)
eval duration:        6.032s
eval rate:            24.87 tokens/s

$ ollama run --verbose qwen2.5:0.5b-f16 'why is the sky blue?'
The sky is blue because it's a mixture of blue and red colors. The sun is a blue light that hits the Earth at high angles and emits
a blue color. Other colors of light, such as green or yellow, do not affect the color of the sky.

If you're experiencing a sudden change in color or feel like your skin feels dark, it might be due to other factors such as exposure
to strong sunlight, lack of air circulation, or a malfunctioning light source.

total duration:       2.148245243s
load duration:        22.371903ms
prompt eval count:    35 token(s)
prompt eval duration: 21ms
prompt eval rate:     1666.67 tokens/s
eval count:           98 token(s)
eval duration:        2.103s
eval rate:            46.60 tokens/s

$ ollama run --verbose qwen2.5:0.5b-instruct-q4_K_M 'why is the sky blue?'
The phenomenon of "blue" in the sky that we often observe is due to the presence of various gases and substances in the
Earth's atmosphere. These include nitrogen (N2), oxygen (O2), argon (Ar), carbon dioxide (CO2), water vapor, and other
trace gases. While these gases can all contribute to blue coloration when they are present in their natural form, they are
not always the primary factor responsible for the sky being blue.

In the Earth's atmosphere, certain concentrations of greenhouse gases like carbon dioxide (CO2) have a particularly strong
impact on producing blue light. Carbon dioxide is a key component that absorbs sunlight and gives us our usual yellowish-white
coloration due to  its high reflectivity. However, not all atmospheric gases contribute equally to this effect.

Some gases in the atmosphere play a more significant role in creating the typical blue coloration of the sky:
1. Water vapor: As mentioned earlier, water vapor is one of the primary contributors to the blue color in the Earth's atmosphere.
2. Oxygen (O2): The presence of oxygen affects the chemical properties of different atmospheric gases, but it plays a minor
role compared to nitrogen (N2) and other gases that contribute more directly to blue light.

In summary, while not all gases affect the sky primarily by changing its color, certain key components like water vapor are
responsible for most of the blue hue we perceive in our atmosphere. The specific contribution of these gases might vary slightly 
depending on their concentration levels or atmospheric conditions.

total duration:       3.344951562s
load duration:        14.236258ms
prompt eval count:    35 token(s)
prompt eval duration: 12ms
prompt eval rate:     2916.67 tokens/s
eval count:           306 token(s)
eval duration:        3.317s
eval rate:            92.25 tokens/s

NAME                ID              SIZE      PROCESSOR    UNTIL   
qwen2.5:0.5b        a8b0c5157701    741 MB    100% CPU     Forever    
qwen2.5:0.5b-f16    54b602a73f75    1.4 GB    100% CPU     Forever    
qwen2.5:0.5b-f32    2fdb3b91e61d    2.4 GB    100% CPU     Forever
<!-- gh-comment-id:2591955311 --> @rick-github commented on GitHub (Jan 15, 2025): Do you have a particular model in mind? Most of the base models I've looked at in HF are BF16. F32 is supported in ollama, just convert the safetensors to f32 with llama.cpp and import to ollama. The quality difference is apparently quite minimal, you just end up with a bigger, slower model. ```console $ ollama run --verbose qwen2.5:0.5b-f32 'why is the sky blue?' The sky is blue because of a combination of factors including sunlight, atmospheric scattering, and the red pigment that makes it colorless. The blue part of the sky, also known as the "blue" or "green", is formed by the combination of light reflected off of the Earth from clouds, oceans, and other water bodies. This light creates a reflective surface that reflects wavelengths in the visible spectrum (visible light) and creates a reddish hue. The red pigment, while not visible to the naked eye, also reflects sunlight from stars and planets to some extent. When light passes through the red pigment, it creates an intense red glow that can be seen by the naked eye but is less noticeable to the eye of other humans or animals. total duration: 6.141259489s load duration: 38.831258ms prompt eval count: 35 token(s) prompt eval duration: 68ms prompt eval rate: 514.71 tokens/s eval count: 150 token(s) eval duration: 6.032s eval rate: 24.87 tokens/s ``` ```console $ ollama run --verbose qwen2.5:0.5b-f16 'why is the sky blue?' The sky is blue because it's a mixture of blue and red colors. The sun is a blue light that hits the Earth at high angles and emits a blue color. Other colors of light, such as green or yellow, do not affect the color of the sky. If you're experiencing a sudden change in color or feel like your skin feels dark, it might be due to other factors such as exposure to strong sunlight, lack of air circulation, or a malfunctioning light source. total duration: 2.148245243s load duration: 22.371903ms prompt eval count: 35 token(s) prompt eval duration: 21ms prompt eval rate: 1666.67 tokens/s eval count: 98 token(s) eval duration: 2.103s eval rate: 46.60 tokens/s ``` ```console $ ollama run --verbose qwen2.5:0.5b-instruct-q4_K_M 'why is the sky blue?' The phenomenon of "blue" in the sky that we often observe is due to the presence of various gases and substances in the Earth's atmosphere. These include nitrogen (N2), oxygen (O2), argon (Ar), carbon dioxide (CO2), water vapor, and other trace gases. While these gases can all contribute to blue coloration when they are present in their natural form, they are not always the primary factor responsible for the sky being blue. In the Earth's atmosphere, certain concentrations of greenhouse gases like carbon dioxide (CO2) have a particularly strong impact on producing blue light. Carbon dioxide is a key component that absorbs sunlight and gives us our usual yellowish-white coloration due to its high reflectivity. However, not all atmospheric gases contribute equally to this effect. Some gases in the atmosphere play a more significant role in creating the typical blue coloration of the sky: 1. Water vapor: As mentioned earlier, water vapor is one of the primary contributors to the blue color in the Earth's atmosphere. 2. Oxygen (O2): The presence of oxygen affects the chemical properties of different atmospheric gases, but it plays a minor role compared to nitrogen (N2) and other gases that contribute more directly to blue light. In summary, while not all gases affect the sky primarily by changing its color, certain key components like water vapor are responsible for most of the blue hue we perceive in our atmosphere. The specific contribution of these gases might vary slightly depending on their concentration levels or atmospheric conditions. total duration: 3.344951562s load duration: 14.236258ms prompt eval count: 35 token(s) prompt eval duration: 12ms prompt eval rate: 2916.67 tokens/s eval count: 306 token(s) eval duration: 3.317s eval rate: 92.25 tokens/s ``` ```console NAME ID SIZE PROCESSOR UNTIL qwen2.5:0.5b a8b0c5157701 741 MB 100% CPU Forever qwen2.5:0.5b-f16 54b602a73f75 1.4 GB 100% CPU Forever qwen2.5:0.5b-f32 2fdb3b91e61d 2.4 GB 100% CPU Forever ```
GiteaMirror commented 2026-04-28 21:17:28 -05:00
Author
Owner
Copy Link

@olumolu commented on GitHub (Jan 15, 2025):

Not available from ollama website. Mainly smaller models are more important to have 32 as most of us could only able to run smaller models. With f32.

<!-- gh-comment-id:2592151336 --> @olumolu commented on GitHub (Jan 15, 2025): Not available from ollama website. Mainly smaller models are more important to have 32 as most of us could only able to run smaller models. With f32.
GiteaMirror commented 2026-04-28 21:17:28 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 15, 2025):

Do you have a particular model in mind?

<!-- gh-comment-id:2592651036 --> @rick-github commented on GitHub (Jan 15, 2025): Do you have a particular model in mind?
GiteaMirror commented 2026-04-28 21:17:29 -05:00
Author
Owner
Copy Link

@olumolu commented on GitHub (Jan 15, 2025):

https://huggingface.co/deepseek-ai/DeepSeek-V3
https://huggingface.co/allenai/OLMo-2-1124-13B

<!-- gh-comment-id:2592685951 --> @olumolu commented on GitHub (Jan 15, 2025): https://huggingface.co/deepseek-ai/DeepSeek-V3 https://huggingface.co/allenai/OLMo-2-1124-13B
GiteaMirror commented 2026-04-28 21:17:30 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 15, 2025):

Neither of these models are small. Until they show up in the library, you can create your own version as described above.

<!-- gh-comment-id:2592717843 --> @rick-github commented on GitHub (Jan 15, 2025): Neither of these models are small. Until they show up in the library, you can create your own version as described above.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#51935
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?