github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 13:19:40 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #6795] there are various models which is default provided by meta llama when downloaded i have tried but couldn't not find it #30045

New Issue
Closed
opened 2026-04-22 09:27:59 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-04-22 09:27:59 -05:00
Owner
Copy Link

Originally created by @olumolu on GitHub (Sep 13, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6795

8b-instruct-fp16
4aacac419454 • 16GB • Updated 3 days ago
8b-instruct-q2_K
44a139eeb344 • 3.2GB • Updated 3 days ago
8b-instruct-q3_K_S
16268e519444 • 3.7GB • Updated 3 days ago
8b-instruct-q3_K_M
4faa21fca5a2 • 4.0GB • Updated 3 days ago
8b-instruct-q3_K_L
04a2f1e44de7 • 4.3GB • Updated 3 days ago
8b-instruct-q4_0
42182419e950 • 4.7GB • Updated 3 days ago
8b-instruct-q4_1
e129e608a752 • 5.1GB • Updated 3 days ago
8b-instruct-q4_K_S
778e1e675704 • 4.7GB • Updated 3 days ago
8b-instruct-q4_K_M
46e0c10c039e • 4.9GB • Updated 3 days ago
8b-instruct-q5_0
26bc223a1709 • 5.6GB • Updated 3 days ago
8b-instruct-q5_1
8faaa53f9cda • 6.1GB • Updated 3 days ago
8b-instruct-q5_K_S
2d79e69bc236 • 5.6GB • Updated 3 days ago
8b-instruct-q5_K_M
27fe1b0ab52c • 5.7GB • Updated 3 days ago
8b-instruct-q6_K
81e7664fda9c • 6.6GB • Updated 3 days ago
8b-instruct-q8_0
b158ded76fa0 • 8.5GB • Updated 3 days ago
8b-text-fp16
722fd1ff1fda • 16GB • Updated 5 weeks ago
8b-text-q2_K
82bedef0ef47 • 3.2GB • Updated 5 weeks ago
8b-text-q3_K_S
92c2cffe1a17 • 3.7GB • Updated 5 weeks ago
8b-text-q3_K_M
abcf9215e3df • 4.0GB • Updated 5 weeks ago
8b-text-q3_K_L
4d9a56f79245 • 4.3GB • Updated 5 weeks ago
8b-text-q4_0
025059e83055 • 4.7GB • Updated 4 weeks ago
8b-text-q4_1
4b32d52187b4 • 5.1GB • Updated 5 weeks ago
8b-text-q4_K_S
d1a421604a57 • 4.7GB • Updated 5 weeks ago
8b-text-q4_K_M
6f98b5a6e4b7 • 4.9GB • Updated 5 weeks ago
8b-text-q5_0
ee0f9a2ffa00 • 5.6GB • Updated 5 weeks ago
8b-text-q5_1
ad68371dbd08 • 6.1GB • Updated 5 weeks ago
8b-text-q5_K_S
a7562b693302 • 5.6GB • Updated 5 weeks ago
8b-text-q5_K_M
e0b14a625560 • 5.7GB • Updated 5 weeks ago
8b-text-q6_K
40e61b3c96cc • 6.6GB • Updated 5 weeks ago
8b-text-q8_0
56fd90f1aa19 • 8.5GB • Updated 5 weeks ago
70b-instruct-fp16
80d34437631f • 141GB • Updated 3 days ago
70b-instruct-q2_K
3cbf499d6905 • 26GB • Updated 3 days ago

Originally created by @olumolu on GitHub (Sep 13, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6795 [8b-instruct-fp16](https://ollama.com/library/llama3.1:8b-instruct-fp16) 4aacac419454 • 16GB • Updated 3 days ago [8b-instruct-q2_K](https://ollama.com/library/llama3.1:8b-instruct-q2_K) 44a139eeb344 • 3.2GB • Updated 3 days ago [8b-instruct-q3_K_S](https://ollama.com/library/llama3.1:8b-instruct-q3_K_S) 16268e519444 • 3.7GB • Updated 3 days ago [8b-instruct-q3_K_M](https://ollama.com/library/llama3.1:8b-instruct-q3_K_M) 4faa21fca5a2 • 4.0GB • Updated 3 days ago [8b-instruct-q3_K_L](https://ollama.com/library/llama3.1:8b-instruct-q3_K_L) 04a2f1e44de7 • 4.3GB • Updated 3 days ago [8b-instruct-q4_0](https://ollama.com/library/llama3.1:8b-instruct-q4_0) 42182419e950 • 4.7GB • Updated 3 days ago [8b-instruct-q4_1](https://ollama.com/library/llama3.1:8b-instruct-q4_1) e129e608a752 • 5.1GB • Updated 3 days ago [8b-instruct-q4_K_S](https://ollama.com/library/llama3.1:8b-instruct-q4_K_S) 778e1e675704 • 4.7GB • Updated 3 days ago [8b-instruct-q4_K_M](https://ollama.com/library/llama3.1:8b-instruct-q4_K_M) 46e0c10c039e • 4.9GB • Updated 3 days ago [8b-instruct-q5_0](https://ollama.com/library/llama3.1:8b-instruct-q5_0) 26bc223a1709 • 5.6GB • Updated 3 days ago [8b-instruct-q5_1](https://ollama.com/library/llama3.1:8b-instruct-q5_1) 8faaa53f9cda • 6.1GB • Updated 3 days ago [8b-instruct-q5_K_S](https://ollama.com/library/llama3.1:8b-instruct-q5_K_S) 2d79e69bc236 • 5.6GB • Updated 3 days ago [8b-instruct-q5_K_M](https://ollama.com/library/llama3.1:8b-instruct-q5_K_M) 27fe1b0ab52c • 5.7GB • Updated 3 days ago [8b-instruct-q6_K](https://ollama.com/library/llama3.1:8b-instruct-q6_K) 81e7664fda9c • 6.6GB • Updated 3 days ago [8b-instruct-q8_0](https://ollama.com/library/llama3.1:8b-instruct-q8_0) b158ded76fa0 • 8.5GB • Updated 3 days ago [8b-text-fp16](https://ollama.com/library/llama3.1:8b-text-fp16) 722fd1ff1fda • 16GB • Updated 5 weeks ago [8b-text-q2_K](https://ollama.com/library/llama3.1:8b-text-q2_K) 82bedef0ef47 • 3.2GB • Updated 5 weeks ago [8b-text-q3_K_S](https://ollama.com/library/llama3.1:8b-text-q3_K_S) 92c2cffe1a17 • 3.7GB • Updated 5 weeks ago [8b-text-q3_K_M](https://ollama.com/library/llama3.1:8b-text-q3_K_M) abcf9215e3df • 4.0GB • Updated 5 weeks ago [8b-text-q3_K_L](https://ollama.com/library/llama3.1:8b-text-q3_K_L) 4d9a56f79245 • 4.3GB • Updated 5 weeks ago [8b-text-q4_0](https://ollama.com/library/llama3.1:8b-text-q4_0) 025059e83055 • 4.7GB • Updated 4 weeks ago [8b-text-q4_1](https://ollama.com/library/llama3.1:8b-text-q4_1) 4b32d52187b4 • 5.1GB • Updated 5 weeks ago [8b-text-q4_K_S](https://ollama.com/library/llama3.1:8b-text-q4_K_S) d1a421604a57 • 4.7GB • Updated 5 weeks ago [8b-text-q4_K_M](https://ollama.com/library/llama3.1:8b-text-q4_K_M) 6f98b5a6e4b7 • 4.9GB • Updated 5 weeks ago [8b-text-q5_0](https://ollama.com/library/llama3.1:8b-text-q5_0) ee0f9a2ffa00 • 5.6GB • Updated 5 weeks ago [8b-text-q5_1](https://ollama.com/library/llama3.1:8b-text-q5_1) ad68371dbd08 • 6.1GB • Updated 5 weeks ago [8b-text-q5_K_S](https://ollama.com/library/llama3.1:8b-text-q5_K_S) a7562b693302 • 5.6GB • Updated 5 weeks ago [8b-text-q5_K_M](https://ollama.com/library/llama3.1:8b-text-q5_K_M) e0b14a625560 • 5.7GB • Updated 5 weeks ago [8b-text-q6_K](https://ollama.com/library/llama3.1:8b-text-q6_K) 40e61b3c96cc • 6.6GB • Updated 5 weeks ago [8b-text-q8_0](https://ollama.com/library/llama3.1:8b-text-q8_0) 56fd90f1aa19 • 8.5GB • Updated 5 weeks ago [70b-instruct-fp16](https://ollama.com/library/llama3.1:70b-instruct-fp16) 80d34437631f • 141GB • Updated 3 days ago [70b-instruct-q2_K](https://ollama.com/library/llama3.1:70b-instruct-q2_K) 3cbf499d6905 • 26GB • Updated 3 days ago
GiteaMirror added the question label 2026-04-22 09:27:59 -05:00
GiteaMirror commented 2026-04-22 09:28:01 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Sep 13, 2024):

These are all quantized versions of the base model provided by Meta. The base model is not available through the ollama library, but you can fetch it from huggingface. The fp16 versions in the ollama library are the closest in fidelity to the base model.

<!-- gh-comment-id:2350719784 --> @rick-github commented on GitHub (Sep 13, 2024): These are all quantized versions of the base model provided by Meta. The base model is not available through the ollama library, but you can fetch it from [huggingface](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct). The fp16 versions in the ollama library are the closest in fidelity to the base model.
GiteaMirror commented 2026-04-22 09:28:02 -05:00
Author
Owner
Copy Link

@olumolu commented on GitHub (Sep 14, 2024):

Are you sure sure i am looking for a the base model. That meta provide if i download from llama.meta.com
Why i put this request because i recently trying to meta.ai and duck.ai which supports llama3.1 70b model but i was using this models on my server with olllama i did not select any quantised version as i thought i will stay with the default but not i understand that answers are better in slightly if i use duck.ai or meta.ai instead of my server this is not that good as compared to duck or meta .ai and i start to find why and that is why i am asking...
If you can confirm.

<!-- gh-comment-id:2350833656 --> @olumolu commented on GitHub (Sep 14, 2024): Are you sure sure i am looking for a the base model. That meta provide if i download from llama.meta.com Why i put this request because i recently trying to meta.ai and duck.ai which supports llama3.1 70b model but i was using this models on my server with olllama i did not select any quantised version as i thought i will stay with the default but not i understand that answers are better in slightly if i use duck.ai or meta.ai instead of my server this is not that good as compared to duck or meta .ai and i start to find why and that is why i am asking... If you can confirm.
GiteaMirror commented 2026-04-22 09:28:02 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Sep 14, 2024):

If you downloaded llama3.1:70b from the ollama library, that is the same as the quantized model llama3.1:70b-instruct-q4_0. This is a 4 bit quantized model compared to the 32 bit base model. All models in the ollama library are quantized and because of that, have less fidelity compared to the safetensor models being run by duck.ai and meta.ai. That's because running an unquantized model requires lots of GPU which most users don't have access to. If you want to run the full model, you can download the unquantized model from huggingface and rent a server with 192G VRAM for $4/hr.

<!-- gh-comment-id:2350843450 --> @rick-github commented on GitHub (Sep 14, 2024): If you downloaded [llama3.1:70b](https://ollama.com/library/llama3.1:70b) from the ollama library, that is the same as the quantized model [llama3.1:70b-instruct-q4_0](https://ollama.com/library/llama3.1:70b-instruct-q4_0). This is a 4 bit quantized model compared to the 32 bit base model. All models in the ollama library are quantized and because of that, have less fidelity compared to the safetensor models being run by duck.ai and meta.ai. That's because running an unquantized model requires lots of GPU which most users don't have access to. If you want to run the full model, you can download the unquantized model from [huggingface](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct) and [rent a server](https://www.runpod.io/pricing) with 192G VRAM for $4/hr.
GiteaMirror commented 2026-04-22 09:28:03 -05:00
Author
Owner
Copy Link

@olumolu commented on GitHub (Sep 14, 2024):

If you downloaded llama3.1:70b from the ollama library, that is the same as the quantized model llama3.1:70b-instruct-q4_0. This is a 4 bit quantized model compared to the 32 bit base model. All models in the ollama library are quantized and because of that, have less fidelity compared to the safetensor models being run by duck.ai and meta.ai. That's because running an unquantized model requires lots of GPU which most users don't have access to. If you want to run the full model, you can download the unquantized model from huggingface and rent a server with 192G VRAM for $4/hr.

I get a better server for free.
If you can provide make llama3.1 (default) one available in ollama so it can be easy to use them. If someone have the power. If default is fp16 then just put a (default) there if it is fp32 in that case provide that model and write default thanks.
I dont have hardware for running 405b but what i find that the size of it is 700gb and fp16 of 405b is 800gb so close to that i assume that fp16 maybe the default for that but provide answer if know.

<!-- gh-comment-id:2350849635 --> @olumolu commented on GitHub (Sep 14, 2024): > If you downloaded [llama3.1:70b](https://ollama.com/library/llama3.1:70b) from the ollama library, that is the same as the quantized model [llama3.1:70b-instruct-q4_0](https://ollama.com/library/llama3.1:70b-instruct-q4_0). This is a 4 bit quantized model compared to the 32 bit base model. All models in the ollama library are quantized and because of that, have less fidelity compared to the safetensor models being run by duck.ai and meta.ai. That's because running an unquantized model requires lots of GPU which most users don't have access to. If you want to run the full model, you can download the unquantized model from [huggingface](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct) and [rent a server](https://www.runpod.io/pricing) with 192G VRAM for $4/hr. I get a better server for free. If you can provide make llama3.1 (default) one available in ollama so it can be easy to use them. If someone have the power. If default is fp16 then just put a (default) there if it is fp32 in that case provide that model and write default thanks. I dont have hardware for running 405b but what i find that the size of it is 700gb and fp16 of 405b is 800gb so close to that i assume that fp16 maybe the default for that but provide answer if know.
GiteaMirror commented 2026-04-22 09:28:04 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Sep 14, 2024):

@olumolu as @rick-github mentioned, if you want the base non-quantized version you can ollama run llama:3.1:70b-instruct-fp16. We provide the 4bit quantized version as the default because most people don't have good enough hardware to support the non-quantized versions.

I'll close out the issue, but feel free to keep commenting.

<!-- gh-comment-id:2351061380 --> @pdevine commented on GitHub (Sep 14, 2024): @olumolu as @rick-github mentioned, if you want the base non-quantized version you can `ollama run llama:3.1:70b-instruct-fp16`. We provide the 4bit quantized version as the default because most people don't have good enough hardware to support the non-quantized versions. I'll close out the issue, but feel free to keep commenting.
GiteaMirror commented 2026-04-22 09:28:04 -05:00
Author
Owner
Copy Link

@olumolu commented on GitHub (Sep 14, 2024):

@olumolu as @rick-github mentioned, if you want the base non-quantized version you can ollama run llama:3.1:70b-instruct-fp16. We provide the 4bit quantized version as the default because most people don't have good enough hardware to support the non-quantized versions.

I'll close out the issue, but feel free to keep commenting.

I just want to know what is the default that meta ai use for llama3.1
Fp16 or fp32

<!-- gh-comment-id:2351088483 --> @olumolu commented on GitHub (Sep 14, 2024): > @olumolu as @rick-github mentioned, if you want the base non-quantized version you can `ollama run llama:3.1:70b-instruct-fp16`. We provide the 4bit quantized version as the default because most people don't have good enough hardware to support the non-quantized versions. > > I'll close out the issue, but feel free to keep commenting. I just want to know what is the default that meta ai use for llama3.1 Fp16 or fp32
GiteaMirror commented 2026-04-22 09:28:05 -05:00
Author
Owner
Copy Link

@buingocminhhuong commented on GitHub (Feb 7, 2025):

@olumolu as @rick-github mentioned, if you want the base non-quantized version you can ollama run llama:3.1:70b-instruct-fp16. We provide the 4bit quantized version as the default because most people don't have good enough hardware to support the non-quantized versions.
I'll close out the issue, but feel free to keep commenting.

I just want to know what is the default that meta ai use for llama3.1 Fp16 or fp32

fp16 means full precision (16-bit floating point), which results in the best accuracy but requires significant VRAM (GPU memory).

<!-- gh-comment-id:2642587833 --> @buingocminhhuong commented on GitHub (Feb 7, 2025): > > [@olumolu](https://github.com/olumolu) as [@rick-github](https://github.com/rick-github) mentioned, if you want the base non-quantized version you can `ollama run llama:3.1:70b-instruct-fp16`. We provide the 4bit quantized version as the default because most people don't have good enough hardware to support the non-quantized versions. > > I'll close out the issue, but feel free to keep commenting. > > I just want to know what is the default that meta ai use for llama3.1 Fp16 or fp32 fp16 means full precision (16-bit floating point), which results in the best accuracy but requires significant VRAM (GPU memory).
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label question
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#30045
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?