[GH-ISSUE #5889] Please provide Q_2 for Llama 3.1 405B #65712

New Issue

GiteaMirror · 2026-05-03T22:19:29-05:00

GiteaMirror commented

2026-05-03 22:19:29 -05:00

Originally created by @gileneusz on GitHub (Jul 23, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/5889

This quantisation is missing rn...

Originally created by @gileneusz on GitHub (Jul 23, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/5889 This quantisation is missing rn...

GiteaMirror added the model label 2026-05-03 22:19:29 -05:00

GiteaMirror closed this issue

2026-05-03 22:19:30 -05:00

GiteaMirror commented

2026-05-03 22:19:32 -05:00

@JamesStallings commented on GitHub (Jul 23, 2024):

I think you can use the ollama command to generate a quantized version of the model to your specification. See the help, it looks something like:

❯ ollama create -h
Create a model from a Modelfile

Usage:
ollama create MODEL [flags]

Flags:
-f, --file string Name of the Modelfile (default "Modelfile")
-h, --help help for create
-q, --quantize string Quantize model to this level (e.g. q4_0)

@JamesStallings commented on GitHub (Jul 23, 2024): I think you can use the ollama command to generate a quantized version of the model to your specification. See the help, it looks something like: ❯ ollama create -h Create a model from a Modelfile Usage: ollama create MODEL [flags] Flags: -f, --file string Name of the Modelfile (default "Modelfile") -h, --help help for create -q, --quantize string Quantize model to this level (e.g. q4_0)

GiteaMirror commented

2026-05-03 22:19:32 -05:00

@chigkim commented on GitHub (Jul 24, 2024):

It's no big deal for small model, but for big model like 405b, it will take ages to download full fp16 and quantize.

It would be amazing if Ollama library has other quants for llama 3.1 405b.

@chigkim commented on GitHub (Jul 24, 2024): It's no big deal for small model, but for big model like 405b, it will take ages to download full fp16 and quantize. It would be amazing if Ollama library has other quants for llama 3.1 405b.

GiteaMirror commented

2026-05-03 22:19:34 -05:00

@gileneusz commented on GitHub (Jul 24, 2024):

We currently don't have access to the full fp16 version on Ollama; the default configuration for the 405B model is Q_4. Maybe it's possible to "downquantize" it to Q_2, or alternatively, source the full fp16 version from another source. Both options involve considerable effort, and I will need to determine whether the 405B in Q_2 offers a significant advantage over the 70B in FP16. 😅

@gileneusz commented on GitHub (Jul 24, 2024): We currently don't have access to the full fp16 version on Ollama; the default configuration for the 405B model is Q_4. Maybe it's possible to "downquantize" it to Q_2, or alternatively, source the full fp16 version from another source. Both options involve considerable effort, and I will need to determine whether the 405B in Q_2 offers a significant advantage over the 70B in FP16. 😅

GiteaMirror commented

2026-05-03 22:19:35 -05:00

@chigkim commented on GitHub (Jul 24, 2024):

We currently don't have access to the full fp16 version on Ollama

I thought Meta provides llama3.1-405b in fp16? Can't Ollama team just convert it to gguf and quantize based on that?

@chigkim commented on GitHub (Jul 24, 2024): > We currently don't have access to the full fp16 version on Ollama I thought Meta provides llama3.1-405b in fp16? Can't Ollama team just convert it to gguf and quantize based on that?

GiteaMirror commented

2026-05-03 22:19:35 -05:00

@gileneusz commented on GitHub (Jul 24, 2024):

I mean we, the users, I send some of my posts to gpt for grammar correction and it sometimes messes it up

@gileneusz commented on GitHub (Jul 24, 2024): I mean we, the users, I send some of my posts to gpt for grammar correction and it sometimes messes it up

GiteaMirror commented

2026-05-03 22:19:35 -05:00

@chigkim commented on GitHub (Jul 24, 2024):

For now, you can download q2 from here and import to Ollama.
https://huggingface.co/mradermacher/Meta-Llama-3.1-405B-Instruct-GGUF/tree/main

@chigkim commented on GitHub (Jul 24, 2024): For now, you can download q2 from here and import to Ollama. https://huggingface.co/mradermacher/Meta-Llama-3.1-405B-Instruct-GGUF/tree/main

GiteaMirror commented

2026-05-03 22:19:36 -05:00

@gileneusz commented on GitHub (Jul 24, 2024):

I tried to do that from this exact source, and I imported it to ollama, but it failed to run imported model properly, I have no experience with adding external models yet. I'll wait few days until someone will solve how to do it :)

@gileneusz commented on GitHub (Jul 24, 2024): I tried to do that from this exact source, and I imported it to ollama, but it failed to run imported model properly, I have no experience with adding external models yet. I'll wait few days until someone will solve how to do it :)

GiteaMirror commented

2026-05-03 22:19:37 -05:00

@kozuch commented on GitHub (Jul 28, 2024):

Well I guess we should be able to quantize lower precision versions like Q2 and Q1 from the Q4 that is already provided? However that means doing some manual work locally... As people reported Llama 3 70B to still be decent in Q2, Q1 may really be exciting with 405B. Can ollama or llama.cpp do this? May be a single command right?

@kozuch commented on GitHub (Jul 28, 2024): Well I guess we should be able to quantize lower precision versions like Q2 and Q1 from the Q4 that is already provided? However that means doing some manual work locally... As people reported Llama 3 70B to still be decent in Q2, Q1 may really be exciting with 405B. Can ollama or llama.cpp do this? May be a single command right?

GiteaMirror commented

2026-05-03 22:19:38 -05:00

@gwillen commented on GitHub (Jul 28, 2024):

For now, you can download q2 from here and import to Ollama. https://huggingface.co/mradermacher/Meta-Llama-3.1-405B-Instruct-GGUF/tree/main

This appears to have been deleted, which is a little concerning.

@gwillen commented on GitHub (Jul 28, 2024): > For now, you can download q2 from here and import to Ollama. https://huggingface.co/mradermacher/Meta-Llama-3.1-405B-Instruct-GGUF/tree/main This appears to have been deleted, which is a little concerning.

GiteaMirror commented

2026-05-03 22:19:39 -05:00

@chigkim commented on GitHub (Jul 28, 2024):

Try this one then:
https://huggingface.co/bullerwins/Meta-Llama-3.1-405B-Instruct-GGUF/tree/main

@chigkim commented on GitHub (Jul 28, 2024): Try this one then: https://huggingface.co/bullerwins/Meta-Llama-3.1-405B-Instruct-GGUF/tree/main

GiteaMirror commented

2026-05-03 22:19:39 -05:00

@igorschlum commented on GitHub (Aug 2, 2024):

It is possible for someone who done it to upload it on ollama on his account and share the link?
Best

@igorschlum commented on GitHub (Aug 2, 2024): It is possible for someone who done it to upload it on ollama on his account and share the link? Best

GiteaMirror commented

2026-05-03 22:19:40 -05:00

@igorschlum commented on GitHub (Aug 6, 2024):

Llama3.1 are being updated : https://ollama.com/library/llama3.1/tags
and 405B is provided in multiple Quantization (Q2... Q8).
@gileneusz Please close this issue to keep issues under 1000.

@igorschlum commented on GitHub (Aug 6, 2024): Llama3.1 are being updated : https://ollama.com/library/llama3.1/tags and 405B is provided in multiple Quantization (Q2... Q8). @gileneusz Please close this issue to keep issues under 1000.

GiteaMirror commented

2026-05-03 22:19:41 -05:00

@kozuch commented on GitHub (Aug 6, 2024):

Great. But can we also request Q1 for experimenting?

@kozuch commented on GitHub (Aug 6, 2024): Great. But can we also request Q1 for experimenting?

GiteaMirror commented

2026-05-03 22:19:41 -05:00

@igorschlum commented on GitHub (Aug 6, 2024):

Why Quantization to 1 Bit (Q1) is Ineffective:
Loss of Precision:
Quantizing to 1 bit means each weight and activation is represented by a single bit, allowing only two states (e.g., -1 and 1). This drastically reduces precision.
Large language models (LLMs) rely heavily on the precision of weights, and such an extreme reduction can significantly degrade the model's performance.
Complex Information:
LLMs handle complex textual data and subtle relationships between words. Q1 quantization cannot capture these nuances, leading to a significant drop in output quality.
Inference Performance:
The loss of information due to aggressive quantization can increase inference errors, possibly requiring costly correction mechanisms or additional inference steps, negating any speed and efficiency gains.
Why Quantization to 2 Bits (Q2) is Not Ideal:
Marginal Improvement:
Although Q2 offers a slight improvement over Q1 by allowing four possible states, it is still very limited compared to 4, 8, 16, or 32-bit representations.
Fine nuances in weights and activations are still largely lost, which can noticeably degrade model performance.

@igorschlum commented on GitHub (Aug 6, 2024): Why Quantization to 1 Bit (Q1) is Ineffective: Loss of Precision: Quantizing to 1 bit means each weight and activation is represented by a single bit, allowing only two states (e.g., -1 and 1). This drastically reduces precision. Large language models (LLMs) rely heavily on the precision of weights, and such an extreme reduction can significantly degrade the model's performance. Complex Information: LLMs handle complex textual data and subtle relationships between words. Q1 quantization cannot capture these nuances, leading to a significant drop in output quality. Inference Performance: The loss of information due to aggressive quantization can increase inference errors, possibly requiring costly correction mechanisms or additional inference steps, negating any speed and efficiency gains. Why Quantization to 2 Bits (Q2) is Not Ideal: Marginal Improvement: Although Q2 offers a slight improvement over Q1 by allowing four possible states, it is still very limited compared to 4, 8, 16, or 32-bit representations. Fine nuances in weights and activations are still largely lost, which can noticeably degrade model performance.

GiteaMirror commented

2026-05-03 22:19:42 -05:00

@kozuch commented on GitHub (Aug 6, 2024):

This is theory which was not really tested yet on large model like 405B. Real world tests done to date showed that lower bit quantizations get better performance the bigger the model is. Llama 3 70B still somehow performs in Q2, 405B may well also run in Q1. So that is why I am suggesting yet to do Q1 here. I would really appreciate it as it may show interesting things.

It is thought that LLMs have some kind of redundancy built inside the network (which may compensate for quantization) so I am really excited to see where is the quantization limit for 405B.

@igorschlum Did you make the Q2 quants yourself or where do they come from?

@kozuch commented on GitHub (Aug 6, 2024): This is theory which was not really tested yet on large model like 405B. Real world tests done to date showed that lower bit quantizations get better performance the bigger the model is. Llama 3 70B still somehow performs in Q2, 405B may well also run in Q1. So that is why I am suggesting yet to do Q1 here. I would really appreciate it as it may show interesting things. It is thought that LLMs have some kind of redundancy built inside the network (which may compensate for quantization) so I am really excited to see where is the quantization limit for 405B. @igorschlum Did you make the Q2 quants yourself or where do they come from?

GiteaMirror commented

2026-05-03 22:19:43 -05:00

@gileneusz commented on GitHub (Aug 6, 2024):

@kozuch do you mean for checking if Q1 405B would be better than FP16 70B? Then it could be worth it to add Q1 405B as well....

I'll close the issue tomorrow if no more answers will drop until then

@gileneusz commented on GitHub (Aug 6, 2024): @kozuch do you mean for checking if Q1 405B would be better than FP16 70B? Then it could be worth it to add Q1 405B as well.... I'll close the issue tomorrow if no more answers will drop until then

GiteaMirror commented

2026-05-03 22:19:44 -05:00

@igorschlum commented on GitHub (Aug 6, 2024):

@kozuch interesting, I will try the 405b-instruct-q2_K model on my MacStudio with 192 GB of RAM. I will try to work with it. I've seen that we can ask macOS to allocate more than 66% of the RAM to the GPU. I would like Meta to launch a 180GB model like Falcon did 9 months ago. It was using 101GB of RAM and that was perfect.

@igorschlum commented on GitHub (Aug 6, 2024): @kozuch interesting, I will try the 405b-instruct-q2_K model on my MacStudio with 192 GB of RAM. I will try to work with it. I've seen that we can ask macOS to allocate more than 66% of the RAM to the GPU. I would like Meta to launch a 180GB model like Falcon did 9 months ago. It was using 101GB of RAM and that was perfect.

GiteaMirror commented

2026-05-03 22:19:44 -05:00

@gileneusz commented on GitHub (Aug 6, 2024):

@igorschlum seems like there is an issue with running q2_K

ollama run llama3.1:405b-instruct-q2_K 
Error: llama runner process has terminated: error:done_getting_tensors: wrong number of tensors; expected 1138, got 1137

I'm using Mac Studio 192GB

You can allocate up to 188 VRAM
https://www.reddit.com/r/LocalLLaMA/comments/192uirj/188gb_vram_on_mac_studio_m2_ultra_easy/

@gileneusz commented on GitHub (Aug 6, 2024): @igorschlum seems like there is an issue with running q2_K ``` ollama run llama3.1:405b-instruct-q2_K Error: llama runner process has terminated: error:done_getting_tensors: wrong number of tensors; expected 1138, got 1137 ``` I'm using Mac Studio 192GB You can allocate up to 188 VRAM https://www.reddit.com/r/LocalLLaMA/comments/192uirj/188gb_vram_on_mac_studio_m2_ultra_easy/

GiteaMirror commented

2026-05-03 22:19:44 -05:00

@kozuch commented on GitHub (Aug 6, 2024):

Well the 405b-instruct-q2_K has 2.98 bits per weight (bpw, 151/405*8) so that is almost 3 bits. We also need lower Q2 quants like Q2_K_S.

@gileneusz You should report the bug above in a separate issue.

I did not know that you can allocate standard RAM to GPU on Mac. very interesting.

@kozuch commented on GitHub (Aug 6, 2024): Well the 405b-instruct-q2_K has 2.98 bits per weight (bpw, 151/405*8) so that is almost 3 bits. We also need lower Q2 quants like Q2_K_S. @gileneusz You should report the bug above in a separate issue. I did not know that you can allocate standard RAM to GPU on Mac. very interesting.

GiteaMirror commented

2026-05-03 22:19:45 -05:00

@chigkim commented on GitHub (Aug 6, 2024):

Also, there are only quants for 405b instruct, not base (text). Recently Ollama library uploaded base model for 8b and 70b, but not 405b.

@chigkim commented on GitHub (Aug 6, 2024): Also, there are only quants for 405b instruct, not base (text). Recently Ollama library uploaded base model for 8b and 70b, but not 405b.

GiteaMirror commented

2026-05-03 22:19:45 -05:00

@igorschlum commented on GitHub (Aug 6, 2024):

@gileneusz I searched on Google for the error: "llama runner process has terminated: error: done_getting_tensors: wrong number of tensors" and it seems that this issue should be resolved in latest versions of Llama 3.1 and Llama.cpp.

Ollama version 0.3.4 is currently available. Have you tried it to see if the problem is resolved? The Llama 3.1 models are being updated right now on ollama.com, and I hope that Ollama 0.3.4 will work well with those updated models.

@igorschlum commented on GitHub (Aug 6, 2024): @gileneusz I searched on Google for the error: "llama runner process has terminated: error: done_getting_tensors: wrong number of tensors" and it seems that this issue should be resolved in latest versions of Llama 3.1 and Llama.cpp. Ollama version 0.3.4 is currently available. Have you tried it to see if the problem is resolved? The Llama 3.1 models are being updated right now on ollama.com, and I hope that Ollama 0.3.4 will work well with those updated models.

GiteaMirror commented

2026-05-03 22:19:47 -05:00

@gileneusz commented on GitHub (Aug 7, 2024):

@igorschlum It runs after update to 0.3.3, however it fits all the 192GB of RAM and even swap for 10GB, I need to run this on fresh system, since I have many other stuff loaded into shared RAM, it should work. Very slow inference speed and quality is not better than q4_0 70B, but it's my opinion after just few tokens 😅

without 2xH100 or 4xH100 it doesn't make any sense...

But I'm going to have access to that hardware soon, so those quants will be very useful 😇

@gileneusz commented on GitHub (Aug 7, 2024): @igorschlum It runs after update to 0.3.3, however it fits all the 192GB of RAM and even swap for 10GB, I need to run this on fresh system, since I have many other stuff loaded into shared RAM, it should work. Very slow inference speed and quality is not better than q4_0 70B, but it's my opinion after just few tokens 😅 without 2xH100 or 4xH100 it doesn't make any sense... But I'm going to have access to that hardware soon, so those quants will be very useful 😇

GiteaMirror commented

2026-05-03 22:19:49 -05:00

@igorschlum commented on GitHub (Aug 8, 2024):

@gileneusz I will use Ollama for translation, so I hope that 405B with works well. I will see. Thank you.

@igorschlum commented on GitHub (Aug 8, 2024): @gileneusz I will use Ollama for translation, so I hope that 405B with works well. I will see. Thank you.

GiteaMirror commented

2026-05-03 22:19:51 -05:00

@chigkim commented on GitHub (Aug 10, 2024):

It seems like meta updated 405b?

https://huggingface.co/collections/meta-llama/llama-31-669fc079a0c406a149a5738f

"Without explanation, Meta changed the number of KV heads from 16 to 8 (which now matches the whitepaper) for the 405B model. This is not just a config change, the whole model has been updated 😵"

https://www.reddit.com/r/LocalLLaMA/comments/1eoin62/meta_just_pushed_a_new_llama_31_405b_to_hf/

@chigkim commented on GitHub (Aug 10, 2024): It seems like meta updated 405b? https://huggingface.co/collections/meta-llama/llama-31-669fc079a0c406a149a5738f "Without explanation, Meta changed the number of KV heads from 16 to 8 (which now matches the whitepaper) for the 405B model. This is not just a config change, the whole model has been updated 😵" https://www.reddit.com/r/LocalLLaMA/comments/1eoin62/meta_just_pushed_a_new_llama_31_405b_to_hf/

GiteaMirror commented

2026-05-03 22:19:53 -05:00

@igorschlum commented on GitHub (Aug 11, 2024):

I try some prompts and yes, the results are worst with 405b:q3_K_S than 70b_q8_0.
I try ollama run llama3.1:405b:q3_K_S and after loads and swaps, I got an Error : Killed.
the model is 195 Gb and at the best, my mac is only 192.

@igorschlum commented on GitHub (Aug 11, 2024): I try some prompts and yes, the results are worst with 405b:q3_K_S than 70b_q8_0. I try ollama run llama3.1:405b:q3_K_S and after loads and swaps, I got an Error : Killed. the model is 195 Gb and at the best, my mac is only 192.

Sign in to join this conversation.

Branches Tags

main

parth-agent-tui-slash-selector

parth-remove-ollama-agent-command

parth-agent-harness-skills-synthetic-tool

hoyyeva/fix-anthropic-text-before-thinking

parth-agent-cli-markdown-rendering

mxyng/docs-cloud

parth-update-hermes-launch

hoyyeva/vscode-extension-docs-update

parth-gemma4-chat-template-renderer

parth-api-status-context-length

hoyyeva/wire-up-context-length

hoyyeva/claude-code-context-doc

jmorganca/investigate-issue-17046

hoyyeva/hermes-docs

jmorganca/agent-loop-style

hoyyeva/openclaw

parth-agent-loop

hoyyeva/ollama-vscode-extension

brucemacd/cache-metrics

brucemacd/hermes-desktop

hoyyeva/docs-vscode

parth-input-style-experiment

brucemacd/docs-glm52

hoyyeva/poc-docs

Parth/mlx-launch-recommendations

parth-first-time-app-cli-experience

test/darwin-xcode-pin

improve-cloud-model-recommendations

hoyyeva/goose-docs

jmorganca/context-limit-fixes

hoyyeva/qwen-doc

hoyyeva/vscode-docs

jmorganca/remove-mlx-imagegen-code

parth-copilot-token-length-defaults

hoyyeva/poolside-windows

laguna-support

jmorganca/harden-markdown-rendering

laguna-renderer-parser

laguna-llamacpp

codex/make-integration-hidden-and-lunchable

brucemacd/omp-docs

pdevine/gguf-mtp-oldstyle

hoyyeva/migrate-pi

hoyyeva/anthropic-local-image-path

parth-launch-codex-app

hoyyeva/anthropic-reference-images-path

parth-anthropic-reference-images-path

brucemacd/download-before-remove

hoyyeva/editor-config-repair

parth-mlx-decode-checkpoints

parth/hide-claude-desktop-till-release

parth-add-claude-code-autoinstall

release_v0.22.0

pdevine/manifest-list

codex/fix-codex-model-metadata-warning

pdevine/addressable-manifest

brucemacd/launch-fetch-reccomended

jmorganca/llama-compat

launch-copilot-cli

release_v0.20.7

parth-auto-save-backup

parth-test

jmorganca/gemma4-audio-replacements

fix-manifest-digest-on-pull

hoyyeva/vscode-improve

brucemacd/install-server-wait

parth/update-claude-docs

brucemac/start-ap-install

pdevine/mlx-update

pdevine/qwen35_vision

drifkin/api-show-fallback

mintlify/image-generation-1773352582

hoyyeva/server-context-length-local-config

jmorganca/faster-reptition-penalties

jmorganca/convert-nemotron

parth-pi-thinking

pdevine/sampling-penalties

jmorganca/fix-create-quantization-memory

dongchen/resumable_transfer_fix

pdevine/sampling-cache-error

jessegross/mlx-usage

hoyyeva/openclaw-config

hoyyeva/app-html

pdevine/qwen3next

brucemacd/sign-sh-install

brucemacd/tui-update

brucemacd/usage-api

jmorganca/launch-empty

fix-app-dist-embed

mxyng/mlx-compile

mxyng/mlx-quant

mxyng/mlx-glm4.7

mxyng/mlx

brucemacd/simplify-model-picker

jmorganca/qwen3-concurrent

fix-glm-4.7-flash-mla-config

drifkin/qwen3-coder-opening-tag

brucemacd/usage-cli

fix-cuda12-fattn-shmem

ollama-imagegen-docs

parth/fix-multiline-inputs

brucemacd/config-docs

mxyng/model-files

mxyng/simple-execute

fix-imagegen-ollama-models

mxyng/async-upload

jmorganca/lazy-no-dtype-changes

imagegen-auto-detect-create

parth/decrease-concurrent-download-hf

fix-mlx-quantize-init

jmorganca/x-cleanup

usage

imagegen-readme

jmorganca/glm-image

mlx-gpu-cd

jmorganca/imagegen-modelfile

parth/agent-skills

parth/agent-allowlist

parth/signed-in-offline

parth/agents

parth/fix-context-chopping

improve-cloud-flow

parth/add-models-websearch

parth/prompt-renderer-mcp

jmorganca/native-settings

jmorganca/download-stream-hash

jmorganca/client2-rebased

brucemacd/oai-chat-req-multipart

jessegross/multi_chunk_reserve

grace/additional-omit-empty

grace/mistral-3-large

mxyng/tokenizer2

mxyng/tokenizer

jessegross/flash

hoyyeva/windows-nacked-app

mxyng/cleanup-attention

grace/deepseek-parser

hoyyeva/remember-unsent-prompt

parth/add-lfs-pointer-error-conversion

parth/olmo2-test2

hoyyeva/ollama-launchagent-plist

nicole/olmo-model

parth/olmo-test

mxyng/remove-embedded

parth/render-template

jmorganca/intellect-3

parth/remove-prealloc-linter

jmorganca/cmd-eval

nicole/nomic-embed-text-fix

mxyng/lint-2

hoyyeva/add-gemini-3-pro-preview

hoyyeva/load-model-list

mxyng/expand-path

mxyng/environ-2

hoyyeva/deeplink-json-encoding

parth/improve-tool-calling-tests

hoyyeva/conversation

hoyyeva/assistant-edit-response

hoyyeva/thinking

origin/brucemacd/invalid-char-i-err

parth/improve-tool-calling

jmorganca/required-omitempty

grace/qwen3-vl-tests

mxyng/iter-client

parth/docs-readme

nicole/embed-test

pdevine/integration-benchstat

parth/remove-generate-cmd

parth/add-toolcall-id

mxyng/server-tests

jmorganca/glm-4.6

jmorganca/gin-h-compat

drifkin/stable-tool-args

pdevine/qwen3-more-thinking

parth/add-websearch-client

nicole/websearch_local

jmorganca/qwen3-coder-updates

grace/deepseek-v3-migration-tests

mxyng/fix-create

jmorganca/cloud-errors

pdevine/parser-tidy

revert-12233-parth/simplify-entrypoints-runner

parth/enable-so-gpt-oss

brucemacd/qwen3vl

jmorganca/readme-simplify

parth/gpt-oss-structured-outputs

revert-12039-jmorganca/tools-braces

mxyng/embeddings

mxyng/gguf

mxyng/benchmark

mxyng/types-null

parth/move-parsing

mxyng/gemma2

jmorganca/docs

mxyng/16-bit

mxyng/create-stdin

pdevine/authorizedkeys

mxyng/quant

parth/opt-in-error-context-window

brucemacd/cache-models

brucemacd/runner-completion

jmorganca/llama-update-6

brucemacd/benchmark-list

brucemacd/partial-read-caps

parth/deepseek-r1-tools

mxyng/omit-array

parth/tool-prefix-temp

brucemacd/runner-test

jmorganca/qwen25vl

brucemacd/model-forward-test-ext

parth/python-function-parsing

jmorganca/cuda-compression-none

drifkin/num-parallel

drifkin/chat-truncation-fix

jmorganca/sync

parth/python-tools-calling

drifkin/array-head-count

brucemacd/create-no-loop

parth/server-enable-content-stream-with-tools

qwen25omni

mxyng/v3

brucemacd/ropeconfig

jmorganca/silence-tokenizer

parth/sample-so-test

parth/sampling-structured-outputs

brucemacd/doc-go-engine

parth/constrained-sampling-json

jmorganca/mistral-wip

brucemacd/mistral-small-convert

parth/sample-unmarshal-json-for-params

brucemacd/jomorganca/mistral

pdevine/bfloat16

jmorganca/mistral

brucemacd/mistral

pdevine/logging

parth/sample-correctness-fix

parth/sample-fix-sorting

jmorgan/sample-fix-sorting-extras

jmorganca/temp-0-images

brucemacd/parallel-embed-models

brucemacd/shim-grammar

jmorganca/fix-gguf-error

bmizerany/nameswork

jmorganca/faster-releases

bmizerany/validatenames

brucemacd/err-no-vocab

brucemacd/rope-config

brucemacd/err-hint

brucemacd/qwen2_5

brucemacd/logprobs

brucemacd/new_runner_graph_bench

progress-flicker

brucemacd/forward-test

brucemacd/go_qwen2

pdevine/gemma2

jmorganca/add-missing-symlink-eval

mxyng/next-debug

parth/set-context-size-openai

brucemacd/next-bpe-bench

brucemacd/next-bpe-test

brucemacd/new_runner_e2e

brucemacd/new_runner_qwen2

pdevine/convert-cohere2

brucemacd/convert-cli

parth/log-probs

mxyng/next-mlx

mxyng/cmd-history

parth/templating

parth/tokenize-detokenize

brucemacd/check-key-register

bmizerany/grammar

jmorganca/vendor-081b29bd

mxyng/func-checks

jmorganca/fix-null-format

parth/fix-default-to-warn-json

jmorganca/qwen2vl

jmorganca/no-concat

parth/cmd-cleanup-SO

brucemacd/check-key-register-structured-err

parth/openai-stream-usage

parth/fix-referencing-so

stream-tools-stop

jmorganca/degin-1

brucemacd/install-path-clean

brucemacd/push-name-validation

brucemacd/browser-key-register

jmorganca/openai-fix-first-message

jmorganca/fix-proxy

jessegross/sample

parth/disallow-streaming-tools

dhiltgen/remove_submodule

jmorganca/ga

jmorganca/mllama

pdevine/newlines

pdevine/geems-2b

jmorganca/llama-bump

mxyng/modelname-7

mxyng/gin-slog

mxyng/modelname-6

jyan/convert-prog

jyan/quant5

paligemma-support

pdevine/import-docs

jmorganca/openai-context

jyan/paligemma

jyan/p2

jyan/palitest

bmizerany/embedspeedup

jmorganca/llama-vit

brucemacd/allow-ollama

royh/ep-methods

royh/whisper

mxyng/api-models

mxyng/fix-memory

jyan/q4_4/8

jyan/ollama-v

royh/stream-tools

roy-embed-parallel

bmizerany/hrm

revert-5963-revert-5924-mxyng/llama3.1-rope

royh/embed-viz

jyan/local2

jyan/auth

jyan/local

jyan/parse-temp

jmorganca/template-mistral

jyan/reord-g

royh-openai-suffixdocs

royh-imgembed

royh-embed-parallel

jyan/quant4

royh-precision

jyan/progress

pdevine/fix-template

jyan/quant3

pdevine/ggla

mxyng/update-registry-domain

jmorganca/ggml-static

mxyng/create-context

jyan/v0.146

mxyng/layers-from-files

build_dist

bmizerany/noseek

royh-ls

royh-name

timeout

mxyng/server-timestamp

bmizerany/nosillyggufslurps

royh-params

jmorganca/llama-cpp-7c26775

royh-openai-delete

royh-show-rigid

jmorganca/enable-fa

jmorganca/no-error-template

jyan/format

royh-testdelete

bmizerany/fastverify

language_support

pdevine/ps-glitches

brucemacd/tokenize

bruce/iq-quants

bmizerany/filepathwithcoloninhost

mxyng/split-bin

bmizerany/client-registry

jmorganca/if-none-match

native

jmorganca/native

jmorganca/batch-embeddings

jmorganca/initcmake

jmorganca/mm

pdevine/showggmlinfo

modenameenforcealphanum

bmizerany/modenameenforcealphanum

jmorganca/done-reason

jmorganca/llama-cpp-8960fe8

ollama.com

bmizerany/filepathnobuild

bmizerany/types/model/defaultfix

rmdisplaylong

nogogen

bmizerany/x

modelfile-readme

bmizerany/replacecolon

jmorganca/limit

jmorganca/execstack

jmorganca/replace-assets

mxyng/tune-concurrency

jmorganca/testing

whitespace-detection

jmorganca/options

upgrade-all

scratch

cuda-search

mattw/airenamer

mattw/allmodelsonhuggingface

mattw/quantcontext

mattw/whatneedstorun

brucemacd/llama-mem-calc

mattw/faq-context

mattw/communitylinks

mattw/noprune

mattw/python-functioncalling

rename

mxyng/install

pulse

remove-first

editor

mattw/selfqueryingretrieval

cgo

mattw/howtoquant

api

matt/streamingapi

format-config

mxyng/extra-args

shell

update-nous-hermes

cp-model

upload-progress

fix-unknown-model

fix-model-names

delete-fix

insecure-registry

ls

deletemodels

progressbar

readme-updates

license-layers

skip-list

list-models

modelpath

matt/examplemodelfiles

distribution

go-opts

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/ollama#65712