[GH-ISSUE #4051] Enable Flash Attention on GGML/GGUF (feature now merged into llama.cpp) #28273

New Issue

GiteaMirror · 2026-04-22T06:14:17-05:00

GiteaMirror commented

2026-04-22 06:14:17 -05:00

Originally created by @sammcj on GitHub (Apr 30, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/4051

Flash Attention has landed in llama.cpp (https://github.com/ggerganov/llama.cpp/pull/5021).

The tldr; is simply to pass the -fa flag to llama.cpp’s server.

Can we please have an Ollama server env var to pass this flag to the underlying llama.cpp server?

also a related idea - perhaps there could be a way to pass arbitrary flags down to llama.cpp so that hints like this can be easily enabled? (E.g. OLLAMA_LLAMA_EXTRA_ARGS=-fa,—something-else

Originally created by @sammcj on GitHub (Apr 30, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/4051 Flash Attention has landed in llama.cpp (https://github.com/ggerganov/llama.cpp/pull/5021). The tldr; is simply to pass the -fa flag to llama.cpp’s server. - Can we please have an Ollama server env var to pass this flag to the underlying llama.cpp server? also a related idea - perhaps there could be a way to pass arbitrary flags down to llama.cpp so that hints like this can be easily enabled? (E.g. `OLLAMA_LLAMA_EXTRA_ARGS=-fa,—something-else`

GiteaMirror added the feature request label 2026-04-22 06:14:17 -05:00

GiteaMirror closed this issue

2026-04-22 06:14:18 -05:00

GiteaMirror commented

2026-04-22 06:14:19 -05:00

@DuckyBlender commented on GitHub (Apr 30, 2024):

Are there any cons for using Flash Attention? If not it probably should be the default.

@DuckyBlender commented on GitHub (Apr 30, 2024): Are there any cons for using Flash Attention? If not it probably should be the default.

GiteaMirror commented

2026-04-22 06:14:19 -05:00

@DuckyBlender commented on GitHub (Apr 30, 2024):

Also if there is more memory available we should really set the default quants to Q4_K_M and fallback to Q4 when Q4_K_M is not available

@DuckyBlender commented on GitHub (Apr 30, 2024): Also if there is more memory available we should really set the default quants to Q4_K_M and fallback to Q4 when Q4_K_M is not available

GiteaMirror commented

2026-04-22 06:14:20 -05:00

@jukofyork commented on GitHub (May 1, 2024):

also a related idea - perhaps there could be a way to pass arbitrary flags down to llama.cpp so that hints like this can be easily enabled? (E.g. OLLAMA_LLAMA_EXTRA_ARGS=-fa,—something-else

Yeah, I think this would help a lot.

@jukofyork commented on GitHub (May 1, 2024): > also a related idea - perhaps there could be a way to pass arbitrary flags down to llama.cpp so that hints like this can be easily enabled? (E.g. `OLLAMA_LLAMA_EXTRA_ARGS=-fa,—something-else` Yeah, I think this would help a lot.

GiteaMirror commented

2026-04-22 06:14:20 -05:00

@jukofyork commented on GitHub (May 2, 2024):

Flash Attention has landed in llama.cpp (ggerganov/llama.cpp#5021).

The tldr; is simply to pass the -fa flag to llama.cpp’s server.
* Can we please have an Ollama server env var to pass this flag to the underlying llama.cpp server?
also a related idea - perhaps there could be a way to pass arbitrary flags down to llama.cpp so that hints like this can be easily enabled? (E.g. OLLAMA_LLAMA_EXTRA_ARGS=-fa,—something-else

Just been looking at the source to see if this can be done:

https://github.com/ollama/ollama/blob/main/llm/ext_server/server.cpp

and even though it's importing the latest gpt_params structure (from the lamma.cpp submodule dated 2 days ago), all the parsing is using some server_params_parse function stripped from a outdated version of the llama.cpp server.

Some of the default values (eg: --repeat-penalty) and even meanings (eg: --batch) are also behind because of this.

I guess we could try to make a diff to see if this can be updated, but since PRs never seem to get accepted and random stuff constantly seems to change regarding the passing of command line options, I'm reluctant to even try...

@jukofyork commented on GitHub (May 2, 2024): > Flash Attention has landed in llama.cpp ([ggerganov/llama.cpp#5021](https://github.com/ggerganov/llama.cpp/pull/5021)). > > The tldr; is simply to pass the -fa flag to llama.cpp’s server. > > * Can we please have an Ollama server env var to pass this flag to the underlying llama.cpp server? > > > also a related idea - perhaps there could be a way to pass arbitrary flags down to llama.cpp so that hints like this can be easily enabled? (E.g. `OLLAMA_LLAMA_EXTRA_ARGS=-fa,—something-else` Just been looking at the source to see if this can be done: https://github.com/ollama/ollama/blob/main/llm/ext_server/server.cpp and even though it's importing the latest `gpt_params` structure (from the lamma.cpp submodule dated 2 days ago), all the parsing is using some `server_params_parse` function stripped from a outdated version of the llama.cpp server. Some of the default values (eg: --repeat-penalty) and even meanings (eg: --batch) are also behind because of this. I guess we could try to make a diff to see if this can be updated, but since PRs never seem to get accepted and random stuff constantly seems to change regarding the passing of command line options, I'm reluctant to even try...

GiteaMirror commented

2026-04-22 06:14:22 -05:00

@sammcj commented on GitHub (May 2, 2024):

I had a hot crack at a PR tonight but as you said @jukofyork it seems server.cpp has diverged from llama.cpp a lot and I couldn't get my head around it.

Might have to wait for someone more familiar with Ollama's version of it / more brains than me.

@sammcj commented on GitHub (May 2, 2024): I had a hot crack at a PR tonight but as you said @jukofyork it seems server.cpp has diverged from llama.cpp a lot and I couldn't get my head around it. Might have to wait for someone more familiar with Ollama's version of it / more brains than me.

GiteaMirror commented

2026-04-22 06:14:22 -05:00

@sammcj commented on GitHub (May 3, 2024):

FYI: LM Studio added Flash Attention earlier today: https://www.reddit.com/r/LocalLLaMA/comments/1cir98j/lm_studio_released_new_version_with_flash/

@sammcj commented on GitHub (May 3, 2024): FYI: LM Studio added Flash Attention earlier today: https://www.reddit.com/r/LocalLLaMA/comments/1cir98j/lm_studio_released_new_version_with_flash/

GiteaMirror commented

2026-04-22 06:14:23 -05:00

@sammcj commented on GitHub (May 3, 2024):

Comparing GGUF performance with/without Flash Attention

Hardware

Apple Macbook Pro M2 Max (96GB)

Model

llama3-bartowski-8b-instruct-q8-0.gguf

Request

curl http://localhost:11434/v1/chat/completions -H "Content-Type: application/json" -d '{
        "model": "registry.internal/ollama/llama3-bartowski:8b-instruct-q8_0",
        "messages": [
            {
                "role": "user",
                "content": "Tell me two short jokes."
            }
        ]
    }'

LM Studio

With Flash Attention

Start time: 14:10:20.401
End time: 14:10:21.636
"prompt_tokens": 38,
"completion_tokens": 37,
"total_tokens": 75

1.235 seconds~
60.65 tokens/s~

Ollama

Without Flash Attention

total duration:       1.866343375s
load duration:        1.299853709s
prompt eval count:    15 token(s)
prompt eval duration: 84.22ms
prompt eval rate:     178.10 tokens/s
eval count:           18 token(s)
eval duration:        481.335ms
eval rate:            37.40 tokens/s

Start TIMESTAMP: 1714709035
End TIMESTAMP: 1714709036

1.866 seconds
40.38 tokens/s

@sammcj commented on GitHub (May 3, 2024): Comparing GGUF performance with/without Flash Attention ## Hardware - Apple Macbook Pro M2 Max (96GB) ## Model - llama3-bartowski-8b-instruct-q8-0.gguf ## Request ``` curl http://localhost:11434/v1/chat/completions -H "Content-Type: application/json" -d '{ "model": "registry.internal/ollama/llama3-bartowski:8b-instruct-q8_0", "messages": [ { "role": "user", "content": "Tell me two short jokes." } ] }' ``` ## LM Studio With Flash Attention ``` Start time: 14:10:20.401 End time: 14:10:21.636 "prompt_tokens": 38, "completion_tokens": 37, "total_tokens": 75 ``` - 1.235 seconds~ - **60.65 tokens/s~** ## Ollama Without Flash Attention ```` total duration: 1.866343375s load duration: 1.299853709s prompt eval count: 15 token(s) prompt eval duration: 84.22ms prompt eval rate: 178.10 tokens/s eval count: 18 token(s) eval duration: 481.335ms eval rate: 37.40 tokens/s Start TIMESTAMP: 1714709035 End TIMESTAMP: 1714709036 ```` - 1.866 seconds - **40.38 tokens/s**

GiteaMirror commented

2026-04-22 06:14:24 -05:00

@wanderingmeow commented on GitHub (May 3, 2024):

Got it working by adding the OLLAMA_LLAMA_EXTRA_ARGS environment variable in llm/server.go as @sammcj suggested. This allows the -fa flag be passed into ext_server/server.cpp. If anyone is interested, my hack can be found here. Hopefully this can help others get Flash Attention working too.

@wanderingmeow commented on GitHub (May 3, 2024): Got it working by adding the `OLLAMA_LLAMA_EXTRA_ARGS` environment variable in `llm/server.go` as @sammcj suggested. This allows the `-fa` flag be passed into `ext_server/server.cpp`. If anyone is interested, my hack can be found [here](https://github.com/wanderingmeow/ollama). Hopefully this can help others get Flash Attention working too.

GiteaMirror commented

2026-04-22 06:14:26 -05:00

@sammcj commented on GitHub (May 3, 2024):

That looks VERY similar to what I tried @wanderingmeow, perhaps I just wasn't parsing the parameters correctly 🤔 , that's awesome!

@sammcj commented on GitHub (May 3, 2024): That looks VERY similar to what I tried @wanderingmeow, perhaps I just wasn't parsing the parameters correctly 🤔 , that's awesome!

GiteaMirror commented

2026-04-22 06:14:27 -05:00

@sammcj commented on GitHub (May 3, 2024):

That worked instantly, still no where near as fast as LM Studio but it's a start.

export OLLAMA_LLAMA_EXTRA_ARGS="-fa"

ollama serve

...
llama_new_context_with_model: n_ctx      = 2048
llama_new_context_with_model: n_batch    = 512
llama_new_context_with_model: n_ubatch   = 512
llama_new_context_with_model: flash_attn = 1
llama_new_context_with_model: freq_base  = 500000.0
llama_new_context_with_model: freq_scale = 1
...

ollama run registry.internal/ollama/llama3-bartowski:8b-instruct-q8_0 'tell me two short jokes' --verbose
Here are two short jokes:

1. Why don't scientists trust atoms? Because they make up everything!
2. Why don't eggs tell jokes? They'd crack each other up!

Hope you find them amusing!

total duration:       2.539137625s
load duration:        1.286854542s
prompt eval count:    15 token(s)
prompt eval duration: 83.363ms
prompt eval rate:     179.94 tokens/s
eval count:           44 token(s)
eval duration:        1.168093s
eval rate:            37.67 tokens/s

@sammcj commented on GitHub (May 3, 2024): That worked instantly, still no where near as fast as LM Studio but it's a start. ``` export OLLAMA_LLAMA_EXTRA_ARGS="-fa" ollama serve ... llama_new_context_with_model: n_ctx = 2048 llama_new_context_with_model: n_batch = 512 llama_new_context_with_model: n_ubatch = 512 llama_new_context_with_model: flash_attn = 1 llama_new_context_with_model: freq_base = 500000.0 llama_new_context_with_model: freq_scale = 1 ... ``` ``` ollama run registry.internal/ollama/llama3-bartowski:8b-instruct-q8_0 'tell me two short jokes' --verbose Here are two short jokes: 1. Why don't scientists trust atoms? Because they make up everything! 2. Why don't eggs tell jokes? They'd crack each other up! Hope you find them amusing! total duration: 2.539137625s load duration: 1.286854542s prompt eval count: 15 token(s) prompt eval duration: 83.363ms prompt eval rate: 179.94 tokens/s eval count: 44 token(s) eval duration: 1.168093s eval rate: 37.67 tokens/s ```

GiteaMirror commented

2026-04-22 06:14:28 -05:00

@sammcj commented on GitHub (May 3, 2024):

@wanderingmeow I've created a PR with the changes - https://github.com/ollama/ollama/pull/4120

@sammcj commented on GitHub (May 3, 2024): @wanderingmeow I've created a PR with the changes - https://github.com/ollama/ollama/pull/4120

GiteaMirror commented

2026-04-22 06:14:28 -05:00

@jukofyork commented on GitHub (May 3, 2024):

Awesome! 👍

@jukofyork commented on GitHub (May 3, 2024): Awesome! :+1:

GiteaMirror commented

2026-04-22 06:14:29 -05:00

@jukofyork commented on GitHub (May 3, 2024):

I wonder if we can extend this:

	if other_args := os.Getenv("OLLAMA_LLAMA_EXTRA_ARGS"); other_args != "" {
		params = append(params, strings.Split(other_args, ",")...)
	}

To use a reg-ex to match against the model name passed in the model string?

Then we can just completely bypass the modelfile code and pass model-specific parameters as well? This would save all the hassle of PRs not getting accepted and then breaking because the code all got moved around!

It looks like we could just copy in server_params_parse from the sub-module (possibly automatically using AWK or something) and we would have access to any parameter added to llama.cpp.

It probably doesn't want to be split using "," though as that is used by llama.cpp for some options like tensor-split, etc.

@jukofyork commented on GitHub (May 3, 2024): I wonder if we can extend this: ``` if other_args := os.Getenv("OLLAMA_LLAMA_EXTRA_ARGS"); other_args != "" { params = append(params, strings.Split(other_args, ",")...) } ``` To use a reg-ex to match against the model name passed in the `model` string? Then we can just completely bypass the modelfile code and pass model-specific parameters as well? This would save all the hassle of PRs not getting accepted and then breaking because the code all got moved around! It looks like we could just copy in `server_params_parse` from the sub-module (possibly automatically using AWK or something) and we would have access to any parameter added to `llama.cpp`. It probably doesn't want to be split using "," though as that is used by `llama.cpp` for some options like `tensor-split`, etc.

GiteaMirror commented

2026-04-22 06:14:31 -05:00

@jukofyork commented on GitHub (May 3, 2024):

Something like:

export OLLAMA_LLAMA_EXTRA_ARGS=".* ; -fa | qwen.* ; -sm row ; -ts 3,2 | llama2.* ; --rope-freq-base 8192"

Although I'm not sure if the model string will actually be the name or the weird sha hash name that points to the GGUF file?

@jukofyork commented on GitHub (May 3, 2024): Something like: ``` export OLLAMA_LLAMA_EXTRA_ARGS=".* ; -fa | qwen.* ; -sm row ; -ts 3,2 | llama2.* ; --rope-freq-base 8192" ``` Although I'm not sure if the `model` string will actually be the name or the weird sha hash name that points to the GGUF file?

GiteaMirror commented

2026-04-22 06:14:32 -05:00

@wanderingmeow commented on GitHub (May 3, 2024):

still no where near as fast as LM Studio but it's a start.

From my testing, the current ollama ext_server implementation performs on par with the latest server example from llama.cpp, with only a slight slowdown (<1%).

@wanderingmeow commented on GitHub (May 3, 2024): > still no where near as fast as LM Studio but it's a start. From my testing, the current ollama `ext_server` implementation performs on par with the latest `server` example from llama.cpp, with only a slight slowdown (<1%).

GiteaMirror commented

2026-04-22 06:14:33 -05:00

@sammcj commented on GitHub (May 3, 2024):

It looks like we could just copy in server_params_parse from the sub-module (possibly automatically using AWK or something) and we would have access to any parameter added to llama.cpp.

@jukofyork I had a look at this, but it was getting messy, fast.

What I think probably could work is:

Split out Ollama's custom server configuration from the model server parameters.
Do the same in llama.cpp in PR (if @ggerganov thinks this might be a good idea).
Then Ollama or any project that wants to use llama.cpp's model server parameters library can do so separate from their server configuration logic.

That's way out of my ball park though 😅

@sammcj commented on GitHub (May 3, 2024): > It looks like we could just copy in server_params_parse from the sub-module (possibly automatically using AWK or something) and we would have access to any parameter added to llama.cpp. @jukofyork I had a look at this, but it was getting messy, fast. What I think probably could work is: 1. Split out Ollama's custom server configuration from the model server parameters. 2. Do the same in llama.cpp in PR (if @ggerganov thinks this might be a good idea). 3. Then Ollama or any project that wants to use llama.cpp's model server parameters library can do so separate from their server configuration logic. That's way out of my ball park though 😅

GiteaMirror commented

2026-04-22 06:14:34 -05:00

@jukofyork commented on GitHub (May 4, 2024):

still no where near as fast as LM Studio but it's a start.

From my testing, the current ollama ext_server implementation performs on par with the latest server example from llama.cpp, with only a slight slowdown (<1%).

It really helps reduce the VRAM use of long-context models: some I could only run at 16k or 32k are now running with 32k or 64k for the same quant!

@jukofyork commented on GitHub (May 4, 2024): > > still no where near as fast as LM Studio but it's a start. > > From my testing, the current ollama `ext_server` implementation performs on par with the latest `server` example from llama.cpp, with only a slight slowdown (<1%). It really helps reduce the VRAM use of long-context models: some I could only run at 16k or 32k are now running with 32k or 64k for the same quant!

GiteaMirror commented

2026-04-22 06:14:36 -05:00

@jukofyork commented on GitHub (May 4, 2024):

It looks like we could just copy in server_params_parse from the sub-module (possibly automatically using AWK or something) and we would have access to any parameter added to llama.cpp.

@jukofyork I had a look at this, but it was getting messy, fast.

What I think probably could work is:
1. Split out Ollama's custom server configuration from the model server parameters.

2. Do the same in llama.cpp in PR (if @ggerganov thinks this might be a good idea).

3. Then Ollama or any project that wants to use llama.cpp's model server parameters library can do so separate from their server configuration logic.
That's way out of my ball park though 😅

Yeah, but it would be good for the long-run: some of the parameters and their default settings that Ollama is using are very out of date compared to llama.cpp's server now and I can only see things getting worse and worse if nothing is done about it.

@jukofyork commented on GitHub (May 4, 2024): > > It looks like we could just copy in server_params_parse from the sub-module (possibly automatically using AWK or something) and we would have access to any parameter added to llama.cpp. > > @jukofyork I had a look at this, but it was getting messy, fast. > > What I think probably could work is: > > 1. Split out Ollama's custom server configuration from the model server parameters. > > 2. Do the same in llama.cpp in PR (if @ggerganov thinks this might be a good idea). > > 3. Then Ollama or any project that wants to use llama.cpp's model server parameters library can do so separate from their server configuration logic. > > > That's way out of my ball park though 😅 Yeah, but it would be good for the long-run: some of the parameters and their default settings that Ollama is using are very out of date compared to llama.cpp's server now and I can only see things getting worse and worse if nothing is done about it.

GiteaMirror commented

2026-04-22 06:14:38 -05:00

@sammcj commented on GitHub (May 4, 2024):

Anyway, https://github.com/ollama/ollama/pull/4120 provides the functionality for now, just waiting on someone to approve it...

@sammcj commented on GitHub (May 4, 2024): Anyway, https://github.com/ollama/ollama/pull/4120 provides the functionality for now, just waiting on someone to approve it...

GiteaMirror commented

2026-04-22 06:14:39 -05:00

@sammcj commented on GitHub (May 11, 2024):

#4120 is still sitting waiting for approval to be merged, I've been trying to keep it up and fix conflicts etc as other PRs are merged in.

@sammcj commented on GitHub (May 11, 2024): #4120 is still sitting waiting for approval to be merged, I've been trying to keep it up and fix conflicts etc as other PRs are merged in.

GiteaMirror commented

2026-04-22 06:14:41 -05:00

@enzomich commented on GitHub (Jul 18, 2024):

FA is also required for another important feature already present in Llama.cpp and hopefully coming to Ollama: KV cache quantization, with the two options -ctk and -ctv . In particular, it's required for V:
llama_new_context_with_model: V cache quantization requires flash_attn

@enzomich commented on GitHub (Jul 18, 2024): FA is also required for another important feature already present in Llama.cpp and hopefully coming to Ollama: KV cache quantization, with the two options -ctk and -ctv . In particular, it's required for V: `llama_new_context_with_model: V cache quantization requires flash_attn`

Sign in to join this conversation.

Branches Tags

main

hoyyeva/anthropic-local-image-path

dhiltgen/ci

dhiltgen/llama-runner

parth-remove-claude-desktop-launch

hoyyeva/anthropic-reference-images-path

parth-anthropic-reference-images-path

brucemacd/download-before-remove

hoyyeva/editor-config-repair

parth-mlx-decode-checkpoints

parth-launch-codex-app

hoyyeva/fix-codex-model-metadata-warning

hoyyeva/qwen

parth/hide-claude-desktop-till-release

hoyyeva/opencode-image-modality

parth-add-claude-code-autoinstall

release_v0.22.0

pdevine/manifest-list

codex/fix-codex-model-metadata-warning

pdevine/addressable-manifest

brucemacd/launch-fetch-reccomended

jmorganca/llama-compat

launch-copilot-cli

hoyyeva/opencode-thinking

release_v0.20.7

parth-auto-save-backup

parth-test

jmorganca/gemma4-audio-replacements

fix-manifest-digest-on-pull

hoyyeva/vscode-improve

brucemacd/install-server-wait

parth/update-claude-docs

brucemac/start-ap-install

pdevine/mlx-update

pdevine/qwen35_vision

drifkin/api-show-fallback

mintlify/image-generation-1773352582

hoyyeva/server-context-length-local-config

jmorganca/faster-reptition-penalties

jmorganca/convert-nemotron

parth-pi-thinking

pdevine/sampling-penalties

jmorganca/fix-create-quantization-memory

dongchen/resumable_transfer_fix

pdevine/sampling-cache-error

jessegross/mlx-usage

hoyyeva/openclaw-config

hoyyeva/app-html

pdevine/qwen3next

brucemacd/sign-sh-install

brucemacd/tui-update

brucemacd/usage-api

jmorganca/launch-empty

fix-app-dist-embed

mxyng/mlx-compile

mxyng/mlx-quant

mxyng/mlx-glm4.7

mxyng/mlx

brucemacd/simplify-model-picker

jmorganca/qwen3-concurrent

fix-glm-4.7-flash-mla-config

drifkin/qwen3-coder-opening-tag

brucemacd/usage-cli

fix-cuda12-fattn-shmem

ollama-imagegen-docs

parth/fix-multiline-inputs

brucemacd/config-docs

mxyng/model-files

mxyng/simple-execute

fix-imagegen-ollama-models

mxyng/async-upload

jmorganca/lazy-no-dtype-changes

imagegen-auto-detect-create

parth/decrease-concurrent-download-hf

fix-mlx-quantize-init

jmorganca/x-cleanup

usage

imagegen-readme

jmorganca/glm-image

mlx-gpu-cd

jmorganca/imagegen-modelfile

parth/agent-skills

parth/agent-allowlist

parth/signed-in-offline

parth/agents

parth/fix-context-chopping

improve-cloud-flow

parth/add-models-websearch

parth/prompt-renderer-mcp

jmorganca/native-settings

jmorganca/download-stream-hash

jmorganca/client2-rebased

brucemacd/oai-chat-req-multipart

jessegross/multi_chunk_reserve

grace/additional-omit-empty

grace/mistral-3-large

mxyng/tokenizer2

mxyng/tokenizer

jessegross/flash

hoyyeva/windows-nacked-app

mxyng/cleanup-attention

grace/deepseek-parser

hoyyeva/remember-unsent-prompt

parth/add-lfs-pointer-error-conversion

parth/olmo2-test2

hoyyeva/ollama-launchagent-plist

nicole/olmo-model

parth/olmo-test

mxyng/remove-embedded

parth/render-template

jmorganca/intellect-3

parth/remove-prealloc-linter

jmorganca/cmd-eval

nicole/nomic-embed-text-fix

mxyng/lint-2

hoyyeva/add-gemini-3-pro-preview

hoyyeva/load-model-list

mxyng/expand-path

mxyng/environ-2

hoyyeva/deeplink-json-encoding

parth/improve-tool-calling-tests

hoyyeva/conversation

hoyyeva/assistant-edit-response

hoyyeva/thinking

origin/brucemacd/invalid-char-i-err

parth/improve-tool-calling

jmorganca/required-omitempty

grace/qwen3-vl-tests

mxyng/iter-client

parth/docs-readme

nicole/embed-test

pdevine/integration-benchstat

parth/remove-generate-cmd

parth/add-toolcall-id

mxyng/server-tests

jmorganca/glm-4.6

jmorganca/gin-h-compat

drifkin/stable-tool-args

pdevine/qwen3-more-thinking

parth/add-websearch-client

nicole/websearch_local

jmorganca/qwen3-coder-updates

grace/deepseek-v3-migration-tests

mxyng/fix-create

jmorganca/cloud-errors

pdevine/parser-tidy

revert-12233-parth/simplify-entrypoints-runner

parth/enable-so-gpt-oss

brucemacd/qwen3vl

jmorganca/readme-simplify

parth/gpt-oss-structured-outputs

revert-12039-jmorganca/tools-braces

mxyng/embeddings

mxyng/gguf

mxyng/benchmark

mxyng/types-null

parth/move-parsing

mxyng/gemma2

jmorganca/docs

mxyng/16-bit

mxyng/create-stdin

pdevine/authorizedkeys

mxyng/quant

parth/opt-in-error-context-window

brucemacd/cache-models

brucemacd/runner-completion

jmorganca/llama-update-6

brucemacd/benchmark-list

brucemacd/partial-read-caps

parth/deepseek-r1-tools

mxyng/omit-array

parth/tool-prefix-temp

brucemacd/runner-test

jmorganca/qwen25vl

brucemacd/model-forward-test-ext

parth/python-function-parsing

jmorganca/cuda-compression-none

drifkin/num-parallel

drifkin/chat-truncation-fix

jmorganca/sync

parth/python-tools-calling

drifkin/array-head-count

brucemacd/create-no-loop

parth/server-enable-content-stream-with-tools

qwen25omni

mxyng/v3

brucemacd/ropeconfig

jmorganca/silence-tokenizer

parth/sample-so-test

parth/sampling-structured-outputs

brucemacd/doc-go-engine

parth/constrained-sampling-json

jmorganca/mistral-wip

brucemacd/mistral-small-convert

parth/sample-unmarshal-json-for-params

brucemacd/jomorganca/mistral

pdevine/bfloat16

jmorganca/mistral

brucemacd/mistral

pdevine/logging

parth/sample-correctness-fix

parth/sample-fix-sorting

jmorgan/sample-fix-sorting-extras

jmorganca/temp-0-images

brucemacd/parallel-embed-models

brucemacd/shim-grammar

jmorganca/fix-gguf-error

bmizerany/nameswork

jmorganca/faster-releases

bmizerany/validatenames

brucemacd/err-no-vocab

brucemacd/rope-config

brucemacd/err-hint

brucemacd/qwen2_5

brucemacd/logprobs

brucemacd/new_runner_graph_bench

progress-flicker

brucemacd/forward-test

brucemacd/go_qwen2

pdevine/gemma2

jmorganca/add-missing-symlink-eval

mxyng/next-debug

parth/set-context-size-openai

brucemacd/next-bpe-bench

brucemacd/next-bpe-test

brucemacd/new_runner_e2e

brucemacd/new_runner_qwen2

pdevine/convert-cohere2

brucemacd/convert-cli

parth/log-probs

mxyng/next-mlx

mxyng/cmd-history

parth/templating

parth/tokenize-detokenize

brucemacd/check-key-register

bmizerany/grammar

jmorganca/vendor-081b29bd

mxyng/func-checks

jmorganca/fix-null-format

parth/fix-default-to-warn-json

jmorganca/qwen2vl

jmorganca/no-concat

parth/cmd-cleanup-SO

brucemacd/check-key-register-structured-err

parth/openai-stream-usage

parth/fix-referencing-so

stream-tools-stop

jmorganca/degin-1

brucemacd/install-path-clean

brucemacd/push-name-validation

brucemacd/browser-key-register

jmorganca/openai-fix-first-message

jmorganca/fix-proxy

jessegross/sample

parth/disallow-streaming-tools

dhiltgen/remove_submodule

jmorganca/ga

jmorganca/mllama

pdevine/newlines

pdevine/geems-2b

jmorganca/llama-bump

mxyng/modelname-7

mxyng/gin-slog

mxyng/modelname-6

jyan/convert-prog

jyan/quant5

paligemma-support

pdevine/import-docs

jmorganca/openai-context

jyan/paligemma

jyan/p2

jyan/palitest

bmizerany/embedspeedup

jmorganca/llama-vit

brucemacd/allow-ollama

royh/ep-methods

royh/whisper

mxyng/api-models

mxyng/fix-memory

jyan/q4_4/8

jyan/ollama-v

royh/stream-tools

roy-embed-parallel

bmizerany/hrm

revert-5963-revert-5924-mxyng/llama3.1-rope

royh/embed-viz

jyan/local2

jyan/auth

jyan/local

jyan/parse-temp

jmorganca/template-mistral

jyan/reord-g

royh-openai-suffixdocs

royh-imgembed

royh-embed-parallel

jyan/quant4

royh-precision

jyan/progress

pdevine/fix-template

jyan/quant3

pdevine/ggla

mxyng/update-registry-domain

jmorganca/ggml-static

mxyng/create-context

jyan/v0.146

mxyng/layers-from-files

build_dist

bmizerany/noseek

royh-ls

royh-name

timeout

mxyng/server-timestamp

bmizerany/nosillyggufslurps

royh-params

jmorganca/llama-cpp-7c26775

royh-openai-delete

royh-show-rigid

jmorganca/enable-fa

jmorganca/no-error-template

jyan/format

royh-testdelete

bmizerany/fastverify

language_support

pdevine/ps-glitches

brucemacd/tokenize

bruce/iq-quants

bmizerany/filepathwithcoloninhost

mxyng/split-bin

bmizerany/client-registry

jmorganca/if-none-match

native

jmorganca/native

jmorganca/batch-embeddings

jmorganca/initcmake

jmorganca/mm

pdevine/showggmlinfo

modenameenforcealphanum

bmizerany/modenameenforcealphanum

jmorganca/done-reason

jmorganca/llama-cpp-8960fe8

ollama.com

bmizerany/filepathnobuild

bmizerany/types/model/defaultfix

rmdisplaylong

nogogen

bmizerany/x

modelfile-readme

bmizerany/replacecolon

jmorganca/limit

jmorganca/execstack

jmorganca/replace-assets

mxyng/tune-concurrency

jmorganca/testing

whitespace-detection

jmorganca/options

upgrade-all

scratch

cuda-search

mattw/airenamer

mattw/allmodelsonhuggingface

mattw/quantcontext

mattw/whatneedstorun

brucemacd/llama-mem-calc

mattw/faq-context

mattw/communitylinks

mattw/noprune

mattw/python-functioncalling

rename

mxyng/install

pulse

remove-first

editor

mattw/selfqueryingretrieval

cgo

mattw/howtoquant

api

matt/streamingapi

format-config

mxyng/extra-args

shell

update-nous-hermes

cp-model

upload-progress

fix-unknown-model

fix-model-names

delete-fix

insecure-registry

ls

deletemodels

progressbar

readme-updates

license-layers

skip-list

list-models

modelpath

matt/examplemodelfiles

distribution

go-opts

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/ollama#28273