[GH-ISSUE #14049] Qwen3-Coder-Next (Local Model Request) #55694

New Issue

GiteaMirror · 2026-04-29T09:35:33-05:00

GiteaMirror commented

2026-04-29 09:35:33 -05:00

Originally created by @asitwere on GitHub (Feb 3, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/14049

HF: https://huggingface.co/collections/Qwen/qwen3-coder-next

Qwen3-Coder-Next is an 80B MoE model (3B active parameters) with 256K context for fast agentic coding and local use. It delivers performance comparable to LLMs with 10-20× more active parameters and its arch enables fast inference. It excels at long-horizon reasoning & complex tool use.

Works on 46GB RAM/VRAM/unified memory (85GB for 8-bit).

Run local via Dynamic GGUFs (including MXFP4).

Originally created by @asitwere on GitHub (Feb 3, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/14049 HF: https://huggingface.co/collections/Qwen/qwen3-coder-next Qwen3-Coder-Next is an 80B MoE model (3B active parameters) with 256K context for fast agentic coding and local use. It delivers performance comparable to LLMs with 10-20× more active parameters and its arch enables fast inference. It excels at long-horizon reasoning & complex tool use. Works on 46GB RAM/VRAM/unified memory (85GB for 8-bit). Run local via Dynamic GGUFs (including MXFP4).

GiteaMirror closed this issue

2026-04-29 09:35:33 -05:00

GiteaMirror commented

2026-04-29 09:35:34 -05:00

@rick-github commented on GitHub (Feb 3, 2026):

This is qwen3next architecture and so can be imported from the provided GGUFs. Due to the size the GGUFs are split so merging with llama-gguf-split from llama.cpp is required.

@rick-github commented on GitHub (Feb 3, 2026): This is qwen3next architecture and so can be [imported](https://github.com/ollama/ollama/blob/main/docs/import.mdx) from the provided [GGUFs](https://huggingface.co/Qwen/Qwen3-Coder-Next-GGUF). Due to the size the GGUFs are split so merging with `llama-gguf-split` from `llama.cpp` is required.

GiteaMirror commented

2026-04-29 09:35:34 -05:00

@Orbiter commented on GitHub (Feb 3, 2026):

I tried both hf.co/unsloth/Qwen3-Coder-Next-GGUF:Q4_K_M and hf.co/lmstudio-community/Qwen3-Coder-Next-GGUF:Q4_K_M which are single-file gguf and both returned a error loading model: missing tensor 'blk.0.ssm_in.weight'

% ollama --version
ollama version is 0.15.4
% ollama run hf.co/unsloth/Qwen3-Coder-Next-GGUF:Q4_K_M
pulling manifest
pulling e83904064313: 100% ▕██████████████████████████████████████████████████████▏  48 GB
pulling 62fbfd9ed093: 100% ▕██████████████████████████████████████████████████████▏  182 B
pulling 6db07cd2f395: 100% ▕██████████████████████████████████████████████████████▏   57 B
pulling bb61354e916f: 100% ▕██████████████████████████████████████████████████████▏  564 B
verifying sha256 digest
writing manifest
success
Error: 500 Internal Server Error: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight'

% ollama run hf.co/lmstudio-community/Qwen3-Coder-Next-GGUF:Q4_K_M
pulling manifest
pulling b76696cae4f3: 100% ▕██████████████████████████████████████████████████████▏  48 GB
pulling 62fbfd9ed093: 100% ▕██████████████████████████████████████████████████████▏  182 B
pulling 6db07cd2f395: 100% ▕██████████████████████████████████████████████████████▏   57 B
pulling ac42a302f10f: 100% ▕██████████████████████████████████████████████████████▏  564 B
verifying sha256 digest
writing manifest
success
Error: 500 Internal Server Error: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight'

@Orbiter commented on GitHub (Feb 3, 2026): I tried both `hf.co/unsloth/Qwen3-Coder-Next-GGUF:Q4_K_M` and `hf.co/lmstudio-community/Qwen3-Coder-Next-GGUF:Q4_K_M` which are single-file gguf and both returned a `error loading model: missing tensor 'blk.0.ssm_in.weight'` ``` % ollama --version ollama version is 0.15.4 % ollama run hf.co/unsloth/Qwen3-Coder-Next-GGUF:Q4_K_M pulling manifest pulling e83904064313: 100% ▕██████████████████████████████████████████████████████▏ 48 GB pulling 62fbfd9ed093: 100% ▕██████████████████████████████████████████████████████▏ 182 B pulling 6db07cd2f395: 100% ▕██████████████████████████████████████████████████████▏ 57 B pulling bb61354e916f: 100% ▕██████████████████████████████████████████████████████▏ 564 B verifying sha256 digest writing manifest success Error: 500 Internal Server Error: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight' % ollama run hf.co/lmstudio-community/Qwen3-Coder-Next-GGUF:Q4_K_M pulling manifest pulling b76696cae4f3: 100% ▕██████████████████████████████████████████████████████▏ 48 GB pulling 62fbfd9ed093: 100% ▕██████████████████████████████████████████████████████▏ 182 B pulling 6db07cd2f395: 100% ▕██████████████████████████████████████████████████████▏ 57 B pulling ac42a302f10f: 100% ▕██████████████████████████████████████████████████████▏ 564 B verifying sha256 digest writing manifest success Error: 500 Internal Server Error: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight' ```

GiteaMirror commented

2026-04-29 09:35:34 -05:00

@rick-github commented on GitHub (Feb 3, 2026):

I used the GGUFs that Qwen supplied and had no problems:

$ ollama run frob/qwen3-coder-next:80b-a3b-q4_K_M
>>> write a golang program that prints 'hello world'
Here is a simple Go program that prints "hello world":

```go
package main

import "fmt"

func main() {
    fmt.Println("hello world")
}
```

To run this program:

1. Save the code in a file named `main.go`
2. Open a terminal in the same directory
3. Run: `go run main.go`

You should see the output: `hello world`

*(Note: By convention, Go developers typically capitalize the "W" in "Hello World", but I've kept it lowercase exactly as requested.)*

@rick-github commented on GitHub (Feb 3, 2026): I used the GGUFs that Qwen supplied and had no problems: ````console $ ollama run frob/qwen3-coder-next:80b-a3b-q4_K_M >>> write a golang program that prints 'hello world' Here is a simple Go program that prints "hello world": ```go package main import "fmt" func main() { fmt.Println("hello world") } ``` To run this program: 1. Save the code in a file named `main.go` 2. Open a terminal in the same directory 3. Run: `go run main.go` You should see the output: `hello world` *(Note: By convention, Go developers typically capitalize the "W" in "Hello World", but I've kept it lowercase exactly as requested.)* ````

GiteaMirror commented

2026-04-29 09:35:35 -05:00

@Orbiter commented on GitHub (Feb 3, 2026):

great! I can confirm that ollama run frob/qwen3-coder-next:80b-a3b-q4_K_M is working. I will now run benchmarks.

@Orbiter commented on GitHub (Feb 3, 2026): great! I can confirm that `ollama run frob/qwen3-coder-next:80b-a3b-q4_K_M` is working. I will now run benchmarks.

GiteaMirror commented

2026-04-29 09:35:35 -05:00

@snapo commented on GitHub (Feb 3, 2026):

I tried both hf.co/unsloth/Qwen3-Coder-Next-GGUF:Q4_K_M and hf.co/lmstudio-community/Qwen3-Coder-Next-GGUF:Q4_K_M which are single-file gguf and both returned a error loading model: missing tensor 'blk.0.ssm_in.weight'

% ollama --version
ollama version is 0.15.4
% ollama run hf.co/unsloth/Qwen3-Coder-Next-GGUF:Q4_K_M
pulling manifest
pulling e83904064313: 100% ▕██████████████████████████████████████████████████████▏  48 GB
pulling 62fbfd9ed093: 100% ▕██████████████████████████████████████████████████████▏  182 B
pulling 6db07cd2f395: 100% ▕██████████████████████████████████████████████████████▏   57 B
pulling bb61354e916f: 100% ▕██████████████████████████████████████████████████████▏  564 B
verifying sha256 digest
writing manifest
success
Error: 500 Internal Server Error: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight'

% ollama run hf.co/lmstudio-community/Qwen3-Coder-Next-GGUF:Q4_K_M
pulling manifest
pulling b76696cae4f3: 100% ▕██████████████████████████████████████████████████████▏  48 GB
pulling 62fbfd9ed093: 100% ▕██████████████████████████████████████████████████████▏  182 B
pulling 6db07cd2f395: 100% ▕██████████████████████████████████████████████████████▏   57 B
pulling ac42a302f10f: 100% ▕██████████████████████████████████████████████████████▏  564 B
verifying sha256 digest
writing manifest
success
Error: 500 Internal Server Error: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight'

i have exactly the same issue with the unsloth quants

@snapo commented on GitHub (Feb 3, 2026): > I tried both `hf.co/unsloth/Qwen3-Coder-Next-GGUF:Q4_K_M` and `hf.co/lmstudio-community/Qwen3-Coder-Next-GGUF:Q4_K_M` which are single-file gguf and both returned a `error loading model: missing tensor 'blk.0.ssm_in.weight'` > > ``` > % ollama --version > ollama version is 0.15.4 > % ollama run hf.co/unsloth/Qwen3-Coder-Next-GGUF:Q4_K_M > pulling manifest > pulling e83904064313: 100% ▕██████████████████████████████████████████████████████▏ 48 GB > pulling 62fbfd9ed093: 100% ▕██████████████████████████████████████████████████████▏ 182 B > pulling 6db07cd2f395: 100% ▕██████████████████████████████████████████████████████▏ 57 B > pulling bb61354e916f: 100% ▕██████████████████████████████████████████████████████▏ 564 B > verifying sha256 digest > writing manifest > success > Error: 500 Internal Server Error: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight' > > % ollama run hf.co/lmstudio-community/Qwen3-Coder-Next-GGUF:Q4_K_M > pulling manifest > pulling b76696cae4f3: 100% ▕██████████████████████████████████████████████████████▏ 48 GB > pulling 62fbfd9ed093: 100% ▕██████████████████████████████████████████████████████▏ 182 B > pulling 6db07cd2f395: 100% ▕██████████████████████████████████████████████████████▏ 57 B > pulling ac42a302f10f: 100% ▕██████████████████████████████████████████████████████▏ 564 B > verifying sha256 digest > writing manifest > success > Error: 500 Internal Server Error: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight' > ``` i have exactly the same issue with the unsloth quants

GiteaMirror commented

2026-04-29 09:35:35 -05:00

@Orbiter commented on GitHub (Feb 3, 2026):

I will now run benchmarks.

I tested this now with 200 problems from the project euler problem set using python coding only and it performs 10% better than Qwen3-Next and overall better than any other free/open model I tested so far. I continue to test with other languages.

@Orbiter commented on GitHub (Feb 3, 2026): > I will now run benchmarks. I tested this now with 200 problems from the project euler problem set using python coding only and it performs 10% better than Qwen3-Next and overall better than any other free/open model I tested so far. I continue to test with other languages.

GiteaMirror commented

2026-04-29 09:35:36 -05:00

@rick-github commented on GitHub (Feb 3, 2026):

Do you have any comparisons with qwen3-coder:{30b,480b}?

@rick-github commented on GitHub (Feb 3, 2026): Do you have any comparisons with qwen3-coder:{30b,480b}?

GiteaMirror commented

2026-04-29 09:35:36 -05:00

@shuasimodo commented on GitHub (Feb 4, 2026):

I'm pretty sure the 500 error issue with qwen3-next models from other (non ollama library) sources like unsloth are due to llama.cpp updates from around late december. When using llama.cpp to quantize the next series models, it doesn't work with ollama. Similar issues with glm-4.7-flash, but not really relevant to my point.

I've tried to quantize my own qwen3-next models using llama.cpp and get the same error as when I download quants from any of the sources mentioned. Except ollama models work fine.

On december 3rd, ollama updated with qwen3-next support in v0.13.2 - Ollama models, and qwen gguf models work.

On december 16, llama.cpp made some improvements for qwen3-next.

And then there was one more update after that 3 weeks ago.

Around early january is when unsloth updated their qwen3-next models and the dates correlate with qwen3-next updates in llama.cpp. (models we're updated 23 days ago which lines up with the latest llama.cpp update for qwen3-next)

These models no longer work in ollama.

Around the same time as the latest unsloth update, noctrex created an mxfp4 quant from that model, and specifically mentioned that you needed to download the latest llama.cpp to run it.

Before these new updated releases, the old models worked well in ollama. Both the unsloth quants, and the old noctrex mxfp4. After these releases, their qwen3-next models no longer work in ollama and all give the same error:

500: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight'
llama_model_load_from_file_impl: failed to load model

I also could be wrong, I'm just trying to share my experience and issues that seem to be relevant to that update.

@shuasimodo commented on GitHub (Feb 4, 2026): I'm pretty sure the 500 error issue with qwen3-next models from other (non ollama library) sources like unsloth are due to [llama.cpp updates](https://github.com/ggml-org/llama.cpp/releases?q=qwen3-next&expanded=true) from around late december. When using llama.cpp to quantize the next series models, it doesn't work with ollama. Similar issues with glm-4.7-flash, but not really relevant to my point. I've tried to quantize my own qwen3-next models using llama.cpp and get the same error as when I download quants from any of the sources mentioned. Except ollama models work fine. On december 3rd, ollama updated with qwen3-next support in [ v0.13.2](https://github.com/ollama/ollama/releases/tag/v0.13.2) - Ollama models, and qwen gguf models work. On december 16, llama.cpp made some [improvements for qwen3-next](https://github.com/ggml-org/llama.cpp/releases/tag/b7432). And then there was one more update after that 3 weeks ago. Around early january is when unsloth updated their [qwen3-next models](https://huggingface.co/unsloth/Qwen3-Next-80B-A3B-Thinking-GGUF/discussions/2) and the dates correlate with qwen3-next updates in llama.cpp. ([models we're updated 23 days ago](https://huggingface.co/unsloth/Qwen3-Next-80B-A3B-Thinking-GGUF/tree/main) which lines up with the latest llama.cpp update for qwen3-next) - These models no longer work in ollama. Around the same time as the latest unsloth update, [noctrex created an mxfp4 quant from that model](https://huggingface.co/noctrex/Qwen3-Next-80B-A3B-Thinking-MXFP4_MOE-GGUF), and specifically mentioned that you needed to download the latest llama.cpp to run it. Before these new updated releases, the old models worked well in ollama. Both the unsloth quants, and the old noctrex mxfp4. After these releases, their qwen3-next models no longer work in ollama and all give the same error: ``` 500: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight' llama_model_load_from_file_impl: failed to load model ``` I also could be wrong, I'm just trying to share my experience and issues that seem to be relevant to that update.

GiteaMirror commented

2026-04-29 09:35:37 -05:00

@rick-github commented on GitHub (Feb 4, 2026):

0.15.5-rc1 contains a vendor sync which pulls in the latest llama.cpp code, and the unsloth and lmstudio-community models work with that version. There are some problems (#14044, #14045) with 0.15.5-rc1 that relate to the vendor sync, so it may not be in 0.15.5, but it will eventually be merged.

$ ollama -v
ollama version is 0.15.5-rc1
$ ollama run hf.co/lmstudio-community/Qwen3-Coder-Next-GGUF:Q4_K_M hello
Hello! How can I help you today? 😊

@rick-github commented on GitHub (Feb 4, 2026): 0.15.5-rc1 contains a vendor sync which pulls in the latest llama.cpp code, and the unsloth and lmstudio-community models work with that version. There are some problems (#14044, #14045) with 0.15.5-rc1 that relate to the vendor sync, so it may not be in 0.15.5, but it will eventually be merged. ```console $ ollama -v ollama version is 0.15.5-rc1 $ ollama run hf.co/lmstudio-community/Qwen3-Coder-Next-GGUF:Q4_K_M hello Hello! How can I help you today? 😊 ```

GiteaMirror commented

2026-04-29 09:35:37 -05:00

@Orbiter commented on GitHub (Feb 4, 2026):

Do you have any comparisons with qwen3-coder:{30b,480b}?

frob/qwen3-coder-next:80b-a3b-q4_K_M is now leading in the project euler benchmark for instruct models, it is about 4% above qwen3-next (I do 4 bit quantization only) and it is 193% above qwen3-coder-30b (scores almost 3 times better!). For qwen3-coder-480b I have only a older benchmark where qwen3.coder-480b is comparable with qwen3:235b-a22b-instruct-2507; here frob/qwen3-coder-next:80b-a3b-q4_K_M is 22% above qwen3:235b-a22b-instruct-2507.

The main point here for me is if qwen3-coder-next:80b is good at agentic tasks and so I did a tiny manual test with opencode where it worked pretty well at first glance. But I don't want to do topic-stealing here, fortunately there is a fix in 0.15.5 now, thank you all for this great work!

@Orbiter commented on GitHub (Feb 4, 2026): > Do you have any comparisons with qwen3-coder:{30b,480b}? frob/qwen3-coder-next:80b-a3b-q4_K_M is now leading in the [project euler benchmark](https://github.com/Orbiter/project-euler-llm-benchmark) for instruct models, it is about 4% above qwen3-next (I do 4 bit quantization only) and it is 193% above qwen3-coder-30b (scores almost 3 times better!). For qwen3-coder-480b I have only a older benchmark where qwen3.coder-480b is comparable with qwen3:235b-a22b-instruct-2507; here frob/qwen3-coder-next:80b-a3b-q4_K_M is 22% above qwen3:235b-a22b-instruct-2507. The main point here for me is if qwen3-coder-next:80b is good at agentic tasks and so I did a tiny manual test with opencode where it worked pretty well at first glance. But I don't want to do topic-stealing here, fortunately there is a fix in 0.15.5 now, thank you all for this great work!

GiteaMirror commented

2026-04-29 09:35:37 -05:00

@jmorganca commented on GitHub (Feb 4, 2026):

Hi all, it's available in the library here: https://ollama.com/library/qwen3-coder-next

Note: you'll need the 0.15.5-rc2 or later pre-release: http://github.com/ollama/ollama/releases/

@jmorganca commented on GitHub (Feb 4, 2026): Hi all, it's available in the library here: https://ollama.com/library/qwen3-coder-next Note: you'll need the 0.15.5-rc2 or later pre-release: http://github.com/ollama/ollama/releases/

GiteaMirror commented

2026-04-29 09:35:38 -05:00

@ppulwey commented on GitHub (Feb 4, 2026):

After pulling the model with ollama 0.15.5-rc2 I get this

ollama run qwen3-coder-next:q4_K_M
pulling manifest
pulling 30e51a7cb1cf: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████▏  51 GB
pulling 7339fa418c9a: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████▏  11 KB
pulling d3cd8304aca0: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████▏   42 B
pulling 5d55cac51f30: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████▏  490 B
verifying sha256 digest
writing manifest
success
>>> hi
HelloError: an error was encountered while running the model: error:tensor ' (view)' buffer is nil

Is this an error with the model or with my system?

@ppulwey commented on GitHub (Feb 4, 2026): After pulling the model with ollama 0.15.5-rc2 I get this ``` ollama run qwen3-coder-next:q4_K_M pulling manifest pulling 30e51a7cb1cf: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 51 GB pulling 7339fa418c9a: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 11 KB pulling d3cd8304aca0: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 42 B pulling 5d55cac51f30: 100% ▕██████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 490 B verifying sha256 digest writing manifest success >>> hi HelloError: an error was encountered while running the model: error:tensor ' (view)' buffer is nil ``` Is this an error with the model or with my system?

GiteaMirror commented

2026-04-29 09:35:38 -05:00

@rick-github commented on GitHub (Feb 4, 2026):

Server logs may aid in debugging.

@rick-github commented on GitHub (Feb 4, 2026): [Server logs](https://docs.ollama.com/troubleshooting) may aid in debugging.

GiteaMirror commented

2026-04-29 09:35:39 -05:00

@c0008 commented on GitHub (Feb 4, 2026):

Qwen3-Coder-Next is also not working for me with 0.15.5-rc2
I tried different quants:

hf.co/unsloth/Qwen3-Coder-Next-GGUF:Q4_K_XL:

Error: 500 Internal Server Error: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight'
llama_model_load_from_file_impl: failed to load model

hf.co/unsloth/Qwen3-Coder-Next-GGUF:MXFP4_MOE:

Error: 500 Internal Server Error: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight'

qwen3-coder-next:latest:
Here the model loads but it fails at token generation.

Error: 500 Internal Server Error: model runner has unexpectedly stopped, this may be due to resource limitatio
ns or an internal error, check ollama server logs for details

The older Qwen3-Next model is loading and working fine though.

@c0008 commented on GitHub (Feb 4, 2026): Qwen3-Coder-Next is also not working for me with 0.15.5-rc2 I tried different quants: hf.co/unsloth/Qwen3-Coder-Next-GGUF:Q4_K_XL: ``` Error: 500 Internal Server Error: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight' llama_model_load_from_file_impl: failed to load model ``` hf.co/unsloth/Qwen3-Coder-Next-GGUF:MXFP4_MOE: ``` Error: 500 Internal Server Error: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight' ``` qwen3-coder-next:latest: Here the model loads but it fails at token generation. ``` Error: 500 Internal Server Error: model runner has unexpectedly stopped, this may be due to resource limitatio ns or an internal error, check ollama server logs for details ``` The older Qwen3-Next model is loading and working fine though.

GiteaMirror commented

2026-04-29 09:35:39 -05:00

@rick-github commented on GitHub (Feb 4, 2026):

@ppulwey #14076
@c0008 The vendor sync was rolled back so unsloth and lmstudio models will not load. Logs for the failure of qwen3-coder-next:latest would aid in debugging.

@rick-github commented on GitHub (Feb 4, 2026): @ppulwey #14076 @c0008 The vendor sync was rolled back so unsloth and lmstudio models will not load. Logs for the failure of qwen3-coder-next:latest would aid in debugging.

GiteaMirror commented

2026-04-29 09:35:39 -05:00

@lukek commented on GitHub (Feb 5, 2026):

I only have 8GB VRAM and 64GB RAM so this is more for me to try the model, it's not fast enough to use effectively.

hf.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M did not work for me and i got this error (on ollama version is 0.15.5-rc3):
Error: 500 Internal Server Error: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight'

Frob worked perfectly:
ollama run frob/qwen3-coder-next:80b-a3b-q4_K_M

Link: https://ollama.com/frob/qwen3-coder-next

@lukek commented on GitHub (Feb 5, 2026): I only have 8GB VRAM and 64GB RAM so this is more for me to try the model, it's not fast enough to use effectively. hf.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF:Q4_K_M did not work for me and i got this error (on ollama version is 0.15.5-rc3): Error: 500 Internal Server Error: llama runner process has terminated: error loading model: missing tensor 'blk.0.ssm_in.weight' Frob worked perfectly: ollama run frob/qwen3-coder-next:80b-a3b-q4_K_M Link: https://ollama.com/frob/qwen3-coder-next

GiteaMirror commented

2026-04-29 09:35:40 -05:00

@rick-github commented on GitHub (Feb 5, 2026):

The unsloth and lmstudio models (and probably most on HF) were quantized with llama.cpp and need llama.cpp support to run in ollama. That was briefly true in -rc1 which had a vendor sync, but was rolled back in -rc2 because of performance issues with glm-4.7-flash. The frob model was imported from the GGUFs supplied by Qwen, so does not need llama.cpp support to run in ollama.

@rick-github commented on GitHub (Feb 5, 2026): The unsloth and lmstudio models (and probably most on HF) were quantized with llama.cpp and need llama.cpp support to run in ollama. That was briefly true in -rc1 which had a vendor sync, but was rolled back in -rc2 because of performance issues with glm-4.7-flash. The frob model was imported from the GGUFs supplied by Qwen, so does not need llama.cpp support to run in ollama.

GiteaMirror commented

2026-04-29 09:35:41 -05:00

@snapo commented on GitHub (Feb 6, 2026):

The unsloth and lmstudio models (and probably most on HF) were quantized with llama.cpp and need llama.cpp support to run in ollama. That was briefly true in -rc1 which had a vendor sync, but was rolled back in -rc2 because of performance issues with glm-4.7-flash. The frob model was imported from the GGUFs supplied by Qwen, so does not need llama.cpp support to run in ollama.

does that mean it wont be fixed?

@snapo commented on GitHub (Feb 6, 2026): > The unsloth and lmstudio models (and probably most on HF) were quantized with llama.cpp and need llama.cpp support to run in ollama. That was briefly true in -rc1 which had a vendor sync, but was rolled back in -rc2 because of performance issues with glm-4.7-flash. The frob model was imported from the GGUFs supplied by Qwen, so does not need llama.cpp support to run in ollama. does that mean it wont be fixed?

GiteaMirror commented

2026-04-29 09:35:44 -05:00

@Orbiter commented on GitHub (Feb 6, 2026):

When I compare the frob/non-frob qwen3-coder-next models: both work, but the new qwen3-coder-next:Q4_K_M is 3GB larger than the frob/qwen3-coder-next:80b-a3b-q4_K_M model.

% ollama ls
NAME                                          ID              SIZE      MODIFIED
qwen3-coder-next:Q4_K_M                       ca06e9e4087c    51 GB     3 hours ago
frob/qwen3-coder-next:80b-a3b-q4_K_M          a4a1e21bdeb6    48 GB     2 days ago

Is that correct? was there something missing in the frob model?

@Orbiter commented on GitHub (Feb 6, 2026): When I compare the frob/non-frob qwen3-coder-next models: both work, but the new `qwen3-coder-next:Q4_K_M` is 3GB larger than the `frob/qwen3-coder-next:80b-a3b-q4_K_M` model. ``` % ollama ls NAME ID SIZE MODIFIED qwen3-coder-next:Q4_K_M ca06e9e4087c 51 GB 3 hours ago frob/qwen3-coder-next:80b-a3b-q4_K_M a4a1e21bdeb6 48 GB 2 days ago ``` Is that correct? was there something missing in the frob model?

GiteaMirror commented

2026-04-29 09:35:46 -05:00

@rick-github commented on GitHub (Feb 6, 2026):

does that mean it wont be fixed?

It means that the unsloth and lmstudio models will be supported at the next vendor sync.

was there something missing in the frob model?

The frob model is created from the GGUFs supplied by Qwen, the model authors, so it's unlikely to be missing something. Comparing the tensors, it looks ollama has chosen to break some tensors up in some layers and use Q6_K instead of Q4_K in others. Whether or not this makes a qualitative difference in model performance is unknown. If they both work, choosing one over the other depends on the execution environment, eg frob/qwen3-coder-next:80b-a3b-q4_K_M will run faster on GPUs with 48G VRAM than qwen3-coder-next:Q4_K_M.

@rick-github commented on GitHub (Feb 6, 2026): > does that mean it wont be fixed? It means that the unsloth and lmstudio models will be supported at the next vendor sync. > was there something missing in the frob model? The frob model is created from the GGUFs supplied by Qwen, the model authors, so it's unlikely to be missing something. Comparing the tensors, it looks ollama has chosen to break some tensors up in some layers and use Q6_K instead of Q4_K in others. Whether or not this makes a qualitative difference in model performance is unknown. If they both work, choosing one over the other depends on the execution environment, eg frob/qwen3-coder-next:80b-a3b-q4_K_M will run faster on GPUs with 48G VRAM than qwen3-coder-next:Q4_K_M.

GiteaMirror commented

2026-04-29 09:35:46 -05:00

@snapo commented on GitHub (Feb 6, 2026):

any idea "when" this "vendor sync" will happen? (still dont understand what it means, always tought ollama just uses llama.cpp in the background)

@snapo commented on GitHub (Feb 6, 2026): any idea "when" this "vendor sync" will happen? (still dont understand what it means, always tought ollama just uses llama.cpp in the background)

GiteaMirror commented

2026-04-29 09:35:47 -05:00

@rick-github commented on GitHub (Feb 6, 2026):

The vendor sync is when the bits of the llama.cpp repo that ollama uses gets merged into the ollama repo. It's not just a matter of copying the files over and committing, checks needs to be done that the code is properly integrated. For example, the most recent vendor sync in 0.15.5-rc1 adversely affected the performance of glm-4.7-flash, so it was reverted to allow the release of 0.15.5. The developers will now start another branch for the vendor sync, testing the code and making changes where necessary. When the branch presents no issues, it will be merged into main.

The problem with glm-4.7-flash is being investigated in #14045 and a fix has been proposed.

@rick-github commented on GitHub (Feb 6, 2026): The vendor sync is when the bits of the llama.cpp repo that ollama uses gets merged into the ollama repo. It's not just a matter of copying the files over and committing, checks needs to be done that the code is properly integrated. For example, the most recent vendor sync in 0.15.5-rc1 adversely affected the performance of glm-4.7-flash, so it was reverted to allow the release of 0.15.5. The developers will now start another branch for the vendor sync, testing the code and making changes where necessary. When the branch presents no issues, it will be merged into main. The problem with glm-4.7-flash is being investigated in #14045 and a fix has been proposed.

Sign in to join this conversation.

Branches Tags

main

dhiltgen/ci

parth-launch-plan-gating

hoyyeva/anthropic-reference-images-path

parth-anthropic-reference-images-path

brucemacd/download-before-remove

hoyyeva/editor-config-repair

parth-mlx-decode-checkpoints

parth-launch-codex-app

hoyyeva/fix-codex-model-metadata-warning

hoyyeva/qwen

parth/hide-claude-desktop-till-release

hoyyeva/opencode-image-modality

parth-add-claude-code-autoinstall

release_v0.22.0

pdevine/manifest-list

codex/fix-codex-model-metadata-warning

pdevine/addressable-manifest

brucemacd/launch-fetch-reccomended

jmorganca/llama-compat

launch-copilot-cli

hoyyeva/opencode-thinking

release_v0.20.7

parth-auto-save-backup

parth-test

jmorganca/gemma4-audio-replacements

fix-manifest-digest-on-pull

hoyyeva/vscode-improve

brucemacd/install-server-wait

parth/update-claude-docs

brucemac/start-ap-install

pdevine/mlx-update

pdevine/qwen35_vision

drifkin/api-show-fallback

mintlify/image-generation-1773352582

hoyyeva/server-context-length-local-config

jmorganca/faster-reptition-penalties

jmorganca/convert-nemotron

parth-pi-thinking

pdevine/sampling-penalties

jmorganca/fix-create-quantization-memory

dongchen/resumable_transfer_fix

pdevine/sampling-cache-error

jessegross/mlx-usage

hoyyeva/openclaw-config

hoyyeva/app-html

pdevine/qwen3next

brucemacd/sign-sh-install

brucemacd/tui-update

brucemacd/usage-api

jmorganca/launch-empty

fix-app-dist-embed

mxyng/mlx-compile

mxyng/mlx-quant

mxyng/mlx-glm4.7

mxyng/mlx

brucemacd/simplify-model-picker

jmorganca/qwen3-concurrent

fix-glm-4.7-flash-mla-config

drifkin/qwen3-coder-opening-tag

brucemacd/usage-cli

fix-cuda12-fattn-shmem

ollama-imagegen-docs

parth/fix-multiline-inputs

brucemacd/config-docs

mxyng/model-files

mxyng/simple-execute

fix-imagegen-ollama-models

mxyng/async-upload

jmorganca/lazy-no-dtype-changes

imagegen-auto-detect-create

parth/decrease-concurrent-download-hf

fix-mlx-quantize-init

jmorganca/x-cleanup

usage

imagegen-readme

jmorganca/glm-image

mlx-gpu-cd

jmorganca/imagegen-modelfile

parth/agent-skills

parth/agent-allowlist

parth/signed-in-offline

parth/agents

parth/fix-context-chopping

improve-cloud-flow

parth/add-models-websearch

parth/prompt-renderer-mcp

jmorganca/native-settings

jmorganca/download-stream-hash

jmorganca/client2-rebased

brucemacd/oai-chat-req-multipart

jessegross/multi_chunk_reserve

grace/additional-omit-empty

grace/mistral-3-large

mxyng/tokenizer2

mxyng/tokenizer

jessegross/flash

hoyyeva/windows-nacked-app

mxyng/cleanup-attention

grace/deepseek-parser

hoyyeva/remember-unsent-prompt

parth/add-lfs-pointer-error-conversion

parth/olmo2-test2

hoyyeva/ollama-launchagent-plist

nicole/olmo-model

parth/olmo-test

mxyng/remove-embedded

parth/render-template

jmorganca/intellect-3

parth/remove-prealloc-linter

jmorganca/cmd-eval

nicole/nomic-embed-text-fix

mxyng/lint-2

hoyyeva/add-gemini-3-pro-preview

hoyyeva/load-model-list

mxyng/expand-path

mxyng/environ-2

hoyyeva/deeplink-json-encoding

parth/improve-tool-calling-tests

hoyyeva/conversation

hoyyeva/assistant-edit-response

hoyyeva/thinking

origin/brucemacd/invalid-char-i-err

parth/improve-tool-calling

jmorganca/required-omitempty

grace/qwen3-vl-tests

mxyng/iter-client

parth/docs-readme

nicole/embed-test

pdevine/integration-benchstat

parth/remove-generate-cmd

parth/add-toolcall-id

mxyng/server-tests

jmorganca/glm-4.6

jmorganca/gin-h-compat

drifkin/stable-tool-args

pdevine/qwen3-more-thinking

parth/add-websearch-client

nicole/websearch_local

jmorganca/qwen3-coder-updates

grace/deepseek-v3-migration-tests

mxyng/fix-create

jmorganca/cloud-errors

pdevine/parser-tidy

revert-12233-parth/simplify-entrypoints-runner

parth/enable-so-gpt-oss

brucemacd/qwen3vl

jmorganca/readme-simplify

parth/gpt-oss-structured-outputs

revert-12039-jmorganca/tools-braces

mxyng/embeddings

mxyng/gguf

mxyng/benchmark

mxyng/types-null

parth/move-parsing

mxyng/gemma2

jmorganca/docs

mxyng/16-bit

mxyng/create-stdin

pdevine/authorizedkeys

mxyng/quant

parth/opt-in-error-context-window

brucemacd/cache-models

brucemacd/runner-completion

jmorganca/llama-update-6

brucemacd/benchmark-list

brucemacd/partial-read-caps

parth/deepseek-r1-tools

mxyng/omit-array

parth/tool-prefix-temp

brucemacd/runner-test

jmorganca/qwen25vl

brucemacd/model-forward-test-ext

parth/python-function-parsing

jmorganca/cuda-compression-none

drifkin/num-parallel

drifkin/chat-truncation-fix

jmorganca/sync

parth/python-tools-calling

drifkin/array-head-count

brucemacd/create-no-loop

parth/server-enable-content-stream-with-tools

qwen25omni

mxyng/v3

brucemacd/ropeconfig

jmorganca/silence-tokenizer

parth/sample-so-test

parth/sampling-structured-outputs

brucemacd/doc-go-engine

parth/constrained-sampling-json

jmorganca/mistral-wip

brucemacd/mistral-small-convert

parth/sample-unmarshal-json-for-params

brucemacd/jomorganca/mistral

pdevine/bfloat16

jmorganca/mistral

brucemacd/mistral

pdevine/logging

parth/sample-correctness-fix

parth/sample-fix-sorting

jmorgan/sample-fix-sorting-extras

jmorganca/temp-0-images

brucemacd/parallel-embed-models

brucemacd/shim-grammar

jmorganca/fix-gguf-error

bmizerany/nameswork

jmorganca/faster-releases

bmizerany/validatenames

brucemacd/err-no-vocab

brucemacd/rope-config

brucemacd/err-hint

brucemacd/qwen2_5

brucemacd/logprobs

brucemacd/new_runner_graph_bench

progress-flicker

brucemacd/forward-test

brucemacd/go_qwen2

pdevine/gemma2

jmorganca/add-missing-symlink-eval

mxyng/next-debug

parth/set-context-size-openai

brucemacd/next-bpe-bench

brucemacd/next-bpe-test

brucemacd/new_runner_e2e

brucemacd/new_runner_qwen2

pdevine/convert-cohere2

brucemacd/convert-cli

parth/log-probs

mxyng/next-mlx

mxyng/cmd-history

parth/templating

parth/tokenize-detokenize

brucemacd/check-key-register

bmizerany/grammar

jmorganca/vendor-081b29bd

mxyng/func-checks

jmorganca/fix-null-format

parth/fix-default-to-warn-json

jmorganca/qwen2vl

jmorganca/no-concat

parth/cmd-cleanup-SO

brucemacd/check-key-register-structured-err

parth/openai-stream-usage

parth/fix-referencing-so

stream-tools-stop

jmorganca/degin-1

brucemacd/install-path-clean

brucemacd/push-name-validation

brucemacd/browser-key-register

jmorganca/openai-fix-first-message

jmorganca/fix-proxy

jessegross/sample

parth/disallow-streaming-tools

dhiltgen/remove_submodule

jmorganca/ga

jmorganca/mllama

pdevine/newlines

pdevine/geems-2b

jmorganca/llama-bump

mxyng/modelname-7

mxyng/gin-slog

mxyng/modelname-6

jyan/convert-prog

jyan/quant5

paligemma-support

pdevine/import-docs

jmorganca/openai-context

jyan/paligemma

jyan/p2

jyan/palitest

bmizerany/embedspeedup

jmorganca/llama-vit

brucemacd/allow-ollama

royh/ep-methods

royh/whisper

mxyng/api-models

mxyng/fix-memory

jyan/q4_4/8

jyan/ollama-v

royh/stream-tools

roy-embed-parallel

bmizerany/hrm

revert-5963-revert-5924-mxyng/llama3.1-rope

royh/embed-viz

jyan/local2

jyan/auth

jyan/local

jyan/parse-temp

jmorganca/template-mistral

jyan/reord-g

royh-openai-suffixdocs

royh-imgembed

royh-embed-parallel

jyan/quant4

royh-precision

jyan/progress

pdevine/fix-template

jyan/quant3

pdevine/ggla

mxyng/update-registry-domain

jmorganca/ggml-static

mxyng/create-context

jyan/v0.146

mxyng/layers-from-files

build_dist

bmizerany/noseek

royh-ls

royh-name

timeout

mxyng/server-timestamp

bmizerany/nosillyggufslurps

royh-params

jmorganca/llama-cpp-7c26775

royh-openai-delete

royh-show-rigid

jmorganca/enable-fa

jmorganca/no-error-template

jyan/format

royh-testdelete

bmizerany/fastverify

language_support

pdevine/ps-glitches

brucemacd/tokenize

bruce/iq-quants

bmizerany/filepathwithcoloninhost

mxyng/split-bin

bmizerany/client-registry

jmorganca/if-none-match

native

jmorganca/native

jmorganca/batch-embeddings

jmorganca/initcmake

jmorganca/mm

pdevine/showggmlinfo

modenameenforcealphanum

bmizerany/modenameenforcealphanum

jmorganca/done-reason

jmorganca/llama-cpp-8960fe8

ollama.com

bmizerany/filepathnobuild

bmizerany/types/model/defaultfix

rmdisplaylong

nogogen

bmizerany/x

modelfile-readme

bmizerany/replacecolon

jmorganca/limit

jmorganca/execstack

jmorganca/replace-assets

mxyng/tune-concurrency

jmorganca/testing

whitespace-detection

jmorganca/options

upgrade-all

scratch

cuda-search

mattw/airenamer

mattw/allmodelsonhuggingface

mattw/quantcontext

mattw/whatneedstorun

brucemacd/llama-mem-calc

mattw/faq-context

mattw/communitylinks

mattw/noprune

mattw/python-functioncalling

rename

mxyng/install

pulse

remove-first

editor

mattw/selfqueryingretrieval

cgo

mattw/howtoquant

api

matt/streamingapi

format-config

mxyng/extra-args

shell

update-nous-hermes

cp-model

upload-progress

fix-unknown-model

fix-model-names

delete-fix

insecure-registry

ls

deletemodels

progressbar

readme-updates

license-layers

skip-list

list-models

modelpath

matt/examplemodelfiles

distribution

go-opts

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/ollama#55694