github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #13454] Gemma3 QAT models produce gibberish/incoherent output (27b and 4b) #55391

New Issue
Closed
opened 2026-04-29 09:05:38 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-04-29 09:05:38 -05:00
Owner
Copy Link

Originally created by @jhsmith409 on GitHub (Dec 13, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/13454

What is the issue?

Gemma3 QAT models produce gibberish/incoherent output (27b and 4b) │

│ │
│ ## Description │
│ │
│ Gemma3 QAT models (gemma3:27b-it-qat and gemma3:4b-it-qat) produce completely incoherent, gibberish output while their │
│ non-QAT counterparts work perfectly. │
│ │
│ ## Environment │
│ │
│ - Ollama version: 0.13.3 │
│ - OS: Linux (Ubuntu) │
│ - GPU: NVIDIA (remote server at 192.168.30.211) │
│ - Model digests: │
│ - gemma3:27b-it-qat: 29eb0b9aeda3 (matches ollama.com) │
│ │
│ ## Steps to Reproduce │
│ │
│ bash │ │ # Test QAT model - produces garbage │ │ curl -s http://localhost:11434/api/chat -d '{ │ │ "model": "gemma3:27b-it-qat", │ │ "messages": [{"role": "user", "content": "Name three primary colors."}], │ │ "stream": false │ │ }' │ │ │ │ # Test non-QAT model - works correctly │ │ curl -s http://localhost:11434/api/chat -d '{ │ │ "model": "gemma3:27b", │ │ "messages": [{"role": "user", "content": "Name three primary colors."}], │ │ "stream": false │ │ }' │ │ │
│ │
│ ## Actual Behavior │
│ │
│ gemma3:27b-it-qat response: │
│ │ │ How to your │ │ Okay, let's: Let's assume you want to make it │ │ </end_of │ │ │ │ Okay>turn_turn> │ │ Okay, let's begin! turn> │ │ > │ │ │
│ │
│ gemma3:4b-it-qat response (wrong answer): │
│ │ │ Red, blue, green, yellow, and yellow, and orange, and purple │ │ │
│ │
│ Simple math test (2+2): │
│ - gemma3:27b-it-qat: "Okay, I'm a question with_turn> I can help me..." │
│ - gemma3:4b-it-qat: "2." (wrong) │
│ │
│ ## Expected Behavior │
│ │
│ gemma3:27b response (correct): │
│ │ │ The three primary colors are: │ │ * **Red** │ │ * **Yellow** │ │ * **Blue** │ │ │ │ These are called primary because they cannot be created by mixing other colors together, and all other colors can be │ │ created by mixing these three! │ │ │
│ │
│ ## Models Tested │
│ │
│ | Model | Status | Notes | │
│ |-------|--------|-------| │
│ | gemma3:27b-it-qat | ❌ BROKEN | Gibberish, broken tokens | │
│ | gemma3:4b-it-qat | ❌ BROKEN | Wrong answers, repetition | │
│ | gemma3:27b | ✅ OK | Works correctly | │
│ | gemma3:27b-it-fp16 | ✅ OK | Works correctly | │
│ | gemma3:4b | ✅ OK | Works correctly | │
│ | qwen3:8b | ✅ OK | Works correctly | │
│ | qwen3:32b | ✅ OK | Works correctly | │
│ | qwen3-coder:30b | ✅ OK | Works correctly | │
│ | llama3.3:70b | ✅ OK | Works correctly | │
│ | deepseek-r1:32b | ✅ OK | Works correctly | │
│ | gpt-oss:120b | ✅ OK | Works correctly | │
│ │
│ ## Model Info │
│ │
│ │ │ gemma3.rope.freq_base: 1000000 │ │ gemma3.rope.scaling.factor: 8 │ │ gemma3.rope.scaling.type: linear │ │ │
│ │
│ ## Related Issues │
│ │
│ This appears related to previously reported issues: │
│ - #12360 - Gemma3 outputs gibberish on Apple silicon (fixed in 0.12.1 per release notes) │
│ - #10945 - KV Cache Quantization breaks Gemma3 │
│ - #11276 - Gemma 3 QAT 12b/27b issue with </end_of_turn> token │
│ │
│ However, I am running 0.13.3 which should include all previous fixes, and the model digest matches the official one on │
│ ollama.com. The issue persists regardless of context size or other options. │
│ │
│ ## Additional Context │
│ │
│ - Issue occurs with direct curl calls, not specific to any client │
│ - Re-pulling the model does not help │
│ - The model was last updated "7 months ago" on ollama.com │
│ - Flash attention was enabled by default for Gemma3 in v0.12.7 - this may be related │
│ - Setting smaller num_ctx does not help

Relevant log output

see above

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.13.3

Originally created by @jhsmith409 on GitHub (Dec 13, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/13454 ### What is the issue? # Gemma3 QAT models produce gibberish/incoherent output (27b and 4b) │ │ │ │ ## Description │ │ │ │ Gemma3 QAT models (`gemma3:27b-it-qat` and `gemma3:4b-it-qat`) produce completely incoherent, gibberish output while their │ │ non-QAT counterparts work perfectly. │ │ │ │ ## Environment │ │ │ │ - **Ollama version:** 0.13.3 │ │ - **OS:** Linux (Ubuntu) │ │ - **GPU:** NVIDIA (remote server at 192.168.30.211) │ │ - **Model digests:** │ │ - `gemma3:27b-it-qat`: `29eb0b9aeda3` (matches ollama.com) │ │ │ │ ## Steps to Reproduce │ │ │ │ ```bash │ │ # Test QAT model - produces garbage │ │ curl -s http://localhost:11434/api/chat -d '{ │ │ "model": "gemma3:27b-it-qat", │ │ "messages": [{"role": "user", "content": "Name three primary colors."}], │ │ "stream": false │ │ }' │ │ │ │ # Test non-QAT model - works correctly │ │ curl -s http://localhost:11434/api/chat -d '{ │ │ "model": "gemma3:27b", │ │ "messages": [{"role": "user", "content": "Name three primary colors."}], │ │ "stream": false │ │ }' │ │ ``` │ │ │ │ ## Actual Behavior │ │ │ │ **gemma3:27b-it-qat response:** │ │ ``` │ │ How to your │ │ Okay, let's: Let's assume you want to make it │ │ </end_of │ │ │ │ Okay>turn_turn> │ │ Okay, let's begin! turn> │ │ > │ │ ``` │ │ │ │ **gemma3:4b-it-qat response (wrong answer):** │ │ ``` │ │ Red, blue, green, yellow, and yellow, and orange, and purple │ │ ``` │ │ │ │ **Simple math test (2+2):** │ │ - `gemma3:27b-it-qat`: `"Okay, I'm a question with_turn> I can help me..."` │ │ - `gemma3:4b-it-qat`: `"2."` (wrong) │ │ │ │ ## Expected Behavior │ │ │ │ **gemma3:27b response (correct):** │ │ ``` │ │ The three primary colors are: │ │ * **Red** │ │ * **Yellow** │ │ * **Blue** │ │ │ │ These are called primary because they cannot be created by mixing other colors together, and all other colors can be │ │ created by mixing these three! │ │ ``` │ │ │ │ ## Models Tested │ │ │ │ | Model | Status | Notes | │ │ |-------|--------|-------| │ │ | gemma3:27b-it-qat | ❌ BROKEN | Gibberish, broken tokens | │ │ | gemma3:4b-it-qat | ❌ BROKEN | Wrong answers, repetition | │ │ | gemma3:27b | ✅ OK | Works correctly | │ │ | gemma3:27b-it-fp16 | ✅ OK | Works correctly | │ │ | gemma3:4b | ✅ OK | Works correctly | │ │ | qwen3:8b | ✅ OK | Works correctly | │ │ | qwen3:32b | ✅ OK | Works correctly | │ │ | qwen3-coder:30b | ✅ OK | Works correctly | │ │ | llama3.3:70b | ✅ OK | Works correctly | │ │ | deepseek-r1:32b | ✅ OK | Works correctly | │ │ | gpt-oss:120b | ✅ OK | Works correctly | │ │ │ │ ## Model Info │ │ │ │ ``` │ │ gemma3.rope.freq_base: 1000000 │ │ gemma3.rope.scaling.factor: 8 │ │ gemma3.rope.scaling.type: linear │ │ ``` │ │ │ │ ## Related Issues │ │ │ │ This appears related to previously reported issues: │ │ - #12360 - Gemma3 outputs gibberish on Apple silicon (fixed in 0.12.1 per release notes) │ │ - #10945 - KV Cache Quantization breaks Gemma3 │ │ - #11276 - Gemma 3 QAT 12b/27b issue with `</end_of_turn>` token │ │ │ │ However, I am running **0.13.3** which should include all previous fixes, and the model digest matches the official one on │ │ ollama.com. The issue persists regardless of context size or other options. │ │ │ │ ## Additional Context │ │ │ │ - Issue occurs with direct curl calls, not specific to any client │ │ - Re-pulling the model does not help │ │ - The model was last updated "7 months ago" on ollama.com │ │ - Flash attention was enabled by default for Gemma3 in v0.12.7 - this may be related │ │ - Setting smaller `num_ctx` does not help ### Relevant log output ```shell see above ``` ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.13.3
GiteaMirror added the bug label 2026-04-29 09:05:38 -05:00
GiteaMirror commented 2026-04-29 09:05:42 -05:00
Author
Owner
Copy Link

@jhsmith409 commented on GitHub (Dec 13, 2025):

Checked a lot of other models. Errors only appear in Gemma3 QAT models:

Ollama Model Test Results │

│ │
│ Test Date: 2025-12-13 │
│ Ollama Version: 0.13.3 │
│ Server: 192.168.30.211:11434 │
│ Prompt: "Name three primary colors. Answer in one sentence." │
│ │
│ ## Summary │
│ │
│ - Total Models Tested: 67 │
│ - Working: 65 │
│ - Broken: 2 (both Gemma3 QAT variants) │
│ │
│ ## Results Table │
│ │
│ | Model | Status | Response Preview | │
│ |-------|--------|------------------| │
│ | cogito:32b | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | deepseek-ocr:3b | ✅ OK | Three primary colors are red, blue, and yellow. | │
│ | deepseek-r1:14b | ✅ OK | The three primary colors depend on the medium; for traditional art, they are red... | │
│ | deepseek-r1:32b | ✅ OK | The three primary colors are typically red, blue, and yellow in traditional art... | │
│ | deepseek-r1:70b | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | devstral:24b | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | exaone-deep:32b | ✅ OK | The three primary colors are red, yellow, and blue in the traditional... | │
│ | gemma3:27b | ✅ OK | The three primary colors are red, yellow, and blue, as they can be combined to c... | │
│ | gemma3:27b-it-fp16 | ✅ OK | The three primary colors are red, yellow, and blue, as they can be combined to c... | │
│ | gemma3:27b-it-qat | ❌ BROKEN | This is the first, end_sentence_end Name _of_model Answer, which is_turn. end I... | │
│ | gemma3:4b | ✅ OK | The three primary colors are red, blue, and yellow, as they cannot be created by... | │
│ | gemma3:4b-it-qat | ❌ BROKEN | Red, blue, yellow, green, and blue, and red, and yellow... | │
│ | gpt-oss:120b | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | gpt-oss:20b | ✅ OK | Red, blue, and yellow are the three primary colors. | │
│ | granite3.2:8b | ✅ OK | The three primary colors in light-based color models like RGB... | │
│ | granite4:micro-h | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | hf.co/a-m-team/AM-Thinking-v1-gguf:latest | ✅ OK | (thinking model - correct answer) | │
│ | hf.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF:Q4_K_M | ✅ OK | (thinking model - correct answer) | │
│ | hf.co/bartowski/Sky-T1-32B-Preview-GGUF:latest | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | hf.co/bartowski/zai-org_GLM-4.5-Air-GGUF:latest | ✅ OK | (thinking model - correct answer) | │
│ | hf.co/brittlewis12/s1.1-32B-GGUF:Q4_K_M | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | hf.co/mradermacher/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-i1-GGUF:IQ1_M | ✅ OK | (thinking model - correct answer) | │
│ | hf.co/mradermacher/medgemma-27b-multimodal-GGUF:latest | ✅ OK | The three primary colors are red, yellow, and blue. | │
│ | hf.co/mradermacher/Reformed-Christian-Bible-Expert-v2.1-12B-GGUF:Q4_K_M | ✅ OK | Red, blue, and yellow are the primary │
│ colors. | │
│ | hf.co/unsloth/Phi-4-reasoning-plus-GGUF:latest | ✅ OK | Red, blue, and yellow are the three primary colors. | │
│ | llama3.1:8b | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | llama3.3:70b | ✅ OK | The three primary colors are red, blue, and yellow, which cannot be created by m... | │
│ | llama4:16x17b | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | magistral:24b | ✅ OK | The three primary colors in the traditional RYB color model are red, yellow, blue... | │
│ | marco-o1:7b | ✅ OK | (thinking model - correct answer) | │
│ | mathstral:7b | ✅ OK | 1. Red, 2. Blue, 3. Yellow are the primary colors. | │
│ | ministral-3:14b | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | ministral-3:3b | ✅ OK | Three primary colors are red, blue, and yellow. | │
│ | ministral-3:8b | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | mistral:7b-instruct | ✅ OK | Red, blue, and yellow are the three primary colors... | │
│ | mistral-nemo:12b | ✅ OK | Red, blue, and yellow are the three primary colors. | │
│ | openthinker:32b | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | phi4-mini:3.8b | ✅ OK | Red, blue, and yellow are the primary colors. | │
│ | phi4-reasoning:14b | ✅ OK | (thinking model - correct answer) | │
│ | qwen2.5:14b | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | qwen2.5vl:7b | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | qwen3:0.6b | ✅ OK | Three primary colors are red, blue, and yellow. | │
��� | qwen3:0.6b-fp16 | ✅ OK | Three primary colors are red, blue, and yellow. | │
│ | qwen3:0.6b-q8_0 | ✅ OK | The primary colors are red, blue, and yellow. | │
│ | qwen3:1.7b | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | qwen3:1.7b-fp16 | ✅ OK | The three primary colors are red, yellow, and blue. | │
│ | qwen3:1.7b-q8_0 | ✅ OK | The three primary colors are red, yellow, and blue. | │
│ | qwen3:30b-a3b-instruct-2507-fp16 | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | qwen3:30b-a3b-instruct-2507-q4_K_M | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | qwen3:30b-a3b-thinking-2507-fp16 | ✅ OK | The three primary colors are red, yellow, and blue. | │
│ | qwen3:30b-a3b-thinking-2507-q4_K_M | ✅ OK | The three primary colors are red, yellow, and blue. | │
│ | qwen3:32b | ✅ OK | The three primary colors are red, yellow, and blue. | │
│ | qwen3:4b-instruct | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | qwen3:4b-thinking | ✅ OK | The three primary colors are red, yellow, and blue. | │
│ | qwen3:4b-thinking-2507-fp16 | ✅ OK | The three primary colors are red, yellow, and blue. | │
│ | qwen3:4b-thinking-2507-q8_0 | ✅ OK | The three primary colors in traditional art are red, blue, and yellow. | │
│ | qwen3:8b | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | qwen3:8b-fp16 | ✅ OK | The three primary colors in the traditional RYB color model are red, yellow, and... | │
│ | qwen3:8b-q8_0 | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | qwen3-coder:30b | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | qwen3-coder:30b-a3b-fp16 | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | qwen3-next:80b | ✅ OK | The three primary colors are red, yellow, and blue. | │
│ | qwen3-vl:30b | ✅ OK | The three primary colors are red, yellow, and blue. | │
│ | qwen3-vl:32b-instruct | ✅ OK | The three primary colors are red, blue, and yellow. | │
│ | qwen3-vl:32b-thinking | ✅ OK | The three primary colors are red, yellow, and blue. | │
│ | qwq:32b | ✅ OK | (thinking model - correct answer) | │
│ | r1-1776:70b | ✅ OK | (thinking model - correct answer) | │
│ | reflection:70b | ✅ OK | (thinking model - correct answer) | │
│ │
│ ## Broken Models Analysis │
│ │
│ ### gemma3:27b-it-qat │
│ - Status: ❌ BROKEN │
│ - Issue: Produces gibberish with broken tokens │
│ - Sample Output: This is the first, end_sentence_end Name _of_model Answer, which is_turn. end I │
│ │
│ ### gemma3:4b-it-qat │
│ - Status: ❌ BROKEN │
│ - Issue: Produces wrong/repetitive answers │
│ - Sample Output: Red, blue, yellow, green, and blue, and red, and yellow │
│ │
│ ## Conclusion │
│ │
│ Only the Gemma3 QAT models are broken. All other models (65 out of 67) work correctly, including: │
│ - Regular Gemma3 variants (gemma3:27b, gemma3:4b, gemma3:27b-it-fp16) │
│ - All Qwen3 variants (0.6b through 80b, including coder and VL) │
│ - All DeepSeek variants │
│ - All Llama variants │
│ - All Mistral variants │
│ - All Phi4 variants │
│ - All custom HuggingFace models │
│ │
│ This confirms the issue is specific to Gemma3 QAT quantization in Ollama. │
│

<!-- gh-comment-id:3648738032 --> @jhsmith409 commented on GitHub (Dec 13, 2025): Checked a lot of other models. Errors only appear in Gemma3 QAT models: # Ollama Model Test Results │ │ │ │ **Test Date:** 2025-12-13 │ │ **Ollama Version:** 0.13.3 │ │ **Server:** 192.168.30.211:11434 │ │ **Prompt:** "Name three primary colors. Answer in one sentence." │ │ │ │ ## Summary │ │ │ │ - **Total Models Tested:** 67 │ │ - **Working:** 65 │ │ - **Broken:** 2 (both Gemma3 QAT variants) │ │ │ │ ## Results Table │ │ │ │ | Model | Status | Response Preview | │ │ |-------|--------|------------------| │ │ | cogito:32b | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | deepseek-ocr:3b | ✅ OK | Three primary colors are red, blue, and yellow. | │ │ | deepseek-r1:14b | ✅ OK | The three primary colors depend on the medium; for traditional art, they are red... | │ │ | deepseek-r1:32b | ✅ OK | The three primary colors are typically red, blue, and yellow in traditional art... | │ │ | deepseek-r1:70b | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | devstral:24b | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | exaone-deep:32b | ✅ OK | The three primary colors are red, yellow, and blue in the traditional... | │ │ | gemma3:27b | ✅ OK | The three primary colors are red, yellow, and blue, as they can be combined to c... | │ │ | gemma3:27b-it-fp16 | ✅ OK | The three primary colors are red, yellow, and blue, as they can be combined to c... | │ │ | **gemma3:27b-it-qat** | ❌ BROKEN | This is the first, end_sentence_end Name _of_model Answer, which is_turn. end I... | │ │ | gemma3:4b | ✅ OK | The three primary colors are red, blue, and yellow, as they cannot be created by... | │ │ | **gemma3:4b-it-qat** | ❌ BROKEN | Red, blue, yellow, green, and blue, and red, and yellow... | │ │ | gpt-oss:120b | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | gpt-oss:20b | ✅ OK | Red, blue, and yellow are the three primary colors. | │ │ | granite3.2:8b | ✅ OK | The three primary colors in light-based color models like RGB... | │ │ | granite4:micro-h | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | hf.co/a-m-team/AM-Thinking-v1-gguf:latest | ✅ OK | (thinking model - correct answer) | │ │ | hf.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF:Q4_K_M | ✅ OK | (thinking model - correct answer) | │ │ | hf.co/bartowski/Sky-T1-32B-Preview-GGUF:latest | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | hf.co/bartowski/zai-org_GLM-4.5-Air-GGUF:latest | ✅ OK | (thinking model - correct answer) | │ │ | hf.co/brittlewis12/s1.1-32B-GGUF:Q4_K_M | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | hf.co/mradermacher/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-i1-GGUF:IQ1_M | ✅ OK | (thinking model - correct answer) | │ │ | hf.co/mradermacher/medgemma-27b-multimodal-GGUF:latest | ✅ OK | The three primary colors are red, yellow, and blue. | │ │ | hf.co/mradermacher/Reformed-Christian-Bible-Expert-v2.1-12B-GGUF:Q4_K_M | ✅ OK | Red, blue, and yellow are the primary │ │ colors. | │ │ | hf.co/unsloth/Phi-4-reasoning-plus-GGUF:latest | ✅ OK | Red, blue, and yellow are the three primary colors. | │ │ | llama3.1:8b | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | llama3.3:70b | ✅ OK | The three primary colors are red, blue, and yellow, which cannot be created by m... | │ │ | llama4:16x17b | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | magistral:24b | ✅ OK | The three primary colors in the traditional RYB color model are red, yellow, blue... | │ │ | marco-o1:7b | ✅ OK | (thinking model - correct answer) | │ │ | mathstral:7b | ✅ OK | 1. Red, 2. Blue, 3. Yellow are the primary colors. | │ │ | ministral-3:14b | ✅ OK | The three primary colors are **red, blue, and yellow**. | │ │ | ministral-3:3b | ✅ OK | Three primary colors are **red, blue, and yellow**. | │ │ | ministral-3:8b | ✅ OK | The three primary colors are **red, blue, and yellow**. | │ │ | mistral:7b-instruct | ✅ OK | Red, blue, and yellow are the three primary colors... | │ │ | mistral-nemo:12b | ✅ OK | Red, blue, and yellow are the three primary colors. | │ │ | openthinker:32b | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | phi4-mini:3.8b | ✅ OK | Red, blue, and yellow are the primary colors. | │ │ | phi4-reasoning:14b | ✅ OK | (thinking model - correct answer) | │ │ | qwen2.5:14b | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | qwen2.5vl:7b | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | qwen3:0.6b | ✅ OK | Three primary colors are red, blue, and yellow. | │ ��� | qwen3:0.6b-fp16 | ✅ OK | Three primary colors are red, blue, and yellow. | │ │ | qwen3:0.6b-q8_0 | ✅ OK | The primary colors are red, blue, and yellow. | │ │ | qwen3:1.7b | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | qwen3:1.7b-fp16 | ✅ OK | The three primary colors are red, yellow, and blue. | │ │ | qwen3:1.7b-q8_0 | ✅ OK | The three primary colors are red, yellow, and blue. | │ │ | qwen3:30b-a3b-instruct-2507-fp16 | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | qwen3:30b-a3b-instruct-2507-q4_K_M | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | qwen3:30b-a3b-thinking-2507-fp16 | ✅ OK | The three primary colors are red, yellow, and blue. | │ │ | qwen3:30b-a3b-thinking-2507-q4_K_M | ✅ OK | The three primary colors are red, yellow, and blue. | │ │ | qwen3:32b | ✅ OK | The three primary colors are red, yellow, and blue. | │ │ | qwen3:4b-instruct | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | qwen3:4b-thinking | ✅ OK | The three primary colors are red, yellow, and blue. | │ │ | qwen3:4b-thinking-2507-fp16 | ✅ OK | The three primary colors are red, yellow, and blue. | │ │ | qwen3:4b-thinking-2507-q8_0 | ✅ OK | The three primary colors in traditional art are red, blue, and yellow. | │ │ | qwen3:8b | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | qwen3:8b-fp16 | ✅ OK | The three primary colors in the traditional RYB color model are red, yellow, and... | │ │ | qwen3:8b-q8_0 | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | qwen3-coder:30b | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | qwen3-coder:30b-a3b-fp16 | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | qwen3-next:80b | ✅ OK | The three primary colors are red, yellow, and blue. | │ │ | qwen3-vl:30b | ✅ OK | The three primary colors are red, yellow, and blue. | │ │ | qwen3-vl:32b-instruct | ✅ OK | The three primary colors are red, blue, and yellow. | │ │ | qwen3-vl:32b-thinking | ✅ OK | The three primary colors are red, yellow, and blue. | │ │ | qwq:32b | ✅ OK | (thinking model - correct answer) | │ │ | r1-1776:70b | ✅ OK | (thinking model - correct answer) | │ │ | reflection:70b | ✅ OK | (thinking model - correct answer) | │ │ │ │ ## Broken Models Analysis │ │ │ │ ### gemma3:27b-it-qat │ │ - **Status:** ❌ BROKEN │ │ - **Issue:** Produces gibberish with broken tokens │ │ - **Sample Output:** `This is the first, end_sentence_end Name _of_model Answer, which is_turn. end I` │ │ │ │ ### gemma3:4b-it-qat │ │ - **Status:** ❌ BROKEN │ │ - **Issue:** Produces wrong/repetitive answers │ │ - **Sample Output:** `Red, blue, yellow, green, and blue, and red, and yellow` │ │ │ │ ## Conclusion │ │ │ │ Only the **Gemma3 QAT models** are broken. All other models (65 out of 67) work correctly, including: │ │ - Regular Gemma3 variants (gemma3:27b, gemma3:4b, gemma3:27b-it-fp16) │ │ - All Qwen3 variants (0.6b through 80b, including coder and VL) │ │ - All DeepSeek variants │ │ - All Llama variants │ │ - All Mistral variants │ │ - All Phi4 variants │ │ - All custom HuggingFace models │ │ │ │ This confirms the issue is specific to Gemma3 QAT quantization in Ollama. │ │
GiteaMirror commented 2026-04-29 09:05:43 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Dec 13, 2025):

Sorry about this. A fix was just merged and a new version will be in pre-release very soon.

<!-- gh-comment-id:3648738284 --> @jmorganca commented on GitHub (Dec 13, 2025): Sorry about this. A fix was just merged and a new version will be in pre-release very soon.
GiteaMirror commented 2026-04-29 09:05:44 -05:00
Author
Owner
Copy Link

@jhsmith409 commented on GitHub (Dec 13, 2025):

Thank you for the quick response.

<!-- gh-comment-id:3648739677 --> @jhsmith409 commented on GitHub (Dec 13, 2025): Thank you for the quick response.
GiteaMirror commented 2026-04-29 09:05:45 -05:00
Author
Owner
Copy Link

@jhsmith409 commented on GitHub (Dec 13, 2025):

Looks like 0.13.4rc1 is working correctly now. Thank you.

Model Response Preview
cogito:32b The three primary colors are red, blue, and yellow.
deepseek-ocr:3b Three primary colors are red, blue, and yellow.
deepseek-r1:14b The three primary colors are red, blue, and yellow.
deepseek-r1:32b The three primary colors are red, blue, and yellow.
deepseek-r1:70b The three primary colors are red, blue, and yellow.
devstral:24b The three primary colors are red, blue, and yellow.
exaone-deep:32b red, yellow, and blue (in the traditional RYB color model), though additive pri...
gemma3:27b The three primary colors are red, yellow, and blue, as they can be combined to c...
gemma3:27b-it-fp16 The three primary colors are red, yellow, and blue, as they can be combined to c...
gemma3:27b-it-qat Red, blue, and yellow are the three primary colors.
gemma3:4b The three primary colors are red, blue, and yellow, as they cannot be created by...
gemma3:4b-it-qat The three primary colors are red, yellow, and blue, as they cannot be created by...
gpt-oss:120b The three primary colors are red, blue, and yellow.
gpt-oss:20b Red, blue, and yellow.
granite3.2:8b The three primary colors are red, blue, and yellow.
granite4:micro-h The three primary colors are red, blue, and yellow.
hf.co/a-m-team/AM-Thinking-v1-gguf:latest Okay, the user wants a one-sentence answer. Let me check the question aga...
hf.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF:Q4_K_M ...the question is general, I think it's expecting the traditional a...
hf.co/bartowski/zai-org_GLM-4.5-Air-GGUF:latest Okay, the user asked me to name three primary colors and wants the answer...
hf.co/brittlewis12/s1.1-32B-GGUF:Q4_K_M The three primary colors are red, blue, and yellow.
hf.co/mradermacher/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-i1-GGUF:IQ1_M ...and green. Wait, but I'm not sure. Alternatively, perhaps the answer is red,...
hf.co/mradermacher/medgemma-27b-multimodal-GGUF:latest The three primary colors are red, yellow, and blue.
hf.co/mradermacher/Reformed-Christian-Bible-Expert-v2.1-12B-GGUF:Q4_K_M Red, Blue, Yellow.
hf.co/unsloth/Phi-4-reasoning-plus-GGUF:latest The three primary colors are red, blue, and yellow.
llama3.1:8b The three primary colors are red, blue, and yellow.
llama3.3:70b The three primary colors are red, blue, and yellow, which cannot be created by m...
llama4:16x17b The three primary colors are red, blue, and yellow.
marco-o1:7b Okay, I think I've got that straight. So, to answer the question succinctly: Red...
mathstral:7b 1. Red, 2. Blue, 3. Yellow are the primary colors.
ministral-3:14b The three primary colors are red, blue, and yellow.
ministral-3:3b Three primary colors are red, blue, and yellow.
ministral-3:8b The three primary colors are red, blue, and yellow.
mistral:7b-instruct Red, blue, and yellow are the three primary colors in the additive color system...
mistral-nemo:12b Red, blue, and yellow.
openthinker:32b The three primary colors are red, blue, and yellow.
phi4-mini:3.8b Red, blue, and yellow are the three primary colors.
phi4-reasoning:14b ...primary colors are red, blue, and yellow. That's clear. Thus I'll produce final...
qwen2.5:14b The three primary colors are red, blue, and yellow.
qwen2.5vl:7b The three primary colors are red, blue, and yellow.
qwen3:0.6b The three primary colors are red, blue, and yellow.
qwen3:0.6b-fp16 The primary colors are red, blue, and y
<!-- gh-comment-id:3649470877 --> @jhsmith409 commented on GitHub (Dec 13, 2025): Looks like 0.13.4rc1 is working correctly now. Thank you. | Model | Response Preview | |--------------------------------------------------------------------------|---------------------------------------------------------------------------------------| | cogito:32b | The three primary colors are red, blue, and yellow. | | deepseek-ocr:3b | Three primary colors are red, blue, and yellow. | | deepseek-r1:14b | The three primary colors are red, blue, and yellow. | | deepseek-r1:32b | The three primary colors are red, blue, and yellow. | | deepseek-r1:70b | The three primary colors are red, blue, and yellow. | | devstral:24b | The three primary colors are red, blue, and yellow. | | exaone-deep:32b | red, yellow, and blue (in the traditional RYB color model), though additive pri... | | gemma3:27b | The three primary colors are red, yellow, and blue, as they can be combined to c... | | gemma3:27b-it-fp16 | The three primary colors are red, yellow, and blue, as they can be combined to c... | | gemma3:27b-it-qat | Red, blue, and yellow are the three primary colors. | | gemma3:4b | The three primary colors are red, blue, and yellow, as they cannot be created by... | | gemma3:4b-it-qat | The three primary colors are red, yellow, and blue, as they cannot be created by... | | gpt-oss:120b | The three primary colors are red, blue, and yellow. | | gpt-oss:20b | Red, blue, and yellow. | | granite3.2:8b | The three primary colors are red, blue, and yellow. | | granite4:micro-h | The three primary colors are red, blue, and yellow. | | hf.co/a-m-team/AM-Thinking-v1-gguf:latest | Okay, the user wants a one-sentence answer. Let me check the question aga... | | hf.co/bartowski/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-GGUF:Q4_K_M | ...the question is general, I think it's expecting the traditional a... | | hf.co/bartowski/zai-org_GLM-4.5-Air-GGUF:latest | Okay, the user asked me to name three primary colors and wants the answer... | | hf.co/brittlewis12/s1.1-32B-GGUF:Q4_K_M | The three primary colors are red, blue, and yellow. | | hf.co/mradermacher/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview-i1-GGUF:IQ1_M | ...and green. Wait, but I'm not sure. Alternatively, perhaps the answer is red,... | | hf.co/mradermacher/medgemma-27b-multimodal-GGUF:latest | The three primary colors are red, yellow, and blue. | | hf.co/mradermacher/Reformed-Christian-Bible-Expert-v2.1-12B-GGUF:Q4_K_M | Red, Blue, Yellow. | | hf.co/unsloth/Phi-4-reasoning-plus-GGUF:latest | The three primary colors are red, blue, and yellow. | | llama3.1:8b | The three primary colors are red, blue, and yellow. | | llama3.3:70b | The three primary colors are red, blue, and yellow, which cannot be created by m... | | llama4:16x17b | The three primary colors are red, blue, and yellow. | | marco-o1:7b | Okay, I think I've got that straight. So, to answer the question succinctly: Red... | | mathstral:7b | 1. Red, 2. Blue, 3. Yellow are the primary colors. | | ministral-3:14b | The three primary colors are red, blue, and yellow. | | ministral-3:3b | Three primary colors are red, blue, and yellow. | | ministral-3:8b | The three primary colors are red, blue, and yellow. | | mistral:7b-instruct | Red, blue, and yellow are the three primary colors in the additive color system... | | mistral-nemo:12b | Red, blue, and yellow. | | openthinker:32b | The three primary colors are red, blue, and yellow. | | phi4-mini:3.8b | Red, blue, and yellow are the three primary colors. | | phi4-reasoning:14b | ...primary colors are red, blue, and yellow. That's clear. Thus I'll produce final... | | qwen2.5:14b | The three primary colors are red, blue, and yellow. | | qwen2.5vl:7b | The three primary colors are red, blue, and yellow. | | qwen3:0.6b | The three primary colors are red, blue, and yellow. | | qwen3:0.6b-fp16 | The primary colors are red, blue, and y
GiteaMirror commented 2026-04-29 09:05:47 -05:00
Author
Owner
Copy Link

@jmorganca commented on GitHub (Dec 13, 2025):

@jhsmith409 thanks for confirming and sorry about the issue!

<!-- gh-comment-id:3649720548 --> @jmorganca commented on GitHub (Dec 13, 2025): @jhsmith409 thanks for confirming and sorry about the issue!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#55391
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?