github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 13:19:40 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #10361] Broken long context performance of Gemma3-27B #6808

New Issue
Open
opened 2026-04-12 18:35:41 -05:00 by GiteaMirror · 1 comment
GiteaMirror commented 2026-04-12 18:35:41 -05:00
Owner
Copy Link

Originally created by @vYLQs6 on GitHub (Apr 21, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/10361

What is the issue?

The long context performance of Gemma3-27B is completely broken in Ollama

The gemma3-27b model I used: ollama run gemma3:27b-it-qat

GPU: RTX 4090 24GB

set OLLAMA_FLASH_ATTENTION=1 && set OLLAMA_KV_CACHE_TYPE=q8_0 && ollama serve

Here is the prompt + video subtitles I used for testing: prompt_script.txt

Here are the results:

ollama run gemma3:27b-it-qat: gemma3-ollama.txt

Qwen2.5-14B-Instruct-1M-IQ4_XS: qwen2-5-1M-14B-IQ4XS.txt

openrouter.ai/google/gemma-3-27b-it:free: gemma3-router.txt

As you can see, both the open router version and the Qwen2.5-14B-1M(which is running in ollama) at least knows what they are supposed to do, but gemma3:27b-it-qat just completely lost it, it has no idea what it supposed to do.

Both Qwen2.5 and the gemma3 model use a 70K context window with officially recommended settings, with no system prompt.

For gemma3, this whole prompt only costs 48.3K tokens, which is nowhere near 70K.

You can easily test this yourself.

Relevant log output

OS

Windows

GPU

Nvidia

CPU

AMD

Ollama version

0.6.6-rc2

Originally created by @vYLQs6 on GitHub (Apr 21, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/10361 ### What is the issue? The long context performance of Gemma3-27B is completely broken in Ollama --- The gemma3-27b model I used: `ollama run gemma3:27b-it-qat` GPU: RTX 4090 24GB `set OLLAMA_FLASH_ATTENTION=1 && set OLLAMA_KV_CACHE_TYPE=q8_0 && ollama serve` Here is the prompt + video subtitles I used for testing: [prompt_script.txt](https://github.com/user-attachments/files/19834339/prompt_script.txt) --- Here are the results: `ollama run gemma3:27b-it-qat`: [gemma3-ollama.txt](https://github.com/user-attachments/files/19834345/gemma3-ollama.txt) `Qwen2.5-14B-Instruct-1M-IQ4_XS`: [qwen2-5-1M-14B-IQ4XS.txt](https://github.com/user-attachments/files/19834350/qwen2-5-1M-14B-IQ4XS.txt) `openrouter.ai/google/gemma-3-27b-it:free`: [gemma3-router.txt](https://github.com/user-attachments/files/19834356/gemma3-router.txt) --- As you can see, both the open router version and the Qwen2.5-14B-1M(which is running in ollama) at least knows what they are supposed to do, but gemma3:27b-it-qat just completely lost it, it has no idea what it supposed to do. Both Qwen2.5 and the gemma3 model use a 70K context window with officially recommended settings, with no system prompt. For gemma3, this whole prompt only costs 48.3K tokens, which is nowhere near 70K. You can easily test this yourself. ### Relevant log output ```shell ``` ### OS Windows ### GPU Nvidia ### CPU AMD ### Ollama version 0.6.6-rc2
GiteaMirror added the bug label 2026-04-12 18:35:41 -05:00
GiteaMirror commented 2026-04-12 18:35:43 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Apr 21, 2025):

2x A100-40G

$ OLLAMA_CONTEXT_LENGTH=70000 OLLAMA_DOCKER_TAG=0.6.6 docker compose up -d ollama && ollama run gemma3:27b-it-qat < /tmp/prompt_script.txt
This is a *very* thorough transcript of a recent WAN show, covering a lot of ground! Here's a breakdown, categorized for easier reading, with key takeaways and observations:

**I. Opening & General Chat (First ~10 minutes - mostly setting the stage)**

*   **Topic Overview:** The primary focus is switching to more professional processes.
*   **Lionus's Involvement:**  He's starting to feel like they are ready to go through the motions of official channels and projects.

**II. Nvidia RTX 5060, Budget Laptops, & Hardware (Next ~10 minutes)**

*   **Nvidia RTX 5060:** The show started with the team making a team making a video about a certain product but it was the team making a video about a certain product.
*   **Budget Laptop Review:** They also looked at affordable laptops.
*   **Gym Rat Preferences:** Luke shares his favorite lift (bench press).
...

$ OLLAMA_CONTEXT_LENGTH=70000 OLLAMA_DOCKER_TAG=0.6.6 OLLAMA_FLASH_ATTENTION=1 docker compose up -d ollama && ollama run gemma3:27b-it-qat < /tmp/prompt_script.txt
This is a *very* comprehensive transcript of a recent WAN Show, covering a wide range of topics! Here's a breakdown and some key takeaways, organized for readability:

**I. Initial Topics & Overview**

*   **Nvidia's 5060/RTX Series:** The show started with a discussion of Nvidia's new cards and a comment on the general stock situation.
*   **Transparency & Honesty:** Lionus's philosophy on being open, even if it leads to issues. He acknowledges the inherent messiness but believes it's beneficial.
*   **Current events:** Several new topics were briefly addressed.
*   **Pool Contractor Saga:** Ongoing discussion about issues with the contractor, the desire for transparency, and why the situation occurred.
...

$ OLLAMA_CONTEXT_LENGTH=70000 OLLAMA_DOCKER_TAG=0.6.6 OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 docker compose up -d ollama && ollama run gemma3:27b-it-qat < /tmp/prompt_script.txt 
## Summary of the WAN Show - April 26, 2024

This is a detailed summary of the WAN Show from April 26, 2024.  It's a long, conversational episode, so this is thorough!

**Core Topics & Highlights:**

* **Nvidia 5060 Discussion (briefly):** The show touched on the 5060 launch, but didn’t dwell on it. The core message was that it was difficult to find at a good price, but otherwise unremarkable.
* **Pool Contractor Debacle (continued):**  The story of the extremely frustrating pool experience continued. The conversation centered on the fact that LTT paid way too much for a job that took far too long, and the fact that they’re likely not going to get a 
refund.
* **Transparency & Trust (central theme):** A lot of the show revolved around the idea of LTT's transparency and the challenges that it presents. Luke pointed out that while

...

If part of the model is forced into RAM, it's more like the output in the bug report:

$ OLLAMA_CONTEXT_LENGTH=70000 OLLAMA_GPU_OVERHEAD=20000000000 OLLAMA_DOCKER_TAG=0.6.6 CUDA_VISIBLE_DEVICES=0 docker compose up -d ollama && ollama run gemma3:27b-it-qat < /tmp/prompt_script.txt
This is a fantastic, detailed transcript of the WAN Show! You've captured a lot of the nuances and conversational flow, which is great. Here's a breakdown of why it's good, and some minor suggestions for even better organization/readability:

**Strengths:**

* **Comprehensive:**  You haven’t missed significant portions of the show. The detail is excellent, including the tangents and side conversations.
* **Accurate:** The transcription appears very accurate, capturing the spoken word well.

Forcing all but one layer into RAM had similar results, less focus on the products:

$ echo FROM gemma3:27b-it-qat > Modelfile
$ echo PARAMETER num_gpu 1 >> Modelfile
$ ollama create gemma3:27b-it-qat-g1
$ OLLAMA_CONTEXT_LENGTH=70000 OLLAMA_DOCKER_TAG=0.6.6 docker compose up -d ollama && ollama run gemma3:27b-it-qat-g1 < /tmp/prompt_script.txt 
Okay, here's a detailed summary of the LTT WAN Show from the provided transcript.  It's a *long* post because a lot was covered, but I’ve tried to be thorough. I’ve organized it into sections to make it easier to navigate.

**I. General Overview & Recurring Themes**

*   **Transparency & Honesty (with a caveat):** Luke often states a willingness to be transparent, but acknowledges his personal struggles with being detail-oriented enough to always *avoid* being misunderstood.  He frequently mentions that his honesty leads to 
assumptions and problems that he didn't intend.
*   **Company Culture:**  The team strives to maintain a good culture and make things interesting, but they acknowledge the inherent complexities of working with a large team.
*   **Details & Rigor:** Luke repeatedly stresses the importance of details, accountabiltiy and clear understanding.
...

The non-QAT model also seems to skip over some products:

$ OLLAMA_CONTEXT_LENGTH=70000 OLLAMA_GPU_OVERHEAD=20000000000 OLLAMA_DOCKER_TAG=0.6.6 CUDA_VISIBLE_DEVICES=0 docker compose up -d ollama && ollama run gemma3:27b < /tmp/prompt_script.txt
## Detailed Summary of the WAN Show - April 26, 2024

This is a comprehensive summary of the April 26, 2024 episode of the WAN Show, covering a wide range of topics from product updates and industry news to behind-the-scenes company discussions and humorous tangents.

**I. Introduction & Product News (0:00 - 10:00)**

*   The show opened with a discussion about the extended promotional period for the LTStore, offering free shipping on orders over $150, due to potential tariff increases.
*   Mentioned the ongoing launch of the new website for the Lab, the main focus is on refining and improving the Lab’s online experience.
*   Acknowledged the challenge of balancing transparency with protecting internal processes and the potential for misinterpretation of information shared publicly.

**II. Product Updates & Discussions (10:00 - 30:00)**

*   **Budget Laptops:** The team reviewed budget laptops, highlighting the contrast between affordability and quality.
*   **NVIDIA A100 Server & LLM Integration:** Discussed the potential of using a high-end server (with an A100 GPU) for running large language models (LLMs) and the challenges of setting up such a system.
*   **NAS/Server Builds:**  Highlighted the complexity of building and maintaining a robust home server, and the importance of choosing the right components.
*   **The Team's approach to setting up a remote location is in progress.**
...

It seems like layer spilling causes the model to lose a bit of focus.

<!-- gh-comment-id:2818933232 --> @rick-github commented on GitHub (Apr 21, 2025): 2x A100-40G ```console $ OLLAMA_CONTEXT_LENGTH=70000 OLLAMA_DOCKER_TAG=0.6.6 docker compose up -d ollama && ollama run gemma3:27b-it-qat < /tmp/prompt_script.txt This is a *very* thorough transcript of a recent WAN show, covering a lot of ground! Here's a breakdown, categorized for easier reading, with key takeaways and observations: **I. Opening & General Chat (First ~10 minutes - mostly setting the stage)** * **Topic Overview:** The primary focus is switching to more professional processes. * **Lionus's Involvement:** He's starting to feel like they are ready to go through the motions of official channels and projects. **II. Nvidia RTX 5060, Budget Laptops, & Hardware (Next ~10 minutes)** * **Nvidia RTX 5060:** The show started with the team making a team making a video about a certain product but it was the team making a video about a certain product. * **Budget Laptop Review:** They also looked at affordable laptops. * **Gym Rat Preferences:** Luke shares his favorite lift (bench press). ... $ OLLAMA_CONTEXT_LENGTH=70000 OLLAMA_DOCKER_TAG=0.6.6 OLLAMA_FLASH_ATTENTION=1 docker compose up -d ollama && ollama run gemma3:27b-it-qat < /tmp/prompt_script.txt This is a *very* comprehensive transcript of a recent WAN Show, covering a wide range of topics! Here's a breakdown and some key takeaways, organized for readability: **I. Initial Topics & Overview** * **Nvidia's 5060/RTX Series:** The show started with a discussion of Nvidia's new cards and a comment on the general stock situation. * **Transparency & Honesty:** Lionus's philosophy on being open, even if it leads to issues. He acknowledges the inherent messiness but believes it's beneficial. * **Current events:** Several new topics were briefly addressed. * **Pool Contractor Saga:** Ongoing discussion about issues with the contractor, the desire for transparency, and why the situation occurred. ... $ OLLAMA_CONTEXT_LENGTH=70000 OLLAMA_DOCKER_TAG=0.6.6 OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 docker compose up -d ollama && ollama run gemma3:27b-it-qat < /tmp/prompt_script.txt ## Summary of the WAN Show - April 26, 2024 This is a detailed summary of the WAN Show from April 26, 2024. It's a long, conversational episode, so this is thorough! **Core Topics & Highlights:** * **Nvidia 5060 Discussion (briefly):** The show touched on the 5060 launch, but didn’t dwell on it. The core message was that it was difficult to find at a good price, but otherwise unremarkable. * **Pool Contractor Debacle (continued):** The story of the extremely frustrating pool experience continued. The conversation centered on the fact that LTT paid way too much for a job that took far too long, and the fact that they’re likely not going to get a refund. * **Transparency & Trust (central theme):** A lot of the show revolved around the idea of LTT's transparency and the challenges that it presents. Luke pointed out that while ... ``` If part of the model is forced into RAM, it's more like the output in the bug report: ```console $ OLLAMA_CONTEXT_LENGTH=70000 OLLAMA_GPU_OVERHEAD=20000000000 OLLAMA_DOCKER_TAG=0.6.6 CUDA_VISIBLE_DEVICES=0 docker compose up -d ollama && ollama run gemma3:27b-it-qat < /tmp/prompt_script.txt This is a fantastic, detailed transcript of the WAN Show! You've captured a lot of the nuances and conversational flow, which is great. Here's a breakdown of why it's good, and some minor suggestions for even better organization/readability: **Strengths:** * **Comprehensive:** You haven’t missed significant portions of the show. The detail is excellent, including the tangents and side conversations. * **Accurate:** The transcription appears very accurate, capturing the spoken word well. ``` Forcing all but one layer into RAM had similar results, less focus on the products: ```console $ echo FROM gemma3:27b-it-qat > Modelfile $ echo PARAMETER num_gpu 1 >> Modelfile $ ollama create gemma3:27b-it-qat-g1 $ OLLAMA_CONTEXT_LENGTH=70000 OLLAMA_DOCKER_TAG=0.6.6 docker compose up -d ollama && ollama run gemma3:27b-it-qat-g1 < /tmp/prompt_script.txt Okay, here's a detailed summary of the LTT WAN Show from the provided transcript. It's a *long* post because a lot was covered, but I’ve tried to be thorough. I’ve organized it into sections to make it easier to navigate. **I. General Overview & Recurring Themes** * **Transparency & Honesty (with a caveat):** Luke often states a willingness to be transparent, but acknowledges his personal struggles with being detail-oriented enough to always *avoid* being misunderstood. He frequently mentions that his honesty leads to assumptions and problems that he didn't intend. * **Company Culture:** The team strives to maintain a good culture and make things interesting, but they acknowledge the inherent complexities of working with a large team. * **Details & Rigor:** Luke repeatedly stresses the importance of details, accountabiltiy and clear understanding. ... ``` The non-QAT model also seems to skip over some products: ```console $ OLLAMA_CONTEXT_LENGTH=70000 OLLAMA_GPU_OVERHEAD=20000000000 OLLAMA_DOCKER_TAG=0.6.6 CUDA_VISIBLE_DEVICES=0 docker compose up -d ollama && ollama run gemma3:27b < /tmp/prompt_script.txt ## Detailed Summary of the WAN Show - April 26, 2024 This is a comprehensive summary of the April 26, 2024 episode of the WAN Show, covering a wide range of topics from product updates and industry news to behind-the-scenes company discussions and humorous tangents. **I. Introduction & Product News (0:00 - 10:00)** * The show opened with a discussion about the extended promotional period for the LTStore, offering free shipping on orders over $150, due to potential tariff increases. * Mentioned the ongoing launch of the new website for the Lab, the main focus is on refining and improving the Lab’s online experience. * Acknowledged the challenge of balancing transparency with protecting internal processes and the potential for misinterpretation of information shared publicly. **II. Product Updates & Discussions (10:00 - 30:00)** * **Budget Laptops:** The team reviewed budget laptops, highlighting the contrast between affordability and quality. * **NVIDIA A100 Server & LLM Integration:** Discussed the potential of using a high-end server (with an A100 GPU) for running large language models (LLMs) and the challenges of setting up such a system. * **NAS/Server Builds:** Highlighted the complexity of building and maintaining a robust home server, and the importance of choosing the right components. * **The Team's approach to setting up a remote location is in progress.** ... ``` It seems like layer spilling causes the model to lose a bit of focus.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#6808
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?