github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 21:25:46 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #15024] Qwen3.5 (GGUF via Ollama 0.18.0) causes reproducible GPU driver instability under moderate context load on RTX 3090 (WSL2) #71711

New Issue
Open
opened 2026-05-05 02:23:40 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-05-05 02:23:40 -05:00
Owner
Copy Link

Originally created by @amoselroy on GitHub (Mar 23, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/15024

What is the issue?

Summary

Running Qwen3.5 (Q4_K_M GGUF via Ollama 0.18.0) on an RTX 3090 (24 GB) in WSL2 (Ubuntu 24.04) on Windows 11 causes reproducible GPU driver instability when the model processes moderately populated input (~12k characters).

The issue persists after:

clean driver reinstall (DDU)

CMOS reset

fresh Ollama state (~/.ollama reset)

model re-download

The instability escalates beyond application failure and can corrupt GPU initialization state, leading to:

black screen (no POST)

required CMOS reset + GPU reseat to recover

Environment

GPU: NVIDIA RTX 3090 (24 GB VRAM)

Driver: 595.79 (Studio, clean install via DDU)

OS: Windows 11 + WSL2

WSL distro: Ubuntu 24.04

Ollama: 0.18.0

Model: qwen3.5:latest (GGUF, Q4_K_M, ~9.7B)

Control model: llama3.2:3b (stable)

Reproduction (minimal)

  1. Start Ollama (WSL)

ollama serve

  1. Confirm model works with tiny prompt

ollama run qwen3.5:latest "Reply with exactly: ok"

→ Works reliably

  1. Populate moderate input (~12k chars)

head -c 12000 filler.txt | ollama run qwen3.5:latest

→ Observed:

GPU instability

driver crash (nvlddmkm Event ID 14 / 153)

in some cases system-level failure

Observations

Tiny prompts: stable

Declared large context (e.g., 32k) with tiny input: stable

Moderate populated input (~12k): triggers crash

llama3.2:3b under same conditions: stable

CUDA container benchmarks: stable

Isolation Performed

Eliminated as root causes:

❌ OpenClaw (crash reproduced in Ollama directly)

❌ Corrupted model weights (re-pulled)

❌ Persistent Ollama user state (~/.ollama reset)

❌ Windows vs WSL confusion (confirmed WSL server)

❌ Short-lived cache/session persistence (crashes after reboot)

❌ General GPU instability (other workloads stable)

Critical Finding

The failure is triggered by actual context population, not declared context size.

Model loads fine

Model runs small prompts fine

Model crashes when KV-cache / attention memory expands under real input

Escalation Behavior

Repeated triggering led to:

GPU driver crashes (TDR)

eventual failure to initialize GPU at boot

required:

GPU removal

CMOS reset

reseating

Working Hypothesis

Likely issue in:

Qwen3.5 inference path under GGUF/Ollama runtime, specifically during KV-cache expansion / attention memory allocation

Possible contributing factors:

CUDA kernel behavior under memory pressure

KV-cache shifting / reprocessing logic

interaction with WSL GPU virtualization layer

Workaround

Stable operation achieved by:

FROM qwen3.5:latest
PARAMETER num_ctx 8192
PARAMETER num_batch 32
PARAMETER num_gpu 1

Plus:

avoiding large single-shot prompt ingestion

using incremental context accumulation

Questions for Maintainers

  1. Is Qwen3.5 GGUF known to have instability under KV-cache expansion in Ollama?

  2. Are there known issues with:

KV-cache shifting

large prompt ingestion

CUDA kernels in Qwen models?

  1. Is there a recommended safe configuration for:

context size vs VRAM

batch size

memory strategy?

Impact

This is not just an application crash:

The workload can escalate into GPU driver failure and hardware initialization instability

Reproducibility

✔ Reproducible after full system reset
✔ Independent of user-level state
✔ Specific to Qwen3.5 (not observed with Llama)

Relevant log output

OS

No response

GPU

No response

CPU

No response

Ollama version

No response

Originally created by @amoselroy on GitHub (Mar 23, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/15024 ### What is the issue? Summary Running Qwen3.5 (Q4_K_M GGUF via Ollama 0.18.0) on an RTX 3090 (24 GB) in WSL2 (Ubuntu 24.04) on Windows 11 causes reproducible GPU driver instability when the model processes moderately populated input (~12k characters). The issue persists after: clean driver reinstall (DDU) CMOS reset fresh Ollama state (~/.ollama reset) model re-download The instability escalates beyond application failure and can corrupt GPU initialization state, leading to: black screen (no POST) required CMOS reset + GPU reseat to recover --- Environment GPU: NVIDIA RTX 3090 (24 GB VRAM) Driver: 595.79 (Studio, clean install via DDU) OS: Windows 11 + WSL2 WSL distro: Ubuntu 24.04 Ollama: 0.18.0 Model: qwen3.5:latest (GGUF, Q4_K_M, ~9.7B) Control model: llama3.2:3b (stable) --- Reproduction (minimal) 1. Start Ollama (WSL) ollama serve 2. Confirm model works with tiny prompt ollama run qwen3.5:latest "Reply with exactly: ok" → Works reliably --- 3. Populate moderate input (~12k chars) head -c 12000 filler.txt | ollama run qwen3.5:latest → Observed: GPU instability driver crash (nvlddmkm Event ID 14 / 153) in some cases system-level failure --- Observations Tiny prompts: stable Declared large context (e.g., 32k) with tiny input: stable Moderate populated input (~12k): triggers crash llama3.2:3b under same conditions: stable CUDA container benchmarks: stable --- Isolation Performed Eliminated as root causes: ❌ OpenClaw (crash reproduced in Ollama directly) ❌ Corrupted model weights (re-pulled) ❌ Persistent Ollama user state (~/.ollama reset) ❌ Windows vs WSL confusion (confirmed WSL server) ❌ Short-lived cache/session persistence (crashes after reboot) ❌ General GPU instability (other workloads stable) --- Critical Finding > The failure is triggered by actual context population, not declared context size. Model loads fine Model runs small prompts fine Model crashes when KV-cache / attention memory expands under real input --- Escalation Behavior Repeated triggering led to: GPU driver crashes (TDR) eventual failure to initialize GPU at boot required: GPU removal CMOS reset reseating --- Working Hypothesis Likely issue in: > Qwen3.5 inference path under GGUF/Ollama runtime, specifically during KV-cache expansion / attention memory allocation Possible contributing factors: CUDA kernel behavior under memory pressure KV-cache shifting / reprocessing logic interaction with WSL GPU virtualization layer --- Workaround Stable operation achieved by: FROM qwen3.5:latest PARAMETER num_ctx 8192 PARAMETER num_batch 32 PARAMETER num_gpu 1 Plus: avoiding large single-shot prompt ingestion using incremental context accumulation --- Questions for Maintainers 1. Is Qwen3.5 GGUF known to have instability under KV-cache expansion in Ollama? 2. Are there known issues with: KV-cache shifting large prompt ingestion CUDA kernels in Qwen models? 3. Is there a recommended safe configuration for: context size vs VRAM batch size memory strategy? Impact This is not just an application crash: > The workload can escalate into GPU driver failure and hardware initialization instability Reproducibility ✔ Reproducible after full system reset ✔ Independent of user-level state ✔ Specific to Qwen3.5 (not observed with Llama) ### Relevant log output ```shell ``` ### OS _No response_ ### GPU _No response_ ### CPU _No response_ ### Ollama version _No response_
GiteaMirror added the bugneeds more info labels 2026-05-05 02:23:40 -05:00
GiteaMirror commented 2026-05-05 02:23:42 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Mar 23, 2026):

Does the issue persist if you run native Windows ollama? Context buffer shifting due to space exhaustion is known to sometimes cause a model to lose coherence, but crashing the system is uncommon. Some previous instances of system crashing were caused by subtle problem with the GPU, running a VRAM test (OCCT, cuda_memtest) might give some insight, or at least rule out a GPU problem.

<!-- gh-comment-id:4114298349 --> @rick-github commented on GitHub (Mar 23, 2026): Does the issue persist if you run native Windows ollama? Context buffer shifting due to space exhaustion is known to sometimes cause a model to lose coherence, but crashing the system is uncommon. Some previous instances of system crashing were caused by subtle problem with the GPU, running a VRAM test ([OCCT](https://www.ocbase.com/occt/personal), [cuda_memtest](https://github.com/ComputationalRadiationPhysics/cuda_memtest)) might give some insight, or at least rule out a GPU problem.
GiteaMirror commented 2026-05-05 02:23:42 -05:00
Author
Owner
Copy Link

@amoselroy commented on GitHub (Mar 24, 2026):

I will conduct Cuda memtest when I get a chance. Yeah, the crashes were bad
to the point that eventually I was unable to boot the PC at all without
completely unseating the GPU and resetting the bios. I don't know about
running Ollama natively in Windows as I don't have use for it right now, I
use it purely for my Open Claw under WSL.

On Mon, Mar 23, 2026 at 7:10 PM frob @.***> wrote:

rick-github left a comment (ollama/ollama#15024)
https://github.com/ollama/ollama/issues/15024#issuecomment-4114298349

Does the issue persist if you run native Windows ollama? Context buffer
shifting due to space exhaustion is known to sometimes cause a model to
lose coherence, but crashing the system is uncommon. Some previous
instances of system crashing were caused by subtle problem with the GPU,
running a VRAM test (OCCT https://www.ocbase.com/occt/personal,
cuda_memtest
https://github.com/ComputationalRadiationPhysics/cuda_memtest) might
give some insight, or at least rule out a GPU problem.

—
Reply to this email directly, view it on GitHub
https://github.com/ollama/ollama/issues/15024?email_source=notifications&email_token=AIOUCGOHLJVDU4SZPETFYZL4SG77NA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJRGQZDSOBTGQ42M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-4114298349,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIOUCGKLOIJKRX5EGG2HJQD4SG77NAVCNFSM6AAAAACW4WDXYKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DCMJUGI4TQMZUHE
.
You are receiving this because you authored the thread.Message ID:
@.***>

<!-- gh-comment-id:4119596312 --> @amoselroy commented on GitHub (Mar 24, 2026): I will conduct Cuda memtest when I get a chance. Yeah, the crashes were bad to the point that eventually I was unable to boot the PC at all without completely unseating the GPU and resetting the bios. I don't know about running Ollama natively in Windows as I don't have use for it right now, I use it purely for my Open Claw under WSL. On Mon, Mar 23, 2026 at 7:10 PM frob ***@***.***> wrote: > *rick-github* left a comment (ollama/ollama#15024) > <https://github.com/ollama/ollama/issues/15024#issuecomment-4114298349> > > Does the issue persist if you run native Windows ollama? Context buffer > shifting due to space exhaustion is known to sometimes cause a model to > lose coherence, but crashing the system is uncommon. Some previous > instances of system crashing were caused by subtle problem with the GPU, > running a VRAM test (OCCT <https://www.ocbase.com/occt/personal>, > cuda_memtest > <https://github.com/ComputationalRadiationPhysics/cuda_memtest>) might > give some insight, or at least rule out a GPU problem. > > — > Reply to this email directly, view it on GitHub > <https://github.com/ollama/ollama/issues/15024?email_source=notifications&email_token=AIOUCGOHLJVDU4SZPETFYZL4SG77NA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIMJRGQZDSOBTGQ42M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-4114298349>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AIOUCGKLOIJKRX5EGG2HJQD4SG77NAVCNFSM6AAAAACW4WDXYKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DCMJUGI4TQMZUHE> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >
GiteaMirror commented 2026-05-05 02:23:43 -05:00
Author
Owner
Copy Link

@jagmarques commented on GitHub (Apr 7, 2026):

The GPU driver instability under moderate context load on RTX 3090 / WSL2 is likely hitting the KV cache memory ceiling — the cache grows linearly with context length and at some point you're triggering OOM pressure that destabilizes the driver stack under WSL2's memory virtualization.

A few things to try:

  1. Reduce context length () to confirm the hypothesis — if instability disappears, it's definitely KV cache pressure.
  2. Use Q4 KV quantization if your GGUF build supports — cuts KV memory ~4x, which should push the OOM threshold well above normal usage.
  3. WSL2-specific: set in to give WSL2 a hard memory ceiling before it destabilizes the host driver.

For longer-term context extension without driver issues, KV cache compression is the right tool. NexusQuant achieves 7–10x KV compression training-free (E8 lattice quantization), though it requires PyTorch-based inference rather than GGUF currently. If this gets integrated into llama.cpp/ollama it would let 3090-class hardware handle 500K+ token contexts without hitting the memory wall.

<!-- gh-comment-id:4201232013 --> @jagmarques commented on GitHub (Apr 7, 2026): The GPU driver instability under moderate context load on RTX 3090 / WSL2 is likely hitting the KV cache memory ceiling — the cache grows linearly with context length and at some point you're triggering OOM pressure that destabilizes the driver stack under WSL2's memory virtualization. A few things to try: 1. **Reduce context length** () to confirm the hypothesis — if instability disappears, it's definitely KV cache pressure. 2. **Use Q4 KV quantization** if your GGUF build supports — cuts KV memory ~4x, which should push the OOM threshold well above normal usage. 3. **WSL2-specific**: set in to give WSL2 a hard memory ceiling before it destabilizes the host driver. For longer-term context extension without driver issues, KV cache compression is the right tool. [NexusQuant](https://github.com/jagmarques/nexusquant) achieves 7–10x KV compression training-free (E8 lattice quantization), though it requires PyTorch-based inference rather than GGUF currently. If this gets integrated into llama.cpp/ollama it would let 3090-class hardware handle 500K+ token contexts without hitting the memory wall.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug needs more info
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#71711
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?