github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 13:19:40 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #15863] RTX 4060 CUDA illegal memory access on tool-calling requests (0.21.2) #87796

New Issue
Open
opened 2026-05-10 06:24:13 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-10 06:24:13 -05:00
Owner
Copy Link

Originally created by @cryptrix1598 on GitHub (Apr 28, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/15863

Bug Description

When using Ollama 0.21.2 with an NVIDIA RTX 4060 (8GB VRAM) on Windows 11, the model runner crashes with a CUDA illegal memory access error specifically when handling tool-calling requests. Simple inference requests (e.g., ollama run qwen2.5:7b "say hello") work perfectly fine, but any request involving tool use causes an immediate crash.

Error Message

CUDA error: an illegal memory access was encountered
current device: 0, in function ggml_backend_cuda_synchronize at ggml-cuda.cu:2981
cudaStreamSynchronize(cuda_ctx->stream())
ggml-cuda.cu:94: CUDA error

System Information

  • OS: Windows 11
  • GPU: NVIDIA GeForce RTX 4060 8GB VRAM
  • GPU Driver Version: 591.86 (also tested with latest driver, same issue)
  • CUDA Version: 13.1
  • Ollama Version: 0.21.2
  • RAM: 32GB
  • Model: qwen2.5:7b

Steps to Reproduce

  1. Install Ollama 0.21.2 on Windows 11 with RTX 4060
  2. Pull qwen2.5:7b
  3. Run a simple inference request — works fine
  4. Run any tool-calling request (e.g., via OpenClaw gateway) — crashes immediately with CUDA illegal memory access

Expected Behavior

Tool-calling requests should execute successfully using GPU acceleration without crashing the model runner.

Actual Behavior

Model runner crashes immediately on tool-calling requests with CUDA illegal memory access error. The runner terminates with exit status 1. Simple inference works fine — the crash is specifically triggered by tool-calling workloads.

Additional Context

  • Downgrading to Ollama 0.19.0 does not fix the issue — that version fails to discover the GPU entirely (timeout during GPU discovery)
  • Setting OLLAMA_NUM_GPU=0 to force CPU mode resolves the crash but makes the tool unusable due to extremely slow inference
  • Setting OLLAMA_GPU_OVERHEAD to 1GB and 2GB does not resolve the issue
  • The crash occurs consistently and reproducibly every single time a tool-calling request is made
  • Tested with both Game Ready Driver 591.86 and latest available driver — same result on both
  • Issue appears to be specific to tool-calling inference patterns that use more complex CUDA memory access patterns than simple generation
Originally created by @cryptrix1598 on GitHub (Apr 28, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/15863 ## Bug Description When using Ollama 0.21.2 with an NVIDIA RTX 4060 (8GB VRAM) on Windows 11, the model runner crashes with a CUDA illegal memory access error specifically when handling tool-calling requests. Simple inference requests (e.g., `ollama run qwen2.5:7b "say hello"`) work perfectly fine, but any request involving tool use causes an immediate crash. ## Error Message CUDA error: an illegal memory access was encountered current device: 0, in function ggml_backend_cuda_synchronize at ggml-cuda.cu:2981 cudaStreamSynchronize(cuda_ctx->stream()) ggml-cuda.cu:94: CUDA error ## System Information - **OS:** Windows 11 - **GPU:** NVIDIA GeForce RTX 4060 8GB VRAM - **GPU Driver Version:** 591.86 (also tested with latest driver, same issue) - **CUDA Version:** 13.1 - **Ollama Version:** 0.21.2 - **RAM:** 32GB - **Model:** qwen2.5:7b ## Steps to Reproduce 1. Install Ollama 0.21.2 on Windows 11 with RTX 4060 2. Pull qwen2.5:7b 3. Run a simple inference request — works fine 4. Run any tool-calling request (e.g., via OpenClaw gateway) — crashes immediately with CUDA illegal memory access ## Expected Behavior Tool-calling requests should execute successfully using GPU acceleration without crashing the model runner. ## Actual Behavior Model runner crashes immediately on tool-calling requests with CUDA illegal memory access error. The runner terminates with exit status 1. Simple inference works fine — the crash is specifically triggered by tool-calling workloads. ## Additional Context - Downgrading to Ollama 0.19.0 does not fix the issue — that version fails to discover the GPU entirely (timeout during GPU discovery) - Setting OLLAMA_NUM_GPU=0 to force CPU mode resolves the crash but makes the tool unusable due to extremely slow inference - Setting OLLAMA_GPU_OVERHEAD to 1GB and 2GB does not resolve the issue - The crash occurs consistently and reproducibly every single time a tool-calling request is made - Tested with both Game Ready Driver 591.86 and latest available driver — same result on both - Issue appears to be specific to tool-calling inference patterns that use more complex CUDA memory access patterns than simple generation
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#87796
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?