github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #11748] Ollama does not natively support offloading model weights to shared memory (system RAM or CPU memory) when dedicated GPU memory is full #69843

New Issue
Closed
opened 2026-05-04 19:32:48 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-05-04 19:32:48 -05:00
Owner
Copy Link

Originally created by @Hacoor on GitHub (Aug 6, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/11748

Originally created by @Hacoor on GitHub (Aug 6, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/11748
GiteaMirror commented 2026-05-04 19:32:49 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 6, 2025):

What operating system? It should offload automatically (depending on driver) on Windows, Linux users need to set an environment variable.

<!-- gh-comment-id:3160909214 --> @rick-github commented on GitHub (Aug 6, 2025): What operating system? It should offload automatically (depending on driver) on Windows, Linux users need to set an environment variable.
GiteaMirror commented 2026-05-04 19:32:50 -05:00
Author
Owner
Copy Link

@jhall39 commented on GitHub (Aug 6, 2025):

I get an error with larger models now.. gpt-oss:20b and even models that worked before like qwen3:30b-a3b on Windows 11 I get errors like 500 Internal Server Error: llama runner process has terminated: cudaMalloc failed: out of memory
I'm on 0.11.3

<!-- gh-comment-id:3161021230 --> @jhall39 commented on GitHub (Aug 6, 2025): I get an error with larger models now.. gpt-oss:20b and even models that worked before like qwen3:30b-a3b on Windows 11 I get errors like 500 Internal Server Error: llama runner process has terminated: cudaMalloc failed: out of memory I'm on 0.11.3
GiteaMirror commented 2026-05-04 19:32:50 -05:00
Author
Owner
Copy Link

@Hacoor commented on GitHub (Aug 6, 2025):

I get an error with larger models now.. gpt-oss:20b and even models that worked before like qwen3:30b-a3b on Windows 11 I get errors like 500 Internal Server Error: llama runner process has terminated: cudaMalloc failed: out of memory I'm on 0.11.3

check your gpu vram it must be at least 16gb or ram must be above 20gb to run it

<!-- gh-comment-id:3161053609 --> @Hacoor commented on GitHub (Aug 6, 2025): > I get an error with larger models now.. gpt-oss:20b and even models that worked before like qwen3:30b-a3b on Windows 11 I get errors like 500 Internal Server Error: llama runner process has terminated: cudaMalloc failed: out of memory I'm on 0.11.3 check your gpu vram it must be at least 16gb or ram must be above 20gb to run it
GiteaMirror commented 2026-05-04 19:32:50 -05:00
Author
Owner
Copy Link

@jhall39 commented on GitHub (Aug 6, 2025):

I have a 5070ti with 16gb ram and also 64GB system ram as well. qwen3:30b-a3b ran perfectly fine on an older version of ollama before the gui was added.

<!-- gh-comment-id:3161407344 --> @jhall39 commented on GitHub (Aug 6, 2025): I have a 5070ti with 16gb ram and also 64GB system ram as well. qwen3:30b-a3b ran perfectly fine on an older version of ollama before the gui was added.
GiteaMirror commented 2026-05-04 19:32:51 -05:00
Author
Owner
Copy Link

@azomDev commented on GitHub (Aug 10, 2025):

@jhall39 This appears to be related to #11676

<!-- gh-comment-id:3172342096 --> @azomDev commented on GitHub (Aug 10, 2025): @jhall39 This appears to be related to #11676
GiteaMirror commented 2026-05-04 19:32:52 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Sep 1, 2025):

The memory management in recent ollama releases may have resolved this. Upgrade and add a comment if the problem persists.

<!-- gh-comment-id:3243159227 --> @rick-github commented on GitHub (Sep 1, 2025): The memory management in recent ollama releases may have resolved this. Upgrade and add a comment if the problem persists.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#69843
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?