github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 08:02:14 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #2235] GPU RAM not released when exiting ollama run #63321

New Issue
Closed
opened 2026-05-03 12:59:17 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-05-03 12:59:17 -05:00
Owner
Copy Link

Originally created by @aseedb on GitHub (Jan 27, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/2235

Originally assigned to: @dhiltgen on GitHub.

I'm running ollama version 0.1.22 under Ubuntu and I installed it with the default procedure. After exiting a run using the command /exit the GPU RAM used by ollama is not released immediately. I either need to restart the ollama service, or wait for several minutes for that to occur. Is it an expected behavior?

Originally created by @aseedb on GitHub (Jan 27, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/2235 Originally assigned to: @dhiltgen on GitHub. I'm running ollama version 0.1.22 under Ubuntu and I installed it with the default procedure. After exiting a run using the command /exit the GPU RAM used by ollama is not released immediately. I either need to restart the ollama service, or wait for several minutes for that to occur. Is it an expected behavior?
GiteaMirror commented 2026-05-03 12:59:20 -05:00
Author
Owner
Copy Link

@easp commented on GitHub (Jan 27, 2024):

Sounds like the expected behavior. Ollama unloads the model after 5m of inactivity.

This will be configurable in an upcoming version.

<!-- gh-comment-id:1913358565 --> @easp commented on GitHub (Jan 27, 2024): Sounds like the expected behavior. Ollama unloads the model after 5m of inactivity. This will be configurable in an upcoming version.
GiteaMirror commented 2026-05-03 12:59:24 -05:00
Author
Owner
Copy Link

@jukofyork commented on GitHub (Jan 28, 2024):

Sounds like the expected behavior. Ollama unloads the model after 5m of inactivity.

This will be configurable in an upcoming version.

Unless this has been fixed in the last week or so (currently running 'main' pulled a few days ago), It still seems to hang on to a around 800 to 1500mb of VRAM for me even when the server unloads the model (even many hours after!). It seems to be a leak in the wrapped lllama.cpp server from what I could see.

This can be quite irritating if you push the number of layers offloaded to the limit as it will work fine with the 3mb-using "freshy crashed" Ollama server but then crash again when switching models and take 30s+ to recover.

<!-- gh-comment-id:1913717846 --> @jukofyork commented on GitHub (Jan 28, 2024): > Sounds like the expected behavior. Ollama unloads the model after 5m of inactivity. > > This will be configurable in an upcoming version. Unless this has been fixed in the last week or so (currently running 'main' pulled a few days ago), It still seems to hang on to a around 800 to 1500mb of VRAM for me even when the server unloads the model (even many hours after!). It seems to be a leak in the wrapped lllama.cpp server from what I could see. This can be quite irritating if you push the number of layers offloaded to the limit as it will work fine with the 3mb-using "freshy crashed" Ollama server but then crash again when switching models and take 30s+ to recover.
GiteaMirror commented 2026-05-03 12:59:27 -05:00
Author
Owner
Copy Link

@remy415 commented on GitHub (Jan 31, 2024):

Most AI servers have this problem, I ran into it a lot with stable-diffusion. I would implement frequent server restarts into an application you want to leave running indefinitely.

<!-- gh-comment-id:1918178846 --> @remy415 commented on GitHub (Jan 31, 2024): Most AI servers have this problem, I ran into it a lot with stable-diffusion. I would implement frequent server restarts into an application you want to leave running indefinitely.
GiteaMirror commented 2026-05-03 12:59:30 -05:00
Author
Owner
Copy Link

@aseedb commented on GitHub (Feb 5, 2024):

It still seems to hang on to a around 800 to 1500mb of VRAM for me even when the server unloads the model (even many hours after!).

Same here and yes it would be nice to have it fixed.

<!-- gh-comment-id:1927347490 --> @aseedb commented on GitHub (Feb 5, 2024): > It still seems to hang on to a around 800 to 1500mb of VRAM for me even when the server unloads the model (even many hours after!). Same here and yes it would be nice to have it fixed.
GiteaMirror commented 2026-05-03 12:59:31 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Mar 11, 2024):

Let's track this via #1848

We fixed a number of scenarios, but there are still some where GPU memory isn't properly freed when we go idle.

<!-- gh-comment-id:1989254932 --> @dhiltgen commented on GitHub (Mar 11, 2024): Let's track this via #1848 We fixed a number of scenarios, but there are still some where GPU memory isn't properly freed when we go idle.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#63321
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?