github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 12:48:05 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #11085] Auto-unload model or release GPU memory when idle #33072

New Issue
Closed
opened 2026-04-22 15:17:00 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-04-22 15:17:00 -05:00
Owner
Copy Link

Originally created by @barthwalsaurabh0 on GitHub (Jun 16, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/11085

Is your feature request related to a problem? Please describe.
Yes. When using ollama serve, models remain loaded in GPU memory even when not actively handling requests. This leads to unnecessary GPU usage, which can be problematic on shared systems or when switching between different workloads.

Describe the solution you'd like
Introduce a feature that automatically unloads a model or releases GPU memory after a configurable period of inactivity. This could include:

  • Monitoring for idle time (no incoming requests or inactivity on the model)
  • Unloading or stopping the model automatically after a timeout
  • Optionally reloading the model automatically when a new request arrives

This would function similarly to cache eviction or auto-scaling mechanisms.

Describe alternatives you’ve considered

  • Manually running ollama stop <model> to free GPU memory
  • Using ollama run instead of serve, but that doesn't support persistent API access
  • Writing a custom idle-time monitoring script to stop models automatically

Additional context
This feature would be very useful in environments with limited GPU resources, such as shared development machines, personal laptops, or multi-tenant servers.

Originally created by @barthwalsaurabh0 on GitHub (Jun 16, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/11085 **Is your feature request related to a problem? Please describe.** Yes. When using `ollama serve`, models remain loaded in GPU memory even when not actively handling requests. This leads to unnecessary GPU usage, which can be problematic on shared systems or when switching between different workloads. --- **Describe the solution you'd like** Introduce a feature that automatically unloads a model or releases GPU memory after a configurable period of inactivity. This could include: * Monitoring for idle time (no incoming requests or inactivity on the model) * Unloading or stopping the model automatically after a timeout * Optionally reloading the model automatically when a new request arrives This would function similarly to cache eviction or auto-scaling mechanisms. --- **Describe alternatives you’ve considered** * Manually running `ollama stop <model>` to free GPU memory * Using `ollama run` instead of `serve`, but that doesn't support persistent API access * Writing a custom idle-time monitoring script to stop models automatically --- **Additional context** This feature would be very useful in environments with limited GPU resources, such as shared development machines, personal laptops, or multi-tenant servers.
GiteaMirror added the feature request label 2026-04-22 15:17:01 -05:00
GiteaMirror commented 2026-04-22 15:17:03 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jun 16, 2025):

https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-keep-a-model-loaded-in-memory-or-make-it-unload-immediately

<!-- gh-comment-id:2975937195 --> @rick-github commented on GitHub (Jun 16, 2025): https://github.com/ollama/ollama/blob/main/docs/faq.md#how-do-i-keep-a-model-loaded-in-memory-or-make-it-unload-immediately
GiteaMirror commented 2026-04-22 15:17:05 -05:00
Author
Owner
Copy Link

@barthwalsaurabh0 commented on GitHub (Jun 23, 2025):

Thanks, missed that.

<!-- gh-comment-id:2995780788 --> @barthwalsaurabh0 commented on GitHub (Jun 23, 2025): Thanks, missed that.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#33072
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?