github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 13:19:40 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #9658] API to get performance status/information about GPU/CPU of instance #52814

New Issue
Open
opened 2026-04-29 00:58:14 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-04-29 00:58:14 -05:00
Owner
Copy Link

Originally created by @trollkarlen on GitHub (Mar 11, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/9658

Originally assigned to: @dhiltgen on GitHub.

This is a feature request but may mitigate instances that break due to current and feature bugs.

I use a proxy infront of my ollama instances to make it multi user/requests.
But some times the ollama server loses the connection with the GPU, and then the perfomance reduces alot.

This happens some times due to the cgroup issue that can be mitigated with the docker/daemon.json

"exec-opts": [
        "native.cgroupdriver=cgroupfs"
    ]

And somtimes due to other reasons(GPU hangs mm).

So it would be nice to be able to query the instance throught the API.
To get the performance of the node CPU/GPU wise, this way the proxy can detect the performance of the instances and detect when performance decline. This data can be used from both a proxy and/or client.

This feature request is on the same theme and maybe could be combined with this request, to have one place to get information about the node.
https://github.com/ollama/ollama/issues/2004

Originally created by @trollkarlen on GitHub (Mar 11, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/9658 Originally assigned to: @dhiltgen on GitHub. This is a feature request but may mitigate instances that break due to current and feature bugs. I use a proxy infront of my ollama instances to make it multi user/requests. But some times the ollama server loses the connection with the GPU, and then the perfomance reduces alot. This happens some times due to the cgroup issue that can be mitigated with the docker/daemon.json ``` json "exec-opts": [ "native.cgroupdriver=cgroupfs" ] ``` And somtimes due to other reasons(GPU hangs mm). So it would be nice to be able to query the instance throught the API. To get the performance of the node CPU/GPU wise, this way the proxy can detect the performance of the instances and detect when performance decline. This data can be used from both a proxy and/or client. This feature request is on the same theme and maybe could be combined with this request, to have one place to get information about the node. https://github.com/ollama/ollama/issues/2004
GiteaMirror added the feature request label 2026-04-29 00:58:14 -05:00
GiteaMirror commented 2026-04-29 00:58:15 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Mar 11, 2025):

https://github.com/ollama/ollama/issues/3144

WRT the GPU hanging, it might be that the model has lost coherence and is "rambling". This can be limited by setting num_predict.

<!-- gh-comment-id:2715525562 --> @rick-github commented on GitHub (Mar 11, 2025): https://github.com/ollama/ollama/issues/3144 WRT the GPU hanging, it might be that the model has lost coherence and is "rambling". This can be limited by setting [`num_predict`](https://github.com/ollama/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values:~:text=stop%20%22AI%20assistant%3A%22-,num_predict,-Maximum%20number%20of).
GiteaMirror commented 2026-04-29 00:58:16 -05:00
Author
Owner
Copy Link

@trollkarlen commented on GitHub (Mar 11, 2025):

#3144

Looks like the no gpus and no gpus avalible is missing in metrics, but i guess it can be added.
Also it looks like the feature request is stalled :/

WRT the GPU hanging, it might be that the model has lost coherence and is "rambling". This can be limited by setting num_predict.

Thanx will look in to the num_predict, and see if it helps mitigate the issue with offline GPU:s

<!-- gh-comment-id:2715859007 --> @trollkarlen commented on GitHub (Mar 11, 2025): > [#3144](https://github.com/ollama/ollama/issues/3144) Looks like the no gpus and no gpus avalible is missing in metrics, but i guess it can be added. Also it looks like the feature request is stalled :/ > WRT the GPU hanging, it might be that the model has lost coherence and is "rambling". This can be limited by setting [`num_predict`](https://github.com/ollama/ollama/blob/main/docs/modelfile.md#valid-parameters-and-values:~:text=stop%20%22AI%20assistant%3A%22-,num_predict,-Maximum%20number%20of). Thanx will look in to the num_predict, and see if it helps mitigate the issue with offline GPU:s
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#52814
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?