github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-17 13:47:08 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #6710] Docker: P8 State Power Usage double with 0.3.8+ #81906

New Issue
Closed
opened 2026-05-09 12:44:42 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-05-09 12:44:42 -05:00
Owner
Copy Link

Originally created by @t3chn0m4g3 on GitHub (Sep 9, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6710

Originally assigned to: @dhiltgen on GitHub.

What is the issue?

Versions of ollama/ollama < 0.3.8 had a P8 state power usage of about 4-5W. With 0.3.8+ this usage doubled to 9-10W.

Running two Nvidia RTX 4060 Ti and the issue is 100% reproducible. Resetting the driver with nvidia-smi -r and then starting ollama/ollama:0.3.7 shows the expected 4-5W in P8 state, while resetting and doing the same for ollama/ollama:0.3.8 and ollama/ollama:0.3.9 increases the P8 state to 9-10W again.

Happy to provide more information, just let me know.

OS

Docker

GPU

Nvidia

CPU

AMD

Ollama version

<=0.3.7 (works), >=0.3.8 (issue)

Originally created by @t3chn0m4g3 on GitHub (Sep 9, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6710 Originally assigned to: @dhiltgen on GitHub. ### What is the issue? Versions of `ollama/ollama` < 0.3.8 had a P8 state power usage of about 4-5W. With 0.3.8+ this usage doubled to 9-10W. Running two Nvidia RTX 4060 Ti and the issue is 100% reproducible. Resetting the driver with `nvidia-smi -r` and then starting `ollama/ollama:0.3.7` shows the expected 4-5W in P8 state, while resetting and doing the same for `ollama/ollama:0.3.8` and `ollama/ollama:0.3.9` increases the P8 state to 9-10W again. Happy to provide more information, just let me know. ### OS Docker ### GPU Nvidia ### CPU AMD ### Ollama version <=0.3.7 (works), >=0.3.8 (issue)
GiteaMirror added the bugnvidia labels 2026-05-09 12:44:43 -05:00
GiteaMirror commented 2026-05-09 12:44:46 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Sep 9, 2024):

Is that before loading any models, ie immediately after docker up? Because until a model is loaded, the only thing that ollama does with the GPU is check for its existence, which shouldn't be a constant power draw.

<!-- gh-comment-id:2338387884 --> @rick-github commented on GitHub (Sep 9, 2024): Is that before loading any models, ie immediately after `docker up`? Because until a model is loaded, the only thing that ollama does with the GPU is check for its existence, which shouldn't be a constant power draw.
GiteaMirror commented 2026-05-09 12:44:48 -05:00
Author
Owner
Copy Link

@t3chn0m4g3 commented on GitHub (Sep 9, 2024):

@rick-github It is even before a model is loaded. Loading the model, actively using it and then waiting for P8 state makes no difference regarding P8 state's power draw and the described change of power draw between 0.3.7 and 0.3.8/9.

<!-- gh-comment-id:2338398106 --> @t3chn0m4g3 commented on GitHub (Sep 9, 2024): @rick-github It is even before a model is loaded. Loading the model, actively using it and then waiting for P8 state makes no difference regarding P8 state's power draw and the described change of power draw between 0.3.7 and 0.3.8/9.
GiteaMirror commented 2026-05-09 12:44:48 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Sep 9, 2024):

On my 3080, 3080Ti and 4070 there's no difference in P8 state for 0.3.6 to 0.3.9.

What's the output of nvidia-smi -q -d POWER,TEMPERATURE,PERFORMANCE for the different ollama versions?

<!-- gh-comment-id:2338505759 --> @rick-github commented on GitHub (Sep 9, 2024): On my 3080, 3080Ti and 4070 there's no difference in P8 state for 0.3.6 to 0.3.9. What's the output of `nvidia-smi -q -d POWER,TEMPERATURE,PERFORMANCE` for the different ollama versions?
GiteaMirror commented 2026-05-09 12:44:49 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Sep 9, 2024):

Does nvidia-smi show the ollama executable as a process running on the GPU when idle?

In 0.3.7 we added cuda v12 support, but you mention that version has the lower power usage compared to 0.3.8. Skimming through the diffs between these versions doesn't show any obvious changes that could explain the regression.

<!-- gh-comment-id:2338511871 --> @dhiltgen commented on GitHub (Sep 9, 2024): Does `nvidia-smi` show the ollama executable as a process running on the GPU when idle? In 0.3.7 we added cuda v12 support, but you mention that version has the lower power usage compared to 0.3.8. Skimming through the diffs between these versions doesn't show any obvious changes that could explain the regression.
GiteaMirror commented 2026-05-09 12:44:49 -05:00
Author
Owner
Copy Link

@t3chn0m4g3 commented on GitHub (Sep 9, 2024):

@dhiltgen Thank you ❤️ I just found some more time to debug this.

I reviewed the commits for 0.3.7+ and like to confirm, there seems to be nothing that could possibly induce the observed behavior.

I also re-tested for all versions from 0.3.6+ and can confirm that all version show the same power usage in P8 which is around 9-10W once the ollama container is started. Just the start of the container (without active ollama executable) is enough to bump P8 from 4-5W to 9-10W.

Just stopping the container is not enough for P8 returning to 4-5W, it is necessary to reset the driver with nvidia-smi -r.

Solution: On the host nvidia-persistenced was running, once the process was killed and the driver reset everything is working as expected again.

Thanks @rick-github @dhiltgen for the debug help 🚀, very much appreciated!

<!-- gh-comment-id:2338768119 --> @t3chn0m4g3 commented on GitHub (Sep 9, 2024): @dhiltgen Thank you ❤️ I just found some more time to debug this. I reviewed the commits for `0.3.7+` and like to confirm, there seems to be nothing that could possibly induce the observed behavior. I also re-tested for all versions from `0.3.6+` and can confirm that all version show the same power usage in P8 which is around 9-10W once the `ollama` container is started. Just the start of the container (without active `ollama` executable) is enough to bump P8 from 4-5W to 9-10W. Just stopping the container is not enough for P8 returning to 4-5W, it is necessary to reset the driver with `nvidia-smi -r`. Solution: On the host `nvidia-persistenced` was running, once the process was killed and the driver reset everything is working as expected again. Thanks @rick-github @dhiltgen for the debug help 🚀, very much appreciated!
GiteaMirror commented 2026-05-09 12:44:49 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Sep 9, 2024):

My understanding is without nvidia-persistenced running, the uvm driver will eventually unload, and Ollama will stop working, but the failure mode is tricky to spot (if I remember correctly, we get an "unknown error" code 999 from one of the library APIs we call to discover the GPU.)

<!-- gh-comment-id:2338860058 --> @dhiltgen commented on GitHub (Sep 9, 2024): My understanding is without `nvidia-persistenced` running, the uvm driver will eventually unload, and Ollama will stop working, but the failure mode is tricky to spot (if I remember correctly, we get an "unknown error" code 999 from one of the library APIs we call to discover the GPU.)
GiteaMirror commented 2026-05-09 12:44:50 -05:00
Author
Owner
Copy Link

@t3chn0m4g3 commented on GitHub (Sep 10, 2024):

@dhiltgen Thanks for the info, just updated the Ollama monitoring accordingly and will update this issue and / or create a new one should this lead to a problem.

<!-- gh-comment-id:2340102104 --> @t3chn0m4g3 commented on GitHub (Sep 10, 2024): @dhiltgen Thanks for the info, just updated the Ollama monitoring accordingly and will update this issue and / or create a new one should this lead to a problem.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug nvidia
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#81906
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?