github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-21 05:01:48 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #2270] When I run a local model, GPU is used, but the CPU is 100% #1304

New Issue
Closed
opened 2026-04-12 11:07:37 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-04-12 11:07:37 -05:00
Owner
Copy Link

Originally created by @thugbobby on GitHub (Jan 30, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/2270

Originally assigned to: @dhiltgen on GitHub.

When I run a local model, GPU is used, but the CPU is 100%, and it will be crashed.
image
image

Originally created by @thugbobby on GitHub (Jan 30, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/2270 Originally assigned to: @dhiltgen on GitHub. When I run a local model, GPU is used, but the CPU is 100%, and it will be crashed. ![image](https://github.com/ollama/ollama/assets/68416779/2dc6dbf4-b786-4250-9996-20915a5b5ee5) ![image](https://github.com/ollama/ollama/assets/68416779/89c31672-2d47-4cd1-ad34-9e47fb2063af)
GiteaMirror added the gpu label 2026-04-12 11:07:37 -05:00
GiteaMirror commented 2026-04-12 11:07:39 -05:00
Author
Owner
Copy Link

@mehdiataei commented on GitHub (Jan 30, 2024):

Same issue. GPU is not used at all (the memory is allocated though)

<!-- gh-comment-id:1916824426 --> @mehdiataei commented on GitHub (Jan 30, 2024): Same issue. GPU is not used at all (the memory is allocated though)
GiteaMirror commented 2026-04-12 11:07:40 -05:00
Author
Owner
Copy Link

@ltomes commented on GitHub (Jan 30, 2024):

This has been brought up on this ticket as well: https://github.com/ollama/ollama/issues/1663
I have similar symptoms but using an A5000.

<!-- gh-comment-id:1917264420 --> @ltomes commented on GitHub (Jan 30, 2024): This has been brought up on this ticket as well: https://github.com/ollama/ollama/issues/1663 I have similar symptoms but using an A5000.
GiteaMirror commented 2026-04-12 11:07:41 -05:00
Author
Owner
Copy Link

@penouc commented on GitHub (Feb 3, 2024):

This seems to be a new version issue. I tried using ollma0.1.20 and found that the CPU's percentage could go over 100%, without crashing.
image

<!-- gh-comment-id:1925328957 --> @penouc commented on GitHub (Feb 3, 2024): This seems to be a new version issue. I tried using ollma0.1.20 and found that the CPU's percentage could go over 100%, without crashing. ![image](https://github.com/ollama/ollama/assets/1774022/6e03b496-786c-45f1-8919-215579fc6039)
GiteaMirror commented 2026-04-12 11:07:41 -05:00
Author
Owner
Copy Link

@easp commented on GitHub (Feb 3, 2024):

What model are you using?

<!-- gh-comment-id:1925433359 --> @easp commented on GitHub (Feb 3, 2024): What model are you using?
GiteaMirror commented 2026-04-12 11:07:42 -05:00
Author
Owner
Copy Link

@thugbobby commented on GitHub (Feb 8, 2024):

What model are you using?

yi:34b-chat

<!-- gh-comment-id:1933494512 --> @thugbobby commented on GitHub (Feb 8, 2024): > What model are you using? yi:34b-chat
GiteaMirror commented 2026-04-12 11:07:43 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Mar 12, 2024):

@thugbobby from your screenshot, it looks like python is using up most of your VRAM, leaving ollama with very little to fit in, so that implies it's loading only a small number of layers into the GPUs, and most of the work is being done by the CPU. Either load Ollama before your python app, or somehow configure python to use less VRAM, and you should get more layers loaded into the GPU. While we probably could load a few more layers from the looks of things, we try to be a little conservative in our memory prediction calculations, and we're continuing to refine those.

If you're still having problems, please share your server log and I'll re-open the issue.

<!-- gh-comment-id:1992615948 --> @dhiltgen commented on GitHub (Mar 12, 2024): @thugbobby from your screenshot, it looks like python is using up most of your VRAM, leaving ollama with very little to fit in, so that implies it's loading only a small number of layers into the GPUs, and most of the work is being done by the CPU. Either load Ollama before your python app, or somehow configure python to use less VRAM, and you should get more layers loaded into the GPU. While we probably could load a few more layers from the looks of things, we try to be a little conservative in our memory prediction calculations, and we're continuing to refine those. If you're still having problems, please share your server log and I'll re-open the issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label gpu
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#1304
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?