github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 05:05:37 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #3975] When used, it is always cpu full instead of gpu full, and gpu usage is almost zero #28218

New Issue
Closed
opened 2026-04-22 06:07:53 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-04-22 06:07:53 -05:00
Owner
Copy Link

Originally created by @KritoAndAsuna on GitHub (Apr 27, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3975

Originally assigned to: @dhiltgen on GitHub.

What is the issue?

When used, it is always cpu full instead of gpu full, and gpu usage is almost zero

OS

Windows

GPU

AMD

CPU

AMD

Ollama version

0.1.32

Originally created by @KritoAndAsuna on GitHub (Apr 27, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3975 Originally assigned to: @dhiltgen on GitHub. ### What is the issue? When used, it is always cpu full instead of gpu full, and gpu usage is almost zero ### OS Windows ### GPU AMD ### CPU AMD ### Ollama version 0.1.32
GiteaMirror added the needs more info label 2026-04-22 06:07:53 -05:00
GiteaMirror commented 2026-04-22 06:07:57 -05:00
Author
Owner
Copy Link

@helium729 commented on GitHub (Apr 28, 2024):

this usually happens when you VRAM is not enough

<!-- gh-comment-id:2081510185 --> @helium729 commented on GitHub (Apr 28, 2024): this usually happens when you VRAM is not enough
GiteaMirror commented 2026-04-22 06:07:58 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (May 2, 2024):

@KritoAndAsuna we'll need some more information. What type of GPU do you have, and how much VRAM does it have? What model are you trying to load? As @helium729 mentioned, if you're trying to load a model that doesn't fit in VRAM, some of it will be loaded in system memory and will be processed by the CPU, which is typically much slower than the GPU, so the GPU winds up waiting most of the time for the CPU to process its portion. If you look in the server log, you'll see a line that looks something like this:

llm_load_tensors: offloaded 33/33 layers to GPU
<!-- gh-comment-id:2091205112 --> @dhiltgen commented on GitHub (May 2, 2024): @KritoAndAsuna we'll need some more information. What type of GPU do you have, and how much VRAM does it have? What model are you trying to load? As @helium729 mentioned, if you're trying to load a model that doesn't fit in VRAM, some of it will be loaded in system memory and will be processed by the CPU, which is typically much slower than the GPU, so the GPU winds up waiting most of the time for the CPU to process its portion. If you look in the server log, you'll see a line that looks something like this: ``` llm_load_tensors: offloaded 33/33 layers to GPU ```
GiteaMirror commented 2026-04-22 06:07:59 -05:00
Author
Owner
Copy Link

@GreatBigWhiteWorld commented on GitHub (May 3, 2024):

@KritoAndAsuna we'll need some more information. What type of GPU do you have, and how much VRAM does it have? What model are you trying to load? As @helium729 mentioned, if you're trying to load a model that doesn't fit in VRAM, some of it will be loaded in system memory and will be processed by the CPU, which is typically much slower than the GPU, so the GPU winds up waiting most of the time for the CPU to process its portion. If you look in the server log, you'll see a line that looks something like this:

llm_load_tensors: offloaded 33/33 layers to GPU

I run llama3(8B) on a machine with Nvidia 4080 super (16G VRAM), Ryzen 7800x3d and 32G ram.
Ollama still uses 100% CPU while GPU temp stays unchanged (while GPU fans sit still). Is this GPU still not powerful enough, not even helping a bit the task at least?

When I do sudo journalctl -u ollama | grep llm_load_tensors,
I get lots of messages like:

llm_load_tensors: ggml ctx size =    0.15 MiB
llm_load_tensors:        CPU buffer size =  4437.80 MiB
<!-- gh-comment-id:2093193185 --> @GreatBigWhiteWorld commented on GitHub (May 3, 2024): > @KritoAndAsuna we'll need some more information. What type of GPU do you have, and how much VRAM does it have? What model are you trying to load? As @helium729 mentioned, if you're trying to load a model that doesn't fit in VRAM, some of it will be loaded in system memory and will be processed by the CPU, which is typically much slower than the GPU, so the GPU winds up waiting most of the time for the CPU to process its portion. If you look in the server log, you'll see a line that looks something like this: > > ``` > llm_load_tensors: offloaded 33/33 layers to GPU > ``` I run llama3(8B) on a machine with Nvidia 4080 super (16G VRAM), Ryzen 7800x3d and 32G ram. Ollama still uses 100% CPU while GPU temp stays unchanged (while GPU fans sit still). Is this GPU still not powerful enough, not even helping a bit the task at least? When I do `sudo journalctl -u ollama | grep llm_load_tensors`, I get lots of messages like: ``` llm_load_tensors: ggml ctx size = 0.15 MiB llm_load_tensors: CPU buffer size = 4437.80 MiB ```
GiteaMirror commented 2026-04-22 06:08:00 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (May 3, 2024):

Can you share more of the server log? It sounds like it's not detecting your GPU for some reason and using the CPU runner instead.

<!-- gh-comment-id:2093379096 --> @dhiltgen commented on GitHub (May 3, 2024): Can you share more of the server log? It sounds like it's not detecting your GPU for some reason and using the CPU runner instead.
GiteaMirror commented 2026-04-22 06:08:01 -05:00
Author
Owner
Copy Link

@GreatBigWhiteWorld commented on GitHub (May 4, 2024):

Can you share more of the server log? It sounds like it's not detecting your GPU for some reason and using the CPU runner instead.

I did some research and added 'ollama' user to the 'video' group. Now in the ollama service log it has 'GPU detected' instead of just 'detecting GPU'. Running ollama task is also much faster and brings GPU temp up.
Is this related to how I installed ollama cause I installed it following the "manual install" steps, which don't involve adding ollama to the video group. I wonder what else I might have missed when not using the installation script.

<!-- gh-comment-id:2093960867 --> @GreatBigWhiteWorld commented on GitHub (May 4, 2024): > Can you share more of the server log? It sounds like it's not detecting your GPU for some reason and using the CPU runner instead. I did some research and added 'ollama' user to the 'video' group. Now in the ollama service log it has 'GPU detected' instead of just 'detecting GPU'. Running ollama task is also much faster and brings GPU temp up. Is this related to how I installed ollama cause I installed it following the "manual install" steps, which don't involve adding ollama to the video group. I wonder what else I might have missed when not using the installation script.
GiteaMirror commented 2026-04-22 06:08:02 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (May 4, 2024):

Glad to hear you found the problem @GreatBigWhiteWorld. We recommend using the install script to make sure everything gets set up correctly, but you can inspect the script itself to see what all it does for manual setups.

Given your scenario was linux and unix permission based, it's unlikely related to @KritoAndAsuna's problem on windows.

<!-- gh-comment-id:2094374287 --> @dhiltgen commented on GitHub (May 4, 2024): Glad to hear you found the problem @GreatBigWhiteWorld. We recommend using the install script to make sure everything gets set up correctly, but you can inspect the [script itself](https://github.com/ollama/ollama/blob/main/scripts/install.sh) to see what all it does for manual setups. Given your scenario was linux and unix permission based, it's unlikely related to @KritoAndAsuna's problem on windows.
GiteaMirror commented 2026-04-22 06:08:04 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (May 21, 2024):

@KritoAndAsuna if you're still having problems, please let us know. The new ollama ps command will help show how much of the model is loaded into GPU. Performance will be significantly impacted the more of the model is running on the CPU, to the point that the GPU will appear idle as it waits for the CPU to keep up.

<!-- gh-comment-id:2123189099 --> @dhiltgen commented on GitHub (May 21, 2024): @KritoAndAsuna if you're still having problems, please let us know. The new `ollama ps` command will help show how much of the model is loaded into GPU. Performance will be significantly impacted the more of the model is running on the CPU, to the point that the GPU will appear idle as it waits for the CPU to keep up.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label needs more info
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#28218
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?