github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #6152] I read that ollama now supports AMD GPUs but it's not using it #3842

New Issue
Closed
opened 2026-04-12 14:40:49 -05:00 by GiteaMirror · 16 comments
GiteaMirror commented 2026-04-12 14:40:49 -05:00
Owner
Copy Link

Originally created by @klausbu on GitHub (Aug 3, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6152

Originally assigned to: @dhiltgen on GitHub.

I installed ollama on ubuntu 22.04 with AMD ROCm installed. The 6700M GPU with 10GB RAM runs fine and is used by simulation programs and stable diffusion. Ollama uses only the CPU and requires 9GB RAM. I read that ollama now supports AMD GPUs but it's not using it on my setup.

OS: ubuntu 22.04

AMD ROCm setup in .bashrc
export HSA_OVERRIDE_GFX_VERSION=10.3.0
export HIP_VISIBLE_DEVICES=0

How to enable my AMD GPU to accelerate ollama?

Originally created by @klausbu on GitHub (Aug 3, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6152 Originally assigned to: @dhiltgen on GitHub. I installed ollama on ubuntu 22.04 with AMD ROCm installed. The 6700M GPU with 10GB RAM runs fine and is used by simulation programs and stable diffusion. Ollama uses only the CPU and requires 9GB RAM. I read that ollama now supports AMD GPUs but it's not using it on my setup. OS: ubuntu 22.04 AMD ROCm setup in .bashrc export HSA_OVERRIDE_GFX_VERSION=10.3.0 export HIP_VISIBLE_DEVICES=0 How to enable my AMD GPU to accelerate ollama?
GiteaMirror added the questionamd labels 2026-04-12 14:40:49 -05:00
GiteaMirror commented 2026-04-12 14:40:50 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 3, 2024):

Server logs may help in debugging.

<!-- gh-comment-id:2266794744 --> @rick-github commented on GitHub (Aug 3, 2024): [Server logs](https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md#how-to-troubleshoot-issues) may help in debugging.
GiteaMirror commented 2026-04-12 14:40:50 -05:00
Author
Owner
Copy Link

@klausbu commented on GitHub (Aug 3, 2024):

ok, the AMD 6700M GPU is not supported export HSA_OVERRIDE_GFX_VERSION=10.3.0 doesn't help. Unfortunately, the notebook RDNA2 GPUs are all gfx1031 usually HSA_OVERRIDE_GFX_VERSION=10.3.0 does the trick if the gfx1031 support included in ROCm is not sufficient.

journalctl -u ollama
ago 02 20:27:15 klaus-Delta-15-A5EFK systemd[1]: Started Ollama Service.
ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: Couldn't find '/usr/share/ollama/.ollama/id_ed25519'. Generating new private key.
ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: Your new public key is:
ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIP8pJZ0GxbaFIrBrFsuF+4psdrrZQcRhJLCdMu6igpku
ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: 2024/08/02 20:27:15 routes.go:1109: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION>
ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:15.342+02:00 level=INFO source=images.go:781 msg="total blobs: 0"
ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:15.342+02:00 level=INFO source=images.go:788 msg="total unused blobs removed: 0"
ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:15.342+02:00 level=INFO source=routes.go:1156 msg="Listening on 127.0.0.1:11434 (version 0.3.2)"
ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:15.343+02:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama1547319424/runners
ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.150+02:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11 rocm_v60102]"
ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.150+02:00 level=INFO source=gpu.go:205 msg="looking for compatible GPUs"
ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.167+02:00 level=WARN source=amd_linux.go:59 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" e>
ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.192+02:00 level=WARN source=amd_linux.go:340 msg="amdgpu is not supported" gpu=0 gpu_type=gfx1031 library=/opt/rocm/lib suppor>
ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.192+02:00 level=WARN source=amd_linux.go:342 msg="See https://github.com/ollama/ollama/blob/main/docs/gpu.md#overrides for HSA>
ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.192+02:00 level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=1 total="512.0 MiB"
ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.192+02:00 level=INFO source=amd_linux.go:360 msg="no compatible amdgpu devices detected"
ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.192+02:00 level=INFO source=gpu.go:346 msg="no compatible GPUs were discovered"
ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.192+02:00 level=INFO source=types.go:105 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="15.0 Gi>
ago 02 20:29:09 klaus-Delta-15-A5EFK ollama[78976]: [GIN] 2024/08/02 - 20:29:09 | 200 | 56.246µs | 127.0.0.1 | HEAD "/"
ago 02 20:29:09 klaus-Delta-15-A5EFK ollama[78976]: [GIN] 2024/08/02 - 20:29:09 | 404 | 131.288µs | 127.0.0.1 | POST "/api/show"
ago 02 20:29:11 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:29:11.953+02:00 level=INFO source=download.go:173 msg="downloading 87048bcd5521 in 47 100 MB part(s)"

<!-- gh-comment-id:2267030012 --> @klausbu commented on GitHub (Aug 3, 2024): ok, the AMD 6700M GPU is not supported export HSA_OVERRIDE_GFX_VERSION=10.3.0 doesn't help. Unfortunately, the notebook RDNA2 GPUs are all gfx1031 usually HSA_OVERRIDE_GFX_VERSION=10.3.0 does the trick if the gfx1031 support included in ROCm is not sufficient. journalctl -u ollama ago 02 20:27:15 klaus-Delta-15-A5EFK systemd[1]: Started Ollama Service. ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: Couldn't find '/usr/share/ollama/.ollama/id_ed25519'. Generating new private key. ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: Your new public key is: ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIP8pJZ0GxbaFIrBrFsuF+4psdrrZQcRhJLCdMu6igpku ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: 2024/08/02 20:27:15 routes.go:1109: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION> ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:15.342+02:00 level=INFO source=images.go:781 msg="total blobs: 0" ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:15.342+02:00 level=INFO source=images.go:788 msg="total unused blobs removed: 0" ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:15.342+02:00 level=INFO source=routes.go:1156 msg="Listening on 127.0.0.1:11434 (version 0.3.2)" ago 02 20:27:15 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:15.343+02:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama1547319424/runners ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.150+02:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11 rocm_v60102]" ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.150+02:00 level=INFO source=gpu.go:205 msg="looking for compatible GPUs" ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.167+02:00 level=WARN source=amd_linux.go:59 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" e> ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.192+02:00 level=WARN source=amd_linux.go:340 msg="amdgpu is not supported" gpu=0 gpu_type=gfx1031 library=/opt/rocm/lib suppor> ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.192+02:00 level=WARN source=amd_linux.go:342 msg="See https://github.com/ollama/ollama/blob/main/docs/gpu.md#overrides for HSA> ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.192+02:00 level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=1 total="512.0 MiB" ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.192+02:00 level=INFO source=amd_linux.go:360 msg="no compatible amdgpu devices detected" ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.192+02:00 level=INFO source=gpu.go:346 msg="no compatible GPUs were discovered" ago 02 20:27:18 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:27:18.192+02:00 level=INFO source=types.go:105 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="15.0 Gi> ago 02 20:29:09 klaus-Delta-15-A5EFK ollama[78976]: [GIN] 2024/08/02 - 20:29:09 | 200 | 56.246µs | 127.0.0.1 | HEAD "/" ago 02 20:29:09 klaus-Delta-15-A5EFK ollama[78976]: [GIN] 2024/08/02 - 20:29:09 | 404 | 131.288µs | 127.0.0.1 | POST "/api/show" ago 02 20:29:11 klaus-Delta-15-A5EFK ollama[78976]: time=2024-08-02T20:29:11.953+02:00 level=INFO source=download.go:173 msg="downloading 87048bcd5521 in 47 100 MB part(s)"
GiteaMirror commented 2026-04-12 14:40:51 -05:00
Author
Owner
Copy Link

@asknight1980 commented on GitHub (Aug 4, 2024):

Your startup log is showing env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION> and not HSA_OVERRIDE_GFX_VERSION=10.3.0 because export is not the correct way to pass hardware configuration info.

You need to put the following in the [Service] section of the ollama.service config file. Mine is located at /etc/systemd/system/ollama.service

Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"

This is what works for me with an RX6700 (RDNA2, Navi22)
Don't forget to 'sudo systemctl daemon-reload && sudo systemctl restart ollama.service' after you change the config file.

<!-- gh-comment-id:2267390837 --> @asknight1980 commented on GitHub (Aug 4, 2024): Your startup log is showing `env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION>` and not `HSA_OVERRIDE_GFX_VERSION=10.3.0` because export is not the correct way to pass hardware configuration info. You need to put the following in the [Service] section of the ollama.service config file. Mine is located at /etc/systemd/system/ollama.service `Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"` This is what works for me with an RX6700 (RDNA2, Navi22) Don't forget to 'sudo systemctl daemon-reload && sudo systemctl restart ollama.service' after you change the config file.
GiteaMirror commented 2026-04-12 14:40:51 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 4, 2024):

The log is incomplete because of the use of a pager by journalctl, try journalctl -u ollama --no-pager.

<!-- gh-comment-id:2267441482 --> @rick-github commented on GitHub (Aug 4, 2024): The log is incomplete because of the use of a pager by `journalctl`, try `journalctl -u ollama --no-pager`.
GiteaMirror commented 2026-04-12 14:40:51 -05:00
Author
Owner
Copy Link

@Cephra commented on GitHub (Aug 4, 2024):

I've had some problems on my manjaro box as well. Even though ROCm was installed, it was still using the CPU. I assume there was some very specific dependency missing, however there was no error found in the logs. It was simply not utilizing my GPU. What I ended up doing was installing the docker image with rocm support. And it works like a charm, despite my RX 6600 XT not being officially supported. Also that approach has the advantage of the docker image being updated more frequently. So if you have the option to install docker on your box, I would give that a try. It might just work.

<!-- gh-comment-id:2267539699 --> @Cephra commented on GitHub (Aug 4, 2024): I've had some problems on my manjaro box as well. Even though ROCm was installed, it was still using the CPU. I assume there was some very specific dependency missing, however there was no error found in the logs. It was simply not utilizing my GPU. What I ended up doing was installing the docker image with rocm support. And it works like a charm, despite my RX 6600 XT not being officially supported. Also that approach has the advantage of the docker image being updated more frequently. So if you have the option to install docker on your box, I would give that a try. It might just work.
GiteaMirror commented 2026-04-12 14:40:52 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Aug 4, 2024):

+1 for docker install.

<!-- gh-comment-id:2267540518 --> @rick-github commented on GitHub (Aug 4, 2024): +1 for docker install.
GiteaMirror commented 2026-04-12 14:40:52 -05:00
Author
Owner
Copy Link

@klausbu commented on GitHub (Aug 4, 2024):

Adding the following line under [Service]

Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"

to the ollama service file: /etc/systemd/system/ollama.service fixed the problem. The GPU is now quite a booster.

Details can be found here: https://forum.level1techs.com/t/ubuntu-22-04-from-zero-to-70b-llama-with-both-nvidia-and-amd-7xxx-series-gpus/206411/13

<!-- gh-comment-id:2267543373 --> @klausbu commented on GitHub (Aug 4, 2024): Adding the following line under [Service] Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0" to the ollama service file: /etc/systemd/system/ollama.service fixed the problem. The GPU is now quite a booster. Details can be found here: https://forum.level1techs.com/t/ubuntu-22-04-from-zero-to-70b-llama-with-both-nvidia-and-amd-7xxx-series-gpus/206411/13
GiteaMirror commented 2026-04-12 14:40:52 -05:00
Author
Owner
Copy Link

@Cephra commented on GitHub (Aug 4, 2024):

Great! Please close this issue if it's resolved.

<!-- gh-comment-id:2267552782 --> @Cephra commented on GitHub (Aug 4, 2024): Great! Please close this issue if it's resolved.
GiteaMirror commented 2026-04-12 14:40:52 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Aug 9, 2024):

Sounds like the GPU is working now with the override set.

<!-- gh-comment-id:2278741622 --> @dhiltgen commented on GitHub (Aug 9, 2024): Sounds like the GPU is working now with the override set.
GiteaMirror commented 2026-04-12 14:40:53 -05:00
Author
Owner
Copy Link

@Abdorithm commented on GitHub (Nov 21, 2024):

I've had some problems on my manjaro box as well. Even though ROCm was installed, it was still using the CPU. I assume there was some very specific dependency missing, however there was no error found in the logs. It was simply not utilizing my GPU. What I ended up doing was installing the docker image with rocm support. And it works like a charm, despite my RX 6600 XT not being officially supported. Also that approach has the advantage of the docker image being updated more frequently. So if you have the option to install docker on your box, I would give that a try. It might just work.

can you provide which docker image with rocm support did you install? I have the same problem on Arch with getting ollama to work on my not-officially-supported AMD GPU.

<!-- gh-comment-id:2491977319 --> @Abdorithm commented on GitHub (Nov 21, 2024): > I've had some problems on my manjaro box as well. Even though ROCm was installed, it was still using the CPU. I assume there was some very specific dependency missing, however there was no error found in the logs. It was simply not utilizing my GPU. What I ended up doing was installing the docker image with rocm support. And it works like a charm, despite my RX 6600 XT not being officially supported. Also that approach has the advantage of the docker image being updated more frequently. So if you have the option to install docker on your box, I would give that a try. It might just work. can you provide which docker image with rocm support did you install? I have the same problem on Arch with getting ollama to work on my not-officially-supported AMD GPU.
GiteaMirror commented 2026-04-12 14:40:53 -05:00
Author
Owner
Copy Link

@chandujr commented on GitHub (Feb 5, 2025):

I have the same issue. I have installed Ollama via docker:

docker run -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocm

But as you can see in the btop screenshot below, when I run a model, it is using all CPU, GPUs are not working at all. Mine is a laptop with hybrid graphics (Ryzen9 5980HX and Radeon 6800M).

Image

rocminfo shows gfx1031 for the dGPU (which is GPU0 in btop)

<!-- gh-comment-id:2637911253 --> @chandujr commented on GitHub (Feb 5, 2025): I have the same issue. I have installed Ollama via docker: ``` docker run -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocm ``` But as you can see in the btop screenshot below, when I run a model, it is using all CPU, GPUs are not working at all. Mine is a laptop with hybrid graphics (Ryzen9 5980HX and Radeon 6800M). ![Image](https://github.com/user-attachments/assets/fa418ee6-8aa4-4c51-be39-3165a699016f) `rocminfo` shows `gfx1031` for the dGPU (which is GPU0 in btop)
GiteaMirror commented 2026-04-12 14:40:53 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 5, 2025):

https://github.com/ollama/ollama/issues/6152#issuecomment-2266794744

<!-- gh-comment-id:2637924088 --> @rick-github commented on GitHub (Feb 5, 2025): https://github.com/ollama/ollama/issues/6152#issuecomment-2266794744
GiteaMirror commented 2026-04-12 14:40:54 -05:00
Author
Owner
Copy Link

@chandujr commented on GitHub (Feb 6, 2025):

@rick-github I tried setting the environment variable in "docker run" but the model is still not using GPU.

<!-- gh-comment-id:2640677309 --> @chandujr commented on GitHub (Feb 6, 2025): @rick-github I tried setting the environment variable in "docker run" but the model is still not using GPU.
GiteaMirror commented 2026-04-12 14:40:54 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 6, 2025):

It's too bad there isn't some way to verify settings or look at the internal workings of the server.

<!-- gh-comment-id:2640691120 --> @rick-github commented on GitHub (Feb 6, 2025): It's too bad there isn't some way to verify settings or look at the internal workings of the server.
GiteaMirror commented 2026-04-12 14:40:54 -05:00
Author
Owner
Copy Link

@chandujr commented on GitHub (Feb 6, 2025):

Alright, the docker version still only runs on CPU. I removed it and installed the normal version and added the environment variables. That worked! and wow, what a difference in speed!

Image

<!-- gh-comment-id:2640774381 --> @chandujr commented on GitHub (Feb 6, 2025): Alright, the docker version still only runs on CPU. I removed it and installed the normal version and added the environment variables. That worked! and wow, what a difference in speed! ![Image](https://github.com/user-attachments/assets/b76de58f-37af-4fbc-816b-621f8347b8b7)
GiteaMirror commented 2026-04-12 14:40:54 -05:00
Author
Owner
Copy Link

@feraxhp commented on GitHub (Nov 13, 2025):

@chandujr

You just have to add the env variable in the docker command:

docker run -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -e HSA_OVERRIDE_GFX_VERSION=10.3.0 -p 11434:11434 --name ollama ollama/ollama:rocm
<!-- gh-comment-id:3528049249 --> @feraxhp commented on GitHub (Nov 13, 2025): @chandujr You just have to add the env variable in the docker command: ~~~bash docker run -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -e HSA_OVERRIDE_GFX_VERSION=10.3.0 -p 11434:11434 --name ollama ollama/ollama:rocm ~~~
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label amd question
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#3842
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?