github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-21 13:11:39 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #12945] Wiki addendum: Grabbing AMD GPU Ids via rocminfo for GPU isolation / restricting ollama GPUs via ROCR_VISIBLE_DEVICES or ROCM_VISIBLE_DEVICES (ROCM) #8583

New Issue
Open
opened 2026-04-12 21:19:12 -05:00 by GiteaMirror · 4 comments
GiteaMirror commented 2026-04-12 21:19:12 -05:00
Owner
Copy Link

Originally created by @ksaadDE on GitHub (Nov 4, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/12945

If you want to figure out the GPU Ids to limit GPU selection to certain GPUs (being visible) and you have an AMD GPU.
You want to specify ROCR_VISIBLE_DEVICES or earlier's ROCM_VISIBLE_DEVICES (did that even work at some point?) to allow it being included as env vars during container build or ollama start in advance

Prerequisites (e.g. on Arch)

pacman -Qyuuu # (abort updating, just refreshing the repos)
pacman -S  rocm-smi-lib rocm-hip-sdk

Identification of the rocminfo and rocm-smi binaries

a whereis rocminfo (and rocm-smi)

May return something like /opt/rocm/bin/rocminfo

Grabbing the GPU Ids using rocminfo

/opt/rocm/bin/rocminfo | grep -Ei "Marketing|Uuid"

That should output something like:

Uuid:                    CPU-XX
Marketing Name:          AMD XXXXXX # <-- CPU's one
Uuid:                    GPU-XXXXXXXXXXXXXX # <--- thats what we need the ded. GPU's uuid
Marketing Name:          AMD Radeon XXXXX # <-- to verify the card's common product name!

Adding GPU Ids to env ``ROCR_VISIBLE_DEVICES` (or ROCM..._DEVICES) var

Finally, add the uuid (here the 2nd, dedicated GPU; not the CPU's one) to your docker build file as env ENV ROCR_VISIBLE_DEVICES=GPU-XXXXXXX

Testing the results

After rebuilding the container and starting ollama (or on baremetal), the logs should show something like:

time=XXXXX level=INFO source=routes.go:1511 msg="server config" env="map[CUDA_VISIBLE_DEVICES[...]:[....] OLLAMA_REMOTES:[ollama.com] [...] ROCR_VISIBLE_DEVICES:GPU-XXXXXXXXXXX  [...] ]"

A bit later (downwards) in the logs we should see that this gpu is actually loaded ONLY.

If our previously grabbed GPU-Id is present, the test result shall be successful. Otherwise something is wrong.

Note: I added [...] to skip the noisy other env vars, to make it more visible to you. Once the ROCR env var is filled, and detected as server config var, it should skip the detection algorithms and just use the gpu ("faster" startup?), and usually only use that one.

Important note: ROCM_VISIBLE_DEVICES did nothing for me! However, ROCR does work!

Sources

Originally created by @ksaadDE on GitHub (Nov 4, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/12945 If you want to figure out the GPU Ids to limit GPU selection to certain GPUs (being visible) and you have an AMD GPU. You want to specify `ROCR_VISIBLE_DEVICES` or earlier's `ROCM_VISIBLE_DEVICES` (did that even work at some point?) to allow it being included as env vars during container build or ollama start in advance ## Prerequisites (e.g. on Arch) ```bash pacman -Qyuuu # (abort updating, just refreshing the repos) pacman -S rocm-smi-lib rocm-hip-sdk ``` ## Identification of the rocminfo and rocm-smi binaries a ` whereis rocminfo` (and rocm-smi) May return something like `/opt/rocm/bin/rocminfo ` ## Grabbing the GPU Ids using rocminfo ``` /opt/rocm/bin/rocminfo | grep -Ei "Marketing|Uuid" ``` That should output something like: ``` Uuid: CPU-XX Marketing Name: AMD XXXXXX # <-- CPU's one Uuid: GPU-XXXXXXXXXXXXXX # <--- thats what we need the ded. GPU's uuid Marketing Name: AMD Radeon XXXXX # <-- to verify the card's common product name! ``` ## Adding GPU Ids to env ``ROCR_VISIBLE_DEVICES` (or ROCM..._DEVICES) var Finally, add the `uuid` (here the 2nd, dedicated GPU; not the CPU's one) to your docker build file as env `ENV ROCR_VISIBLE_DEVICES=GPU-XXXXXXX` ## Testing the results After rebuilding the container and starting ollama (or on baremetal), the logs should show something like: ``` time=XXXXX level=INFO source=routes.go:1511 msg="server config" env="map[CUDA_VISIBLE_DEVICES[...]:[....] OLLAMA_REMOTES:[ollama.com] [...] ROCR_VISIBLE_DEVICES:GPU-XXXXXXXXXXX [...] ]" ``` A bit later (downwards) in the logs we should see that this gpu is actually loaded ONLY. **If our previously grabbed GPU-Id is present, the test result shall be successful. Otherwise something is wrong.** **Note**: I added [...] to skip the noisy other env vars, to make it more visible to you. Once the ROCR env var is filled, and detected as server config var, it should skip the detection algorithms and just use the gpu ("faster" startup?), and usually only use that one. **Important note**: `ROCM_VISIBLE_DEVICES` did nothing for me! However, **ROCR** does work! ## Sources - https://rocm.docs.amd.com/en/latest/conceptual/gpu-isolation.html - https://gitlab.informatik.uni-halle.de/ambcj/ollama/-/blob/d1692fd3e0b4a80ff55ba052b430207134df4714/docs/gpu.md
GiteaMirror added the feature request label 2026-04-12 21:19:12 -05:00
GiteaMirror commented 2026-04-12 21:19:14 -05:00
Author
Owner
Copy Link

@ksaadDE commented on GitHub (Nov 4, 2025):

Sry did some title engineering :)

<!-- gh-comment-id:3485492157 --> @ksaadDE commented on GitHub (Nov 4, 2025): Sry did some title engineering :)
GiteaMirror commented 2026-04-12 21:19:15 -05:00
Author
Owner
Copy Link

@ksaadDE commented on GitHub (Nov 4, 2025):

As far as I am seeing, perhaps this could be an addition to the well-written https://docs.ollama.com/gpu#gpu-selection and https://docs.ollama.com/gpu#gpu-selection-2

I was not able to find it via Search Engines (e.g. Google, StartPage & DuckDuck; prob. because its heavy-loaded with JS templating stuff - SEs being somehow unable to parse that; or due to its ranking). I intentionally looked for that, after posting this, via the docs own search.

It could also be used for testing in automation of these configurations, perhaps being a deeper guide for it. Maybe it saves someone time?

<!-- gh-comment-id:3485548593 --> @ksaadDE commented on GitHub (Nov 4, 2025): As far as I am seeing, perhaps this could be an addition to the well-written https://docs.ollama.com/gpu#gpu-selection and https://docs.ollama.com/gpu#gpu-selection-2 I was not able to find it via Search Engines (e.g. Google, StartPage & DuckDuck; prob. because its heavy-loaded with JS templating stuff - SEs being somehow unable to parse that; or due to its ranking). I intentionally looked for that, after posting this, via the docs own search. It could also be used for testing in automation of these configurations, perhaps being a deeper guide for it. Maybe it saves someone time?
GiteaMirror commented 2026-04-12 21:19:16 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Nov 5, 2025):

Feel free to PR against https://github.com/ollama/ollama/blob/main/docs/gpu.mdx#gpu-selection-1 which will get reflected at docs.ollama.com after changes are merged to main.

<!-- gh-comment-id:3493623624 --> @dhiltgen commented on GitHub (Nov 5, 2025): Feel free to PR against https://github.com/ollama/ollama/blob/main/docs/gpu.mdx#gpu-selection-1 which will get reflected at docs.ollama.com after changes are merged to main.
GiteaMirror commented 2026-04-12 21:19:16 -05:00
Author
Owner
Copy Link

@ksaadDE commented on GitHub (Nov 12, 2025):

Feel free to PR against https://github.com/ollama/ollama/blob/main/docs/gpu.mdx#gpu-selection-1 which will get reflected at docs.ollama.com after changes are merged to main.

Thank you! I will do once workload permits! Perhaps there is something that I can improve.

<!-- gh-comment-id:3522489689 --> @ksaadDE commented on GitHub (Nov 12, 2025): > Feel free to PR against https://github.com/ollama/ollama/blob/main/docs/gpu.mdx#gpu-selection-1 which will get reflected at docs.ollama.com after changes are merged to main. Thank you! I will do once workload permits! Perhaps there is something that I can improve.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#8583
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?