github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-08 00:51:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #2685] v0.1.26 and v0.1.25 do not use AMD GPU on Linux #48113

New Issue
Closed
opened 2026-04-28 06:47:05 -05:00 by GiteaMirror · 8 comments
GiteaMirror commented 2026-04-28 06:47:05 -05:00
Owner
Copy Link

Originally created by @TimTheBig on GitHub (Feb 22, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/2685

v0.1.26 and v0.1.25 do not use GPU(7900xtx) on Nobara Linux 39 when I use the install script. https://github.com/ollama/ollama/issues/2502#issuecomment-1949514130

Originally created by @TimTheBig on GitHub (Feb 22, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/2685 v0.1.26 and v0.1.25 do not use GPU(7900xtx) on [Nobara Linux 39](https://nobaraproject.org) when I use the install script. https://github.com/ollama/ollama/issues/2502#issuecomment-1949514130
GiteaMirror commented 2026-04-28 06:47:08 -05:00
Author
Owner
Copy Link

@TimTheBig commented on GitHub (Feb 22, 2024):

I installed ROCm and Cl-blast.

<!-- gh-comment-id:1959821304 --> @TimTheBig commented on GitHub (Feb 22, 2024): I installed *ROCm* and *Cl-blast*.
GiteaMirror commented 2026-04-28 06:47:10 -05:00
Author
Owner
Copy Link

@TimTheBig commented on GitHub (Feb 22, 2024):

Are CUDA libraries required for ollama ROCm? https://github.com/ollama/ollama/issues/2503

<!-- gh-comment-id:1959917443 --> @TimTheBig commented on GitHub (Feb 22, 2024): Are CUDA libraries required for ollama ROCm? https://github.com/ollama/ollama/issues/2503
GiteaMirror commented 2026-04-28 06:47:12 -05:00
Author
Owner
Copy Link

@sid-cypher commented on GitHub (Feb 22, 2024):

I've successfully compiled and ran both llama.cpp separately and ollama without CUDA libraries (it looks to me like GGML code comes directly from llama.cpp).
The projects often rely on compilers in /opt/rocm to HIPify all the .cu stuff.

Could it be that the releases are not compiled with HIP, therefore non-NVidia HW support is not present?

<!-- gh-comment-id:1959937668 --> @sid-cypher commented on GitHub (Feb 22, 2024): I've successfully compiled and ran both `llama.cpp` separately and `ollama` without CUDA libraries (it looks to me like GGML code comes directly from llama.cpp). The projects often rely on compilers in `/opt/rocm` to HIPify all the `.cu` stuff. Could it be that the releases are not compiled with HIP, therefore non-NVidia HW support is not present?
GiteaMirror commented 2026-04-28 06:47:13 -05:00
Author
Owner
Copy Link

@TimTheBig commented on GitHub (Feb 22, 2024):

So I need to compile it. Another binary that the install script could use on ROCm accelerated systems would be useful.

<!-- gh-comment-id:1960220808 --> @TimTheBig commented on GitHub (Feb 22, 2024): So I need to compile it. Another binary that the install script could use on *ROCm* accelerated systems would be useful.
GiteaMirror commented 2026-04-28 06:47:15 -05:00
Author
Owner
Copy Link

@TimTheBig commented on GitHub (Feb 22, 2024):

Is the solution https://github.com/ollama/ollama/issues/738#issuecomment-1936765124?

<!-- gh-comment-id:1960385416 --> @TimTheBig commented on GitHub (Feb 22, 2024): Is the solution https://github.com/ollama/ollama/issues/738#issuecomment-1936765124?
GiteaMirror commented 2026-04-28 06:47:16 -05:00
Author
Owner
Copy Link

@sid-cypher commented on GitHub (Feb 23, 2024):

#738 (comment)?

Confirmed success: I've downloaded the current release from Github (v0.1.27 8782dd5 by jmorganca 12 hours ago) and it worked with ROCm 6.0.2:

/tmp$ ./ollama-linux-amd64 serve
...
time=2024-02-23T12:03:46.746+01:00 level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: /tmp/ollama3461383549/rocm_v6/libext_server.so"
time=2024-02-23T12:03:46.746+01:00 level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server"
ggml_init_cublas: GGML_CUDA_FORCE_MMQ:   no
ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes
ggml_init_cublas: found 1 ROCm devices:
  Device 0: Radeon RX 7900 XTX, compute capability 11.0, VMM: no
...
lm_load_tensors: offloading 40 repeating layers to GPU
llm_load_tensors: offloading non-repeating layers to GPU
llm_load_tensors: offloaded 41/41 layers to GPU
llm_load_tensors:      ROCm0 buffer size =  8694.21 MiB

Edit: I assumed that the point of opening an issue for v0.1.26 and v0.1.25 is to have it fixed in v0.1.27 or later. It is indeed fixed in v0.1.27, or so it seems on my machine, please test it independently.

<!-- gh-comment-id:1961134720 --> @sid-cypher commented on GitHub (Feb 23, 2024): > [#738 (comment)](https://github.com/ollama/ollama/issues/738#issuecomment-1936765124)? Confirmed success: I've downloaded the current release from Github (v0.1.27 8782dd5 by jmorganca 12 hours ago) and it worked with ROCm 6.0.2: ``` /tmp$ ./ollama-linux-amd64 serve ... time=2024-02-23T12:03:46.746+01:00 level=INFO source=dyn_ext_server.go:90 msg="Loading Dynamic llm server: /tmp/ollama3461383549/rocm_v6/libext_server.so" time=2024-02-23T12:03:46.746+01:00 level=INFO source=dyn_ext_server.go:150 msg="Initializing llama server" ggml_init_cublas: GGML_CUDA_FORCE_MMQ: no ggml_init_cublas: CUDA_USE_TENSOR_CORES: yes ggml_init_cublas: found 1 ROCm devices: Device 0: Radeon RX 7900 XTX, compute capability 11.0, VMM: no ... lm_load_tensors: offloading 40 repeating layers to GPU llm_load_tensors: offloading non-repeating layers to GPU llm_load_tensors: offloaded 41/41 layers to GPU llm_load_tensors: ROCm0 buffer size = 8694.21 MiB ``` Edit: I assumed that the point of opening an issue for v0.1.26 and v0.1.25 is to have it fixed in v0.1.27 or later. It is indeed fixed in v0.1.27, or so it seems on my machine, please test it independently.
GiteaMirror commented 2026-04-28 06:47:17 -05:00
Author
Owner
Copy Link

@3lpsy commented on GitHub (Feb 23, 2024):

For those wanting to try out v0.1.27 on Arch Linux using Rocm on an AMD GPU, here's what I did. First, make sure v0.1.27 is installed. I used the download script and just modified the following:

_U="https://github.com/ollama/ollama/releases/download/v0.1.27/ollama-linux-amd64"
curl --fail --show-error --location --progress-bar -o $TEMP_DIR/ollama "$_U"

Next, I made sure the rocm runtime, hip runtime, and hipblas were installed. This required installing hipblas and rocm-smi-lib. Other libraries may be required but these were the two I installed.

After installing ollama and necessary libs, start tracking the ollama service in a terminal via sudo journalctl -efu ollama.

Run the following to see if libraries are loaded correctly and track the loads from systemd:

ollama run --verbose llama2

Your goal is mitigate the Failed to load dynamic library errors (assuming there aren't others you need to address first). An example looks like:

 level=WARN source=llm.go:162 msg="Failed to load d
ynamic library /tmp/ollama3904863067/rocm_v5/libext_server.so  Unable to load dynamic library: Unable to load dynamic server
library: librocsparse.so.0: cannot open shared object file: No such file or directory"

Take note of the library it can't find (librocsparse.so.0 in the above). This is probably cause rocm6 is installed. You just need to symlink the new versions. So for example, the following worked for me:

cd /opt/rocm/lib
sudo ln -s libhipblas.so.2 libhipblas.so.1
sudo ln -s librocblas.so.4 librocblas.so.3
sudo ln -s libamdhip64.so.6 libamdhip64.so.5
sudo ln -s librocsparse.so.1 librocsparse.so.0

Can't speak to the stability of just symlinking these libraries but it's now super speedy for me. Good luck :)

<!-- gh-comment-id:1961666228 --> @3lpsy commented on GitHub (Feb 23, 2024): For those wanting to try out v0.1.27 on Arch Linux using Rocm on an AMD GPU, here's what I did. First, make sure v0.1.27 is installed. I used the download script and just modified the following: ``` _U="https://github.com/ollama/ollama/releases/download/v0.1.27/ollama-linux-amd64" curl --fail --show-error --location --progress-bar -o $TEMP_DIR/ollama "$_U" ``` Next, I made sure the rocm runtime, hip runtime, and hipblas were installed. This required installing `hipblas` and `rocm-smi-lib`. Other libraries may be required but these were the two I installed. After installing ollama and necessary libs, start tracking the ollama service in a terminal via `sudo journalctl -efu ollama`. Run the following to see if libraries are loaded correctly and track the loads from systemd: ``` ollama run --verbose llama2 ``` Your goal is mitigate the Failed to load dynamic library errors (assuming there aren't others you need to address first). An example looks like: ``` level=WARN source=llm.go:162 msg="Failed to load d ynamic library /tmp/ollama3904863067/rocm_v5/libext_server.so Unable to load dynamic library: Unable to load dynamic server library: librocsparse.so.0: cannot open shared object file: No such file or directory" ``` Take note of the library it can't find (`librocsparse.so.0` in the above). This is probably cause rocm6 is installed. You just need to symlink the new versions. So for example, the following worked for me: ``` cd /opt/rocm/lib sudo ln -s libhipblas.so.2 libhipblas.so.1 sudo ln -s librocblas.so.4 librocblas.so.3 sudo ln -s libamdhip64.so.6 libamdhip64.so.5 sudo ln -s librocsparse.so.1 librocsparse.so.0 ``` Can't speak to the stability of just symlinking these libraries but it's now super speedy for me. Good luck :)
GiteaMirror commented 2026-04-28 06:47:19 -05:00
Author
Owner
Copy Link

@TimTheBig commented on GitHub (Feb 23, 2024):

Should be fixed in v0.1.27.

<!-- gh-comment-id:1961693929 --> @TimTheBig commented on GitHub (Feb 23, 2024): Should be fixed in v0.1.27.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#48113
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?