github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 05:05:37 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #7420] Support AMD GPUs on Ampere, Raspberry Pis (arm64 ROCm) #4719

New Issue
Closed
opened 2026-04-12 15:39:41 -05:00 by GiteaMirror · 5 comments
GiteaMirror commented 2026-04-12 15:39:41 -05:00
Owner
Copy Link

Originally created by @geerlingguy on GitHub (Oct 30, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/7420

I have an AMD RX 6700 XT running on my Raspberry Pi 5 (see https://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/222, I know I'm nuts)... and I tried installing Ollama, but I get the following error:

pi@pi5-pcie:~ $ curl -fsSL https://ollama.com/install.sh | sh
>>> Installing ollama to /usr/local
>>> Downloading Linux arm64 bundle
######################################################################## 100.0%
>>> Creating ollama user...
>>> Adding ollama user to render group...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
Created symlink /etc/systemd/system/default.target.wants/ollama.service -> /etc/systemd/system/ollama.service.
>>> Downloading Linux ROCm arm64 bundle
curl: (22) The requested URL returned error: 404                              


gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now

It tries downloading from https://ollama.com/download/ollama-linux-arm64-rocm.tgz, but right now there's only an ollama-linux-amd64-rocm.tgz included in the builds.

Seeing as there is nobody else asking for it, I thought I'd open an issue. It would be neat to be able to run larger models on tiny SBCs sipping 2-4W at idle.

I'm also perfectly fine with this issue being closed 'wontfix', as AMD as of 2020/2021 said they aren't supporting arm64 with ROCm: https://github.com/ROCm/ROCm/issues/1052 (maybe that's changed, I will ask again in https://github.com/ROCm/ROCm/issues/3960 ...).

Originally created by @geerlingguy on GitHub (Oct 30, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/7420 I have an AMD RX 6700 XT running on my Raspberry Pi 5 (see https://github.com/geerlingguy/raspberry-pi-pcie-devices/issues/222, I know I'm nuts)... and I tried installing Ollama, but I get the following error: ``` pi@pi5-pcie:~ $ curl -fsSL https://ollama.com/install.sh | sh >>> Installing ollama to /usr/local >>> Downloading Linux arm64 bundle ######################################################################## 100.0% >>> Creating ollama user... >>> Adding ollama user to render group... >>> Adding ollama user to video group... >>> Adding current user to ollama group... >>> Creating ollama systemd service... >>> Enabling and starting ollama service... Created symlink /etc/systemd/system/default.target.wants/ollama.service -> /etc/systemd/system/ollama.service. >>> Downloading Linux ROCm arm64 bundle curl: (22) The requested URL returned error: 404 gzip: stdin: unexpected end of file tar: Child returned status 1 tar: Error is not recoverable: exiting now ``` It tries downloading from `https://ollama.com/download/ollama-linux-arm64-rocm.tgz`, but right now there's only an `ollama-linux-amd64-rocm.tgz` included in the builds. Seeing as there is nobody else asking for it, I thought I'd open an issue. It would be neat to be able to run larger models on tiny SBCs sipping 2-4W at idle. I'm also perfectly fine with this issue being closed 'wontfix', as AMD as of 2020/2021 said they aren't supporting arm64 with ROCm: https://github.com/ROCm/ROCm/issues/1052 (maybe that's changed, I will ask again in https://github.com/ROCm/ROCm/issues/3960 ...).
GiteaMirror added the feature request label 2026-04-12 15:39:41 -05:00
GiteaMirror commented 2026-04-12 15:39:42 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Nov 1, 2024):

I think it's unlikely AMD (who makes x86 based CPUs) will invest in supporting competing CPU architectures in their GPU libraries. If they do someday support that, we could fairly easily add support into Ollama, but I'm going to close this for now. Please do let us know if you get traction from AMD and we can reopen.

<!-- gh-comment-id:2452141471 --> @dhiltgen commented on GitHub (Nov 1, 2024): I think it's unlikely AMD (who makes x86 based CPUs) will invest in supporting competing CPU architectures in their GPU libraries. If they do someday support that, we could fairly easily add support into Ollama, but I'm going to close this for now. Please do let us know if you get traction from AMD and we can reopen.
GiteaMirror commented 2026-04-12 15:39:43 -05:00
Author
Owner
Copy Link

@geerlingguy commented on GitHub (Nov 1, 2024):

Fair enough! :)

Mostly wanted a reference point, I will hopefully get some Nvidia cards going at some point too (already have on Ampere at least), which may make it easier to integrate with CUDA.

<!-- gh-comment-id:2452356492 --> @geerlingguy commented on GitHub (Nov 1, 2024): Fair enough! :) Mostly wanted a reference point, I will hopefully get some Nvidia cards going at some point too (already have on Ampere at least), which may make it easier to integrate with CUDA.
GiteaMirror commented 2026-04-12 15:39:44 -05:00
Author
Owner
Copy Link

@iHad168 commented on GitHub (Nov 18, 2024):

I same use AMD RX 6700 XT
My CPU is AMD Ryzen 9 3900X
but i find curl -fsSL https://ollama.com/install.sh | sh cannot auto install AMD ROCM
I must use curl -L https://ollama.com/download/ollama-linux-amd64-rocm.tgz -o ollama-linux-amd64-rocm.tgz sudo tar -C /usr -xzf ollama-linux-amd64-rocm.tgz
install refer this

<!-- gh-comment-id:2481714045 --> @iHad168 commented on GitHub (Nov 18, 2024): I same use AMD RX 6700 XT My CPU is AMD Ryzen 9 3900X but i find `curl -fsSL https://ollama.com/install.sh | sh` cannot auto install AMD ROCM I must use ```curl -L https://ollama.com/download/ollama-linux-amd64-rocm.tgz -o ollama-linux-amd64-rocm.tgz sudo tar -C /usr -xzf ollama-linux-amd64-rocm.tgz``` install refer [this](https://github.com/ollama/ollama/blob/main/docs/linux.md)
GiteaMirror commented 2026-04-12 15:39:44 -05:00
Author
Owner
Copy Link

@juergen0815 commented on GitHub (Aug 13, 2025):

Did you guys make any progress on this? Because

jurgen@workstation-02
---------------------
OS: Kubuntu aarch64
Host: Radxa Orion O6 (1.0)
CPU: Cortex-A720*8 (8) @ 2.50 GHz
GPU: AMD Radeon Pro WX 5100 [Discrete]
Memory: 24.45 GiB / 30.07 GiB (81%)

----------
jurgen@workstation-02:~$ sudo rocminfo 
ROCk module is loaded
=====================    
HSA System Attributes    
=====================    
Runtime Version:         1.1
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE                              
System Endianness:       LITTLE                             
Mwaitx:                  DISABLED
DMAbuf Support:          YES

==========               
HSA Agents               
==========               
*******                  
Agent 1                  
*******                  
  Name:                                                       
  Uuid:                    CPU-XX                             
  Marketing Name:                                             
  Vendor Name:             CPU                                
....
*******                  
Agent 2                  
*******                  
  Name:                    gfx803                             
  Uuid:                    GPU-XX                             
  Marketing Name:          AMD Radeon Pro WX 5100 Graphics    
  Vendor Name:             AMD                                
....
*** Done ***             

And 

jurgen@workstation-02:~$ rocm-podman-setup
Checks
======
  [OK] Key packages are installed
  [OK] Local APT cache detected, make sure to use it
  [OK] /dev/kfd is present
  [OK] Group 'render' is present
  [OK] User 'jurgen' is in group 'render'
  [OK] User 'jurgen' is in group 'video'
  [OK] unprivileged_userns_clone is enabled
  [OK] /etc/subgid contains a subordinate GID mapping for user 'jurgen' group 'render'
  [OK] /etc/subgid contains a subordinate GID mapping for user 'jurgen' group 'video'
  [OK] /etc/subgid contains a large subordinate GID range
  [OK] /etc/subuid contains a large subordinate UID range

Rocm runs just fine on the O6 with a WX5100 - probably won't upgrade that (unless the 7400 comes out and will run).

But I'd think that should run ollama with rocm on that config. I haven't tried compiling myself or do a custom install. I really would like to run this in podman on that machine, though

<!-- gh-comment-id:3185689605 --> @juergen0815 commented on GitHub (Aug 13, 2025): Did you guys make any progress on this? Because ``` jurgen@workstation-02 --------------------- OS: Kubuntu aarch64 Host: Radxa Orion O6 (1.0) CPU: Cortex-A720*8 (8) @ 2.50 GHz GPU: AMD Radeon Pro WX 5100 [Discrete] Memory: 24.45 GiB / 30.07 GiB (81%) ---------- jurgen@workstation-02:~$ sudo rocminfo ROCk module is loaded ===================== HSA System Attributes ===================== Runtime Version: 1.1 System Timestamp Freq.: 1000.000000MHz Sig. Max Wait Duration: 18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count) Machine Model: LARGE System Endianness: LITTLE Mwaitx: DISABLED DMAbuf Support: YES ========== HSA Agents ========== ******* Agent 1 ******* Name: Uuid: CPU-XX Marketing Name: Vendor Name: CPU .... ******* Agent 2 ******* Name: gfx803 Uuid: GPU-XX Marketing Name: AMD Radeon Pro WX 5100 Graphics Vendor Name: AMD .... *** Done *** And jurgen@workstation-02:~$ rocm-podman-setup Checks ====== [OK] Key packages are installed [OK] Local APT cache detected, make sure to use it [OK] /dev/kfd is present [OK] Group 'render' is present [OK] User 'jurgen' is in group 'render' [OK] User 'jurgen' is in group 'video' [OK] unprivileged_userns_clone is enabled [OK] /etc/subgid contains a subordinate GID mapping for user 'jurgen' group 'render' [OK] /etc/subgid contains a subordinate GID mapping for user 'jurgen' group 'video' [OK] /etc/subgid contains a large subordinate GID range [OK] /etc/subuid contains a large subordinate UID range ``` Rocm runs just fine on the O6 with a WX5100 - probably won't upgrade that (unless the 7400 comes out and will run). But I'd think that should run ollama with rocm on that config. I haven't tried compiling myself or do a custom install. I really would like to run this in podman on that machine, though
GiteaMirror commented 2026-04-12 15:39:45 -05:00
Author
Owner
Copy Link

@geerlingguy commented on GitHub (Oct 9, 2025):

I've since switched to using llama.cpp for accelerated LLMs, as it supports ROCm, CUDA, and (most importantly) Vulkan out of the box with no real fuss.

<!-- gh-comment-id:3386134165 --> @geerlingguy commented on GitHub (Oct 9, 2025): I've since switched to using llama.cpp for accelerated LLMs, as it supports ROCm, CUDA, and (most importantly) Vulkan out of the box with no real fuss.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label feature request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#4719
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?