github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 12:48:05 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #3838] On Archlinux and AMD Radeon RX 6800S ollama falls back to CPU #64414

New Issue
Closed
opened 2026-05-03 17:34:18 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-05-03 17:34:18 -05:00
Owner
Copy Link

Originally created by @arael on GitHub (Apr 23, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3838

Originally assigned to: @dhiltgen on GitHub.

What is the issue?

I am not able to use my AMD Radeon RX 6800S with ollama. When I try, it falls back to CPU. I have installed tried both ollama and a fresh install with the scripts/install.sh from the git repo. The result is the same. Please help me.

Command outputs and logs

Here is the output of rocminfo:

(cmd)[~] rocminfo
ROCk module is loaded
=====================    
HSA System Attributes    
=====================    
Runtime Version:         1.1
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE                              
System Endianness:       LITTLE                             
Mwaitx:                  DISABLED
DMAbuf Support:          YES

==========               
HSA Agents               
==========               
*******                  
Agent 1                  
*******                  
  Name:                    AMD Ryzen 7 6800HS with Radeon Graphics
  Uuid:                    CPU-XX                             
  Marketing Name:          AMD Ryzen 7 6800HS with Radeon Graphics
  Vendor Name:             CPU                                
  Feature:                 None specified                     
  Profile:                 FULL_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        0(0x0)                             
  Queue Min Size:          0(0x0)                             
  Queue Max Size:          0(0x0)                             
  Queue Type:              MULTI                              
  Node:                    0                                  
  Device Type:             CPU                                
  Cache Info:              
    L1:                      32768(0x8000) KB                   
  Chip ID:                 0(0x0)                             
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   4785                               
  BDFID:                   0                                  
  Internal Node ID:        0                                  
  Compute Unit:            16                                 
  SIMDs per CU:            0                                  
  Shader Engines:          0                                  
  Shader Arrs. per Eng.:   0                                  
  WatchPts on Addr. Ranges:1                                  
  Features:                None
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    32086940(0x1e99b9c) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    32086940(0x1e99b9c) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    32086940(0x1e99b9c) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
  ISA Info:                
*******                  
Agent 2                  
*******                  
  Name:                    gfx1032                            
  Uuid:                    GPU-XX                             
  Marketing Name:          AMD Radeon RX 6800S                
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    1                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      16(0x10) KB                        
    L2:                      2048(0x800) KB                     
    L3:                      32768(0x8000) KB                   
  Chip ID:                 29679(0x73ef)                      
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   2450                               
  BDFID:                   768                                
  Internal Node ID:        1                                  
  Compute Unit:            32                                 
  SIMDs per CU:            2                                  
  Shader Engines:          2                                  
  Shader Arrs. per Eng.:   2                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    FALSE                              
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          32(0x20)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    1024(0x400)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        4294967295(0xffffffff)             
    y                        4294967295(0xffffffff)             
    z                        4294967295(0xffffffff)             
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 116                                
  SDMA engine uCode::      76                                 
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    8372224(0x7fc000) KB               
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    8372224(0x7fc000) KB               
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx1032         
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
*******                  
Agent 3                  
*******                  
  Name:                    gfx1035                            
  Uuid:                    GPU-XX                             
  Marketing Name:          AMD Radeon Graphics                
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    2                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      16(0x10) KB                        
    L2:                      2048(0x800) KB                     
  Chip ID:                 5761(0x1681)                       
  ASIC Revision:           2(0x2)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   2200                               
  BDFID:                   1792                               
  Internal Node ID:        2                                  
  Compute Unit:            12                                 
  SIMDs per CU:            2                                  
  Shader Engines:          1                                  
  Shader Arrs. per Eng.:   2                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    FALSE                              
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          32(0x20)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    1024(0x400)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        4294967295(0xffffffff)             
    y                        4294967295(0xffffffff)             
    z                        4294967295(0xffffffff)             
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 116                                
  SDMA engine uCode::      47                                 
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    524288(0x80000) KB                 
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    524288(0x80000) KB                 
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx1035         
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
*** Done ***

If I set the entry 0:

(ins)[~] HIP_VISIBLE_DEVICES=0 ollama serve
time=2024-04-23T13:52:54.253+09:00 level=INFO source=images.go:817 msg="total blobs: 0"
time=2024-04-23T13:52:54.253+09:00 level=INFO source=images.go:824 msg="total unused blobs removed: 0"
time=2024-04-23T13:52:54.253+09:00 level=INFO source=routes.go:1143 msg="Listening on 127.0.0.1:11434 (version 0.1.32)"
time=2024-04-23T13:52:54.327+09:00 level=INFO source=payload.go:28 msg="extracting embedded files" dir=/tmp/ollama1769714599/runners
time=2024-04-23T13:52:56.441+09:00 level=INFO source=payload.go:41 msg="Dynamic LLM libraries [rocm_v60002 cpu cpu_avx cpu_avx2 cuda_v11]"
time=2024-04-23T13:52:56.442+09:00 level=INFO source=gpu.go:121 msg="Detecting GPU type"
time=2024-04-23T13:52:56.442+09:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libcudart.so*"
time=2024-04-23T13:52:56.447+09:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [/tmp/ollama1769714599/runners/cuda_v11/libcudart.so.11.0]"
time=2024-04-23T13:52:56.447+09:00 level=INFO source=gpu.go:343 msg="Unable to load cudart CUDA management library /tmp/ollama1769714599/runners/cuda_v11/libcudart.so.11.0: your nvidia driver is too old or missing, please upgrade to run ollama"
time=2024-04-23T13:52:56.447+09:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libnvidia-ml.so"
time=2024-04-23T13:52:56.453+09:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: []"
time=2024-04-23T13:52:56.453+09:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-04-23T13:52:56.453+09:00 level=WARN source=amd_linux.go:53 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers: amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
time=2024-04-23T13:52:56.453+09:00 level=INFO source=amd_linux.go:263 msg="[0] amdgpu totalMemory 8176M"
time=2024-04-23T13:52:56.453+09:00 level=INFO source=amd_linux.go:264 msg="[0] amdgpu freeMemory  8176M"

If I set the entry 1:

(ins)[~] HIP_VISIBLE_DEVICES=1 ollama serve
time=2024-04-23T13:54:03.009+09:00 level=INFO source=images.go:817 msg="total blobs: 0"
time=2024-04-23T13:54:03.010+09:00 level=INFO source=images.go:824 msg="total unused blobs removed: 0"
time=2024-04-23T13:54:03.010+09:00 level=INFO source=routes.go:1143 msg="Listening on 127.0.0.1:11434 (version 0.1.32)"
time=2024-04-23T13:54:03.010+09:00 level=INFO source=payload.go:28 msg="extracting embedded files" dir=/tmp/ollama1225099153/runners
time=2024-04-23T13:54:05.123+09:00 level=INFO source=payload.go:41 msg="Dynamic LLM libraries [cuda_v11 rocm_v60002 cpu cpu_avx cpu_avx2]"
time=2024-04-23T13:54:05.123+09:00 level=INFO source=gpu.go:121 msg="Detecting GPU type"
time=2024-04-23T13:54:05.123+09:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libcudart.so*"
time=2024-04-23T13:54:05.129+09:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [/tmp/ollama1225099153/runners/cuda_v11/libcudart.so.11.0]"
time=2024-04-23T13:54:05.129+09:00 level=INFO source=gpu.go:343 msg="Unable to load cudart CUDA management library /tmp/ollama1225099153/runners/cuda_v11/libcudart.so.11.0: your nvidia driver is too old or missing, please upgrade to run ollama"
time=2024-04-23T13:54:05.129+09:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libnvidia-ml.so"
time=2024-04-23T13:54:05.135+09:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: []"
time=2024-04-23T13:54:05.135+09:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-04-23T13:54:05.135+09:00 level=WARN source=amd_linux.go:53 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers: amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
time=2024-04-23T13:54:05.135+09:00 level=INFO source=amd_linux.go:234 msg="amdgpu [1] appears to be an iGPU with 512M reported total memory, skipping"
panic: assignment to entry in nil map

goroutine 1 [running]:
github.com/ollama/ollama/gpu.amdProcMemLookup(0xc0007659a8, 0x0, {0xc000003c90, 0x1, 0x1})
	github.com/ollama/ollama/gpu/amd_linux.go:235 +0x697
github.com/ollama/ollama/gpu.AMDGetGPUInfo(0xc0007659a8)
	github.com/ollama/ollama/gpu/amd_linux.go:68 +0x1246
github.com/ollama/ollama/gpu.GetGPUInfo()
	github.com/ollama/ollama/gpu/gpu.go:210 +0x83c
github.com/ollama/ollama/gpu.CheckVRAM()
	github.com/ollama/ollama/gpu/gpu.go:256 +0x18c
github.com/ollama/ollama/server.Serve({0x11f3d528, 0xc00045d200})
	github.com/ollama/ollama/server/routes.go:1163 +0x493
github.com/ollama/ollama/cmd.RunServer(0xc0001e6c00?, {0x127bc1c0?, 0x4?, 0x11bf809?})
	github.com/ollama/ollama/cmd/cmd.go:816 +0x1b9
github.com/spf13/cobra.(*Command).execute(0xc000540608, {0x127bc1c0, 0x0, 0x0})
	github.com/spf13/cobra@v1.7.0/command.go:940 +0x882
github.com/spf13/cobra.(*Command).ExecuteC(0xc00017f508)
	github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/cobra@v1.7.0/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
	github.com/spf13/cobra@v1.7.0/command.go:985
main.main()
	github.com/ollama/ollama/main.go:11 +0x4d

If I set the entry 2:

(ins)[~] HIP_VISIBLE_DEVICES=2 ollama serve
time=2024-04-23T13:54:37.171+09:00 level=INFO source=images.go:817 msg="total blobs: 0"
time=2024-04-23T13:54:37.171+09:00 level=INFO source=images.go:824 msg="total unused blobs removed: 0"
time=2024-04-23T13:54:37.171+09:00 level=INFO source=routes.go:1143 msg="Listening on 127.0.0.1:11434 (version 0.1.32)"
time=2024-04-23T13:54:37.249+09:00 level=INFO source=payload.go:28 msg="extracting embedded files" dir=/tmp/ollama1175570358/runners
time=2024-04-23T13:54:39.365+09:00 level=INFO source=payload.go:41 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11 rocm_v60002]"
time=2024-04-23T13:54:39.365+09:00 level=INFO source=gpu.go:121 msg="Detecting GPU type"
time=2024-04-23T13:54:39.365+09:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libcudart.so*"
time=2024-04-23T13:54:39.371+09:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [/tmp/ollama1175570358/runners/cuda_v11/libcudart.so.11.0]"
time=2024-04-23T13:54:39.372+09:00 level=INFO source=gpu.go:343 msg="Unable to load cudart CUDA management library /tmp/ollama1175570358/runners/cuda_v11/libcudart.so.11.0: your nvidia driver is too old or missing, please upgrade to run ollama"
time=2024-04-23T13:54:39.372+09:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libnvidia-ml.so"
time=2024-04-23T13:54:39.377+09:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: []"
time=2024-04-23T13:54:39.377+09:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-04-23T13:54:39.377+09:00 level=WARN source=amd_linux.go:53 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers: amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
time=2024-04-23T13:54:39.377+09:00 level=WARN source=amd_linux.go:229 msg="amdgpu [2] reports zero total memory, skipping"
panic: assignment to entry in nil map

goroutine 1 [running]:
github.com/ollama/ollama/gpu.amdProcMemLookup(0xc00074b9a8, 0x0, {0xc000201bf0, 0x1, 0x1})
	github.com/ollama/ollama/gpu/amd_linux.go:230 +0x8e5
github.com/ollama/ollama/gpu.AMDGetGPUInfo(0xc00074b9a8)
	github.com/ollama/ollama/gpu/amd_linux.go:68 +0x1246
github.com/ollama/ollama/gpu.GetGPUInfo()
	github.com/ollama/ollama/gpu/gpu.go:210 +0x83c
github.com/ollama/ollama/gpu.CheckVRAM()
	github.com/ollama/ollama/gpu/gpu.go:256 +0x18c
github.com/ollama/ollama/server.Serve({0x11f3d528, 0xc000625060})
	github.com/ollama/ollama/server/routes.go:1163 +0x493
github.com/ollama/ollama/cmd.RunServer(0xc000518b00?, {0x127bc1c0?, 0x4?, 0x11bf809?})
	github.com/ollama/ollama/cmd/cmd.go:816 +0x1b9
github.com/spf13/cobra.(*Command).execute(0xc0004b7508, {0x127bc1c0, 0x0, 0x0})
	github.com/spf13/cobra@v1.7.0/command.go:940 +0x882
github.com/spf13/cobra.(*Command).ExecuteC(0xc0004b6908)
	github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5
github.com/spf13/cobra.(*Command).Execute(...)
	github.com/spf13/cobra@v1.7.0/command.go:992
github.com/spf13/cobra.(*Command).ExecuteContext(...)
	github.com/spf13/cobra@v1.7.0/command.go:985
main.main()
	github.com/ollama/ollama/main.go:11 +0x4d

Here are my current packages:

(ins)[~] pacman -Qs 'amd|hip|rocm|opencl|clblast|llama'
local/amd-ucode 20240409.1addd7dc-1
    Microcode update image for AMD CPUs
local/amdvlk 2024.Q1.3-2
    AMD's standalone Vulkan driver
local/clblast 1.6.2-1
    Tuned OpenCL BLAS library
local/clinfo 3.0.23.01.25-1
    Simple OpenCL application that enumerates all available platform and device properties
local/comgr 6.0.2-1
    Compiler support library for ROCm LLVM
local/composable-kernel 6.0.2-1
    High Performance Composable Kernel for AMD GPUs
local/flashrom 1.3.0-3
    Utility for reading, writing, erasing and verifying flash ROM chips
local/gcc-libs 13.2.1-5
    Runtime libraries shipped by GCC
local/hip-runtime-amd 6.0.2-2
    Heterogeneous Interface for Portability ROCm
local/hipblas 6.0.2-1
    ROCm BLAS marshalling library
local/hipcub 6.0.2-1
    Header-only library on top of rocPRIM or CUB
local/hipfft 6.0.2-1
    rocFFT marshalling library.
local/hiprand 6.0.2-1
    rocRAND marshalling library
local/hipsolver 6.0.2-1
    rocSOLVER marshalling library.
local/hipsparse 6.0.2-1
    rocSPARSE marshalling library.
local/hsa-rocr 6.0.2-2
    HSA Runtime API and runtime for ROCm
local/lib32-amdvlk 2024.Q1.3-2
    AMD's standalone Vulkan driver
local/lib32-gcc-libs 13.2.1-5
    32-bit runtime libraries shipped by GCC
local/libftdi 1.5-5
    A library to talk to FTDI chips, optional python bindings.
local/libteam 1.32-1
    Library for controlling team network device
local/magma-hip 2.7.2-4
    Matrix Algebra on GPU and Multicore Architectures (with ROCm/HIP)
local/miopen-hip 6.0.2-1
    AMD's Machine Intelligence Library (HIP backend)
local/nvtop 3.1.0-1
    GPUs process monitoring for AMD, Intel and NVIDIA
local/ocl-icd 2.3.2-1
    OpenCL ICD Bindings
local/opencl-headers 2:2023.04.17-2
    OpenCL (Open Computing Language) header files
local/python-pytorch-opt-rocm 2.2.2-1
    Tensors and Dynamic neural networks in Python with strong GPU acceleration (with ROCm and AVX2 CPU optimizations)
local/rccl 6.0.2-1
    ROCm Communication Collectives Library
local/rocalution 6.0.2-1
    Next generation library for iterative sparse solvers for ROCm platform
local/rocblas 6.0.2-1
    Next generation BLAS implementation for ROCm platform
local/rocfft 6.0.2-1
    Next generation FFT implementation for ROCm
local/rocm-clang-ocl 6.0.2-1
    OpenCL compilation with clang compiler
local/rocm-cmake 6.0.2-1
    CMake modules for common build tasks needed for the ROCm software stack
local/rocm-core 6.0.2-2
    AMD ROCm core package (version files)
local/rocm-device-libs 6.0.2-1
    ROCm Device Libraries
local/rocm-hip-libraries 6.0.2-1
    Develop certain applications using HIP and libraries for AMD platforms
local/rocm-hip-runtime 6.0.2-1
    Packages to run HIP applications on the AMD platform
local/rocm-hip-sdk 6.0.2-1
    Develop applications using HIP and libraries for AMD platforms
local/rocm-language-runtime 6.0.2-1
    ROCm runtime
local/rocm-llvm 6.0.2-1
    Radeon Open Compute - LLVM toolchain (llvm, clang, lld)
local/rocm-opencl-runtime 6.0.2-1
    OpenCL implementation for AMD
local/rocm-opencl-sdk 6.0.2-1
    Develop OpenCL-based applications for AMD platforms
local/rocm-smi-lib 6.0.2-1
    ROCm System Management Interface Library
local/rocminfo 6.0.2-1
    ROCm Application for Reporting System Info
local/rocprim 6.0.2-1
    Header-only library providing HIP parallel primitives
local/rocrand 6.0.2-1
    Pseudo-random and quasi-random number generator on ROCm
local/rocsolver 6.0.2-1
    Subset of LAPACK functionality on the ROCm platform
local/rocsparse 6.0.2-2
    BLAS for sparse computation on top of ROCm
local/rocthrust 6.0.2-1
    Port of the Thrust parallel algorithm library atop HIP/ROCm
local/roctracer 6.0.2-1
    ROCm tracer library for performance tracing
local/supergfxctl 5.1.1-1
    A utility for Linux graphics switching on Intel/AMD iGPU + nVidia dGPU laptops
local/texlive-latexextra 2024.2-1 (texlive)
    TeX Live - LaTeX additional packages
local/vulkan-radeon 1:24.0.5-1
    Open-source Vulkan driver for AMD GPUs
local/xf86-video-amdgpu 23.0.0-2 (xorg-drivers)
    X.org amdgpu video driver

OS

Linux

GPU

AMD

CPU

AMD

Ollama version

0.1.32

Originally created by @arael on GitHub (Apr 23, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3838 Originally assigned to: @dhiltgen on GitHub. ### What is the issue? I am not able to use my AMD Radeon RX 6800S with ollama. When I try, it falls back to CPU. I have installed tried both ollama and a fresh install with the scripts/install.sh from the git repo. The result is the same. Please help me. ### Command outputs and logs Here is the output of rocminfo: ``` (cmd)[~] rocminfo ROCk module is loaded ===================== HSA System Attributes ===================== Runtime Version: 1.1 System Timestamp Freq.: 1000.000000MHz Sig. Max Wait Duration: 18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count) Machine Model: LARGE System Endianness: LITTLE Mwaitx: DISABLED DMAbuf Support: YES ========== HSA Agents ========== ******* Agent 1 ******* Name: AMD Ryzen 7 6800HS with Radeon Graphics Uuid: CPU-XX Marketing Name: AMD Ryzen 7 6800HS with Radeon Graphics Vendor Name: CPU Feature: None specified Profile: FULL_PROFILE Float Round Mode: NEAR Max Queue Number: 0(0x0) Queue Min Size: 0(0x0) Queue Max Size: 0(0x0) Queue Type: MULTI Node: 0 Device Type: CPU Cache Info: L1: 32768(0x8000) KB Chip ID: 0(0x0) ASIC Revision: 0(0x0) Cacheline Size: 64(0x40) Max Clock Freq. (MHz): 4785 BDFID: 0 Internal Node ID: 0 Compute Unit: 16 SIMDs per CU: 0 Shader Engines: 0 Shader Arrs. per Eng.: 0 WatchPts on Addr. Ranges:1 Features: None Pool Info: Pool 1 Segment: GLOBAL; FLAGS: FINE GRAINED Size: 32086940(0x1e99b9c) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Alignment: 4KB Accessible by all: TRUE Pool 2 Segment: GLOBAL; FLAGS: KERNARG, FINE GRAINED Size: 32086940(0x1e99b9c) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Alignment: 4KB Accessible by all: TRUE Pool 3 Segment: GLOBAL; FLAGS: COARSE GRAINED Size: 32086940(0x1e99b9c) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Alignment: 4KB Accessible by all: TRUE ISA Info: ******* Agent 2 ******* Name: gfx1032 Uuid: GPU-XX Marketing Name: AMD Radeon RX 6800S Vendor Name: AMD Feature: KERNEL_DISPATCH Profile: BASE_PROFILE Float Round Mode: NEAR Max Queue Number: 128(0x80) Queue Min Size: 64(0x40) Queue Max Size: 131072(0x20000) Queue Type: MULTI Node: 1 Device Type: GPU Cache Info: L1: 16(0x10) KB L2: 2048(0x800) KB L3: 32768(0x8000) KB Chip ID: 29679(0x73ef) ASIC Revision: 0(0x0) Cacheline Size: 64(0x40) Max Clock Freq. (MHz): 2450 BDFID: 768 Internal Node ID: 1 Compute Unit: 32 SIMDs per CU: 2 Shader Engines: 2 Shader Arrs. per Eng.: 2 WatchPts on Addr. Ranges:4 Coherent Host Access: FALSE Features: KERNEL_DISPATCH Fast F16 Operation: TRUE Wavefront Size: 32(0x20) Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Max Waves Per CU: 32(0x20) Max Work-item Per CU: 1024(0x400) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) Max fbarriers/Workgrp: 32 Packet Processor uCode:: 116 SDMA engine uCode:: 76 IOMMU Support:: None Pool Info: Pool 1 Segment: GLOBAL; FLAGS: COARSE GRAINED Size: 8372224(0x7fc000) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Alignment: 4KB Accessible by all: FALSE Pool 2 Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED Size: 8372224(0x7fc000) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Alignment: 4KB Accessible by all: FALSE Pool 3 Segment: GROUP Size: 64(0x40) KB Allocatable: FALSE Alloc Granule: 0KB Alloc Alignment: 0KB Accessible by all: FALSE ISA Info: ISA 1 Name: amdgcn-amd-amdhsa--gfx1032 Machine Models: HSA_MACHINE_MODEL_LARGE Profiles: HSA_PROFILE_BASE Default Rounding Mode: NEAR Default Rounding Mode: NEAR Fast f16: TRUE Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) FBarrier Max Size: 32 ******* Agent 3 ******* Name: gfx1035 Uuid: GPU-XX Marketing Name: AMD Radeon Graphics Vendor Name: AMD Feature: KERNEL_DISPATCH Profile: BASE_PROFILE Float Round Mode: NEAR Max Queue Number: 128(0x80) Queue Min Size: 64(0x40) Queue Max Size: 131072(0x20000) Queue Type: MULTI Node: 2 Device Type: GPU Cache Info: L1: 16(0x10) KB L2: 2048(0x800) KB Chip ID: 5761(0x1681) ASIC Revision: 2(0x2) Cacheline Size: 64(0x40) Max Clock Freq. (MHz): 2200 BDFID: 1792 Internal Node ID: 2 Compute Unit: 12 SIMDs per CU: 2 Shader Engines: 1 Shader Arrs. per Eng.: 2 WatchPts on Addr. Ranges:4 Coherent Host Access: FALSE Features: KERNEL_DISPATCH Fast F16 Operation: TRUE Wavefront Size: 32(0x20) Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Max Waves Per CU: 32(0x20) Max Work-item Per CU: 1024(0x400) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) Max fbarriers/Workgrp: 32 Packet Processor uCode:: 116 SDMA engine uCode:: 47 IOMMU Support:: None Pool Info: Pool 1 Segment: GLOBAL; FLAGS: COARSE GRAINED Size: 524288(0x80000) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Alignment: 4KB Accessible by all: FALSE Pool 2 Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED Size: 524288(0x80000) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Alignment: 4KB Accessible by all: FALSE Pool 3 Segment: GROUP Size: 64(0x40) KB Allocatable: FALSE Alloc Granule: 0KB Alloc Alignment: 0KB Accessible by all: FALSE ISA Info: ISA 1 Name: amdgcn-amd-amdhsa--gfx1035 Machine Models: HSA_MACHINE_MODEL_LARGE Profiles: HSA_PROFILE_BASE Default Rounding Mode: NEAR Default Rounding Mode: NEAR Fast f16: TRUE Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) FBarrier Max Size: 32 *** Done *** ``` If I set the entry 0: ``` (ins)[~] HIP_VISIBLE_DEVICES=0 ollama serve time=2024-04-23T13:52:54.253+09:00 level=INFO source=images.go:817 msg="total blobs: 0" time=2024-04-23T13:52:54.253+09:00 level=INFO source=images.go:824 msg="total unused blobs removed: 0" time=2024-04-23T13:52:54.253+09:00 level=INFO source=routes.go:1143 msg="Listening on 127.0.0.1:11434 (version 0.1.32)" time=2024-04-23T13:52:54.327+09:00 level=INFO source=payload.go:28 msg="extracting embedded files" dir=/tmp/ollama1769714599/runners time=2024-04-23T13:52:56.441+09:00 level=INFO source=payload.go:41 msg="Dynamic LLM libraries [rocm_v60002 cpu cpu_avx cpu_avx2 cuda_v11]" time=2024-04-23T13:52:56.442+09:00 level=INFO source=gpu.go:121 msg="Detecting GPU type" time=2024-04-23T13:52:56.442+09:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libcudart.so*" time=2024-04-23T13:52:56.447+09:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [/tmp/ollama1769714599/runners/cuda_v11/libcudart.so.11.0]" time=2024-04-23T13:52:56.447+09:00 level=INFO source=gpu.go:343 msg="Unable to load cudart CUDA management library /tmp/ollama1769714599/runners/cuda_v11/libcudart.so.11.0: your nvidia driver is too old or missing, please upgrade to run ollama" time=2024-04-23T13:52:56.447+09:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libnvidia-ml.so" time=2024-04-23T13:52:56.453+09:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: []" time=2024-04-23T13:52:56.453+09:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-04-23T13:52:56.453+09:00 level=WARN source=amd_linux.go:53 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers: amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" time=2024-04-23T13:52:56.453+09:00 level=INFO source=amd_linux.go:263 msg="[0] amdgpu totalMemory 8176M" time=2024-04-23T13:52:56.453+09:00 level=INFO source=amd_linux.go:264 msg="[0] amdgpu freeMemory 8176M" ``` If I set the entry 1: ``` (ins)[~] HIP_VISIBLE_DEVICES=1 ollama serve time=2024-04-23T13:54:03.009+09:00 level=INFO source=images.go:817 msg="total blobs: 0" time=2024-04-23T13:54:03.010+09:00 level=INFO source=images.go:824 msg="total unused blobs removed: 0" time=2024-04-23T13:54:03.010+09:00 level=INFO source=routes.go:1143 msg="Listening on 127.0.0.1:11434 (version 0.1.32)" time=2024-04-23T13:54:03.010+09:00 level=INFO source=payload.go:28 msg="extracting embedded files" dir=/tmp/ollama1225099153/runners time=2024-04-23T13:54:05.123+09:00 level=INFO source=payload.go:41 msg="Dynamic LLM libraries [cuda_v11 rocm_v60002 cpu cpu_avx cpu_avx2]" time=2024-04-23T13:54:05.123+09:00 level=INFO source=gpu.go:121 msg="Detecting GPU type" time=2024-04-23T13:54:05.123+09:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libcudart.so*" time=2024-04-23T13:54:05.129+09:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [/tmp/ollama1225099153/runners/cuda_v11/libcudart.so.11.0]" time=2024-04-23T13:54:05.129+09:00 level=INFO source=gpu.go:343 msg="Unable to load cudart CUDA management library /tmp/ollama1225099153/runners/cuda_v11/libcudart.so.11.0: your nvidia driver is too old or missing, please upgrade to run ollama" time=2024-04-23T13:54:05.129+09:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libnvidia-ml.so" time=2024-04-23T13:54:05.135+09:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: []" time=2024-04-23T13:54:05.135+09:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-04-23T13:54:05.135+09:00 level=WARN source=amd_linux.go:53 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers: amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" time=2024-04-23T13:54:05.135+09:00 level=INFO source=amd_linux.go:234 msg="amdgpu [1] appears to be an iGPU with 512M reported total memory, skipping" panic: assignment to entry in nil map goroutine 1 [running]: github.com/ollama/ollama/gpu.amdProcMemLookup(0xc0007659a8, 0x0, {0xc000003c90, 0x1, 0x1}) github.com/ollama/ollama/gpu/amd_linux.go:235 +0x697 github.com/ollama/ollama/gpu.AMDGetGPUInfo(0xc0007659a8) github.com/ollama/ollama/gpu/amd_linux.go:68 +0x1246 github.com/ollama/ollama/gpu.GetGPUInfo() github.com/ollama/ollama/gpu/gpu.go:210 +0x83c github.com/ollama/ollama/gpu.CheckVRAM() github.com/ollama/ollama/gpu/gpu.go:256 +0x18c github.com/ollama/ollama/server.Serve({0x11f3d528, 0xc00045d200}) github.com/ollama/ollama/server/routes.go:1163 +0x493 github.com/ollama/ollama/cmd.RunServer(0xc0001e6c00?, {0x127bc1c0?, 0x4?, 0x11bf809?}) github.com/ollama/ollama/cmd/cmd.go:816 +0x1b9 github.com/spf13/cobra.(*Command).execute(0xc000540608, {0x127bc1c0, 0x0, 0x0}) github.com/spf13/cobra@v1.7.0/command.go:940 +0x882 github.com/spf13/cobra.(*Command).ExecuteC(0xc00017f508) github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 github.com/spf13/cobra.(*Command).Execute(...) github.com/spf13/cobra@v1.7.0/command.go:992 github.com/spf13/cobra.(*Command).ExecuteContext(...) github.com/spf13/cobra@v1.7.0/command.go:985 main.main() github.com/ollama/ollama/main.go:11 +0x4d ``` If I set the entry 2: ``` (ins)[~] HIP_VISIBLE_DEVICES=2 ollama serve time=2024-04-23T13:54:37.171+09:00 level=INFO source=images.go:817 msg="total blobs: 0" time=2024-04-23T13:54:37.171+09:00 level=INFO source=images.go:824 msg="total unused blobs removed: 0" time=2024-04-23T13:54:37.171+09:00 level=INFO source=routes.go:1143 msg="Listening on 127.0.0.1:11434 (version 0.1.32)" time=2024-04-23T13:54:37.249+09:00 level=INFO source=payload.go:28 msg="extracting embedded files" dir=/tmp/ollama1175570358/runners time=2024-04-23T13:54:39.365+09:00 level=INFO source=payload.go:41 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11 rocm_v60002]" time=2024-04-23T13:54:39.365+09:00 level=INFO source=gpu.go:121 msg="Detecting GPU type" time=2024-04-23T13:54:39.365+09:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libcudart.so*" time=2024-04-23T13:54:39.371+09:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: [/tmp/ollama1175570358/runners/cuda_v11/libcudart.so.11.0]" time=2024-04-23T13:54:39.372+09:00 level=INFO source=gpu.go:343 msg="Unable to load cudart CUDA management library /tmp/ollama1175570358/runners/cuda_v11/libcudart.so.11.0: your nvidia driver is too old or missing, please upgrade to run ollama" time=2024-04-23T13:54:39.372+09:00 level=INFO source=gpu.go:268 msg="Searching for GPU management library libnvidia-ml.so" time=2024-04-23T13:54:39.377+09:00 level=INFO source=gpu.go:314 msg="Discovered GPU libraries: []" time=2024-04-23T13:54:39.377+09:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-04-23T13:54:39.377+09:00 level=WARN source=amd_linux.go:53 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers: amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" time=2024-04-23T13:54:39.377+09:00 level=WARN source=amd_linux.go:229 msg="amdgpu [2] reports zero total memory, skipping" panic: assignment to entry in nil map goroutine 1 [running]: github.com/ollama/ollama/gpu.amdProcMemLookup(0xc00074b9a8, 0x0, {0xc000201bf0, 0x1, 0x1}) github.com/ollama/ollama/gpu/amd_linux.go:230 +0x8e5 github.com/ollama/ollama/gpu.AMDGetGPUInfo(0xc00074b9a8) github.com/ollama/ollama/gpu/amd_linux.go:68 +0x1246 github.com/ollama/ollama/gpu.GetGPUInfo() github.com/ollama/ollama/gpu/gpu.go:210 +0x83c github.com/ollama/ollama/gpu.CheckVRAM() github.com/ollama/ollama/gpu/gpu.go:256 +0x18c github.com/ollama/ollama/server.Serve({0x11f3d528, 0xc000625060}) github.com/ollama/ollama/server/routes.go:1163 +0x493 github.com/ollama/ollama/cmd.RunServer(0xc000518b00?, {0x127bc1c0?, 0x4?, 0x11bf809?}) github.com/ollama/ollama/cmd/cmd.go:816 +0x1b9 github.com/spf13/cobra.(*Command).execute(0xc0004b7508, {0x127bc1c0, 0x0, 0x0}) github.com/spf13/cobra@v1.7.0/command.go:940 +0x882 github.com/spf13/cobra.(*Command).ExecuteC(0xc0004b6908) github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3a5 github.com/spf13/cobra.(*Command).Execute(...) github.com/spf13/cobra@v1.7.0/command.go:992 github.com/spf13/cobra.(*Command).ExecuteContext(...) github.com/spf13/cobra@v1.7.0/command.go:985 main.main() github.com/ollama/ollama/main.go:11 +0x4d ``` Here are my current packages: ``` (ins)[~] pacman -Qs 'amd|hip|rocm|opencl|clblast|llama' local/amd-ucode 20240409.1addd7dc-1 Microcode update image for AMD CPUs local/amdvlk 2024.Q1.3-2 AMD's standalone Vulkan driver local/clblast 1.6.2-1 Tuned OpenCL BLAS library local/clinfo 3.0.23.01.25-1 Simple OpenCL application that enumerates all available platform and device properties local/comgr 6.0.2-1 Compiler support library for ROCm LLVM local/composable-kernel 6.0.2-1 High Performance Composable Kernel for AMD GPUs local/flashrom 1.3.0-3 Utility for reading, writing, erasing and verifying flash ROM chips local/gcc-libs 13.2.1-5 Runtime libraries shipped by GCC local/hip-runtime-amd 6.0.2-2 Heterogeneous Interface for Portability ROCm local/hipblas 6.0.2-1 ROCm BLAS marshalling library local/hipcub 6.0.2-1 Header-only library on top of rocPRIM or CUB local/hipfft 6.0.2-1 rocFFT marshalling library. local/hiprand 6.0.2-1 rocRAND marshalling library local/hipsolver 6.0.2-1 rocSOLVER marshalling library. local/hipsparse 6.0.2-1 rocSPARSE marshalling library. local/hsa-rocr 6.0.2-2 HSA Runtime API and runtime for ROCm local/lib32-amdvlk 2024.Q1.3-2 AMD's standalone Vulkan driver local/lib32-gcc-libs 13.2.1-5 32-bit runtime libraries shipped by GCC local/libftdi 1.5-5 A library to talk to FTDI chips, optional python bindings. local/libteam 1.32-1 Library for controlling team network device local/magma-hip 2.7.2-4 Matrix Algebra on GPU and Multicore Architectures (with ROCm/HIP) local/miopen-hip 6.0.2-1 AMD's Machine Intelligence Library (HIP backend) local/nvtop 3.1.0-1 GPUs process monitoring for AMD, Intel and NVIDIA local/ocl-icd 2.3.2-1 OpenCL ICD Bindings local/opencl-headers 2:2023.04.17-2 OpenCL (Open Computing Language) header files local/python-pytorch-opt-rocm 2.2.2-1 Tensors and Dynamic neural networks in Python with strong GPU acceleration (with ROCm and AVX2 CPU optimizations) local/rccl 6.0.2-1 ROCm Communication Collectives Library local/rocalution 6.0.2-1 Next generation library for iterative sparse solvers for ROCm platform local/rocblas 6.0.2-1 Next generation BLAS implementation for ROCm platform local/rocfft 6.0.2-1 Next generation FFT implementation for ROCm local/rocm-clang-ocl 6.0.2-1 OpenCL compilation with clang compiler local/rocm-cmake 6.0.2-1 CMake modules for common build tasks needed for the ROCm software stack local/rocm-core 6.0.2-2 AMD ROCm core package (version files) local/rocm-device-libs 6.0.2-1 ROCm Device Libraries local/rocm-hip-libraries 6.0.2-1 Develop certain applications using HIP and libraries for AMD platforms local/rocm-hip-runtime 6.0.2-1 Packages to run HIP applications on the AMD platform local/rocm-hip-sdk 6.0.2-1 Develop applications using HIP and libraries for AMD platforms local/rocm-language-runtime 6.0.2-1 ROCm runtime local/rocm-llvm 6.0.2-1 Radeon Open Compute - LLVM toolchain (llvm, clang, lld) local/rocm-opencl-runtime 6.0.2-1 OpenCL implementation for AMD local/rocm-opencl-sdk 6.0.2-1 Develop OpenCL-based applications for AMD platforms local/rocm-smi-lib 6.0.2-1 ROCm System Management Interface Library local/rocminfo 6.0.2-1 ROCm Application for Reporting System Info local/rocprim 6.0.2-1 Header-only library providing HIP parallel primitives local/rocrand 6.0.2-1 Pseudo-random and quasi-random number generator on ROCm local/rocsolver 6.0.2-1 Subset of LAPACK functionality on the ROCm platform local/rocsparse 6.0.2-2 BLAS for sparse computation on top of ROCm local/rocthrust 6.0.2-1 Port of the Thrust parallel algorithm library atop HIP/ROCm local/roctracer 6.0.2-1 ROCm tracer library for performance tracing local/supergfxctl 5.1.1-1 A utility for Linux graphics switching on Intel/AMD iGPU + nVidia dGPU laptops local/texlive-latexextra 2024.2-1 (texlive) TeX Live - LaTeX additional packages local/vulkan-radeon 1:24.0.5-1 Open-source Vulkan driver for AMD GPUs local/xf86-video-amdgpu 23.0.0-2 (xorg-drivers) X.org amdgpu video driver ``` ### OS Linux ### GPU AMD ### CPU AMD ### Ollama version 0.1.32
GiteaMirror added the amdbug labels 2026-05-03 17:34:19 -05:00
GiteaMirror commented 2026-05-03 17:34:22 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Apr 24, 2024):

Thanks for reporting this!

This was fixed with #3418 which refactored the code in question that caused the panic. I just tested on a dual radeon test system I have and confirmed that without HIP_VISIBLE_DEVICES set, it can use both GPUs, and with it set to 0 or 1 correctly toggles which GPU gets used when the model loads. The fix will be in 0.1.33

<!-- gh-comment-id:2075307157 --> @dhiltgen commented on GitHub (Apr 24, 2024): Thanks for reporting this! This was fixed with #3418 which refactored the code in question that caused the panic. I just tested on a dual radeon test system I have and confirmed that without HIP_VISIBLE_DEVICES set, it can use both GPUs, and with it set to 0 or 1 correctly toggles which GPU gets used when the model loads. The fix will be in 0.1.33
GiteaMirror commented 2026-05-03 17:34:23 -05:00
Author
Owner
Copy Link

@arael commented on GitHub (Apr 25, 2024):

I tried running the current version after building it up. I don't think that any of my GPUs are used even by the latest commit. Would you kindly confirm?

Here is the output:

(ins)[ollama] (main) ./ollama serve
time=2024-04-25T10:12:33.130+09:00 level=INFO source=images.go:814 msg="total blobs: 0"
time=2024-04-25T10:12:33.131+09:00 level=INFO source=images.go:821 msg="total unused blobs removed: 0"
[GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.

[GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
 - using env:	export GIN_MODE=release
 - using code:	gin.SetMode(gin.ReleaseMode)

[GIN-debug] POST   /api/pull                 --> github.com/ollama/ollama/server.(*Server).PullModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/generate             --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers)
[GIN-debug] POST   /api/chat                 --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers)
[GIN-debug] POST   /api/embeddings           --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers)
[GIN-debug] POST   /api/create               --> github.com/ollama/ollama/server.(*Server).CreateModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/push                 --> github.com/ollama/ollama/server.(*Server).PushModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/copy                 --> github.com/ollama/ollama/server.(*Server).CopyModelHandler-fm (5 handlers)
[GIN-debug] DELETE /api/delete               --> github.com/ollama/ollama/server.(*Server).DeleteModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/show                 --> github.com/ollama/ollama/server.(*Server).ShowModelHandler-fm (5 handlers)
[GIN-debug] POST   /api/blobs/:digest        --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers)
[GIN-debug] HEAD   /api/blobs/:digest        --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers)
[GIN-debug] POST   /v1/chat/completions      --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers)
[GIN-debug] GET    /                         --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers)
[GIN-debug] GET    /api/tags                 --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers)
[GIN-debug] GET    /api/version              --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
[GIN-debug] HEAD   /                         --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers)
[GIN-debug] HEAD   /api/tags                 --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers)
[GIN-debug] HEAD   /api/version              --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
time=2024-04-25T10:12:33.131+09:00 level=INFO source=routes.go:1064 msg="Listening on 127.0.0.1:11434 (version 0.0.0)"
time=2024-04-25T10:12:33.131+09:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama462407758/runners
time=2024-04-25T10:12:33.338+09:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 rocm]"
time=2024-04-25T10:12:33.338+09:00 level=INFO source=gpu.go:96 msg="Detecting GPUs"
time=2024-04-25T10:12:33.347+09:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-04-25T10:12:33.347+09:00 level=WARN source=amd_linux.go:49 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
time=2024-04-25T10:12:33.347+09:00 level=INFO source=amd_linux.go:217 msg="amdgpu memory" gpu=0 total="8176.0 MiB"
time=2024-04-25T10:12:33.347+09:00 level=INFO source=amd_linux.go:218 msg="amdgpu memory" gpu=0 available="8176.0 MiB"
time=2024-04-25T10:12:33.366+09:00 level=WARN source=amd_linux.go:271 msg="amdgpu is not supported" gpu=0 gpu_type=gfx1032 library=/opt/rocm/lib supported_types="[gfx1030 gfx1100 gfx1101 gfx1102 gfx900 gfx906 gfx908 gfx90a gfx940 gfx941 gfx942]"
time=2024-04-25T10:12:33.366+09:00 level=WARN source=amd_linux.go:273 msg="See https://github.com/ollama/ollama/blob/main/docs/gpu.md#overrides for HSA_OVERRIDE_GFX_VERSION usage"
time=2024-04-25T10:12:33.367+09:00 level=INFO source=amd_linux.go:213 msg="amdgpu appears to be an iGPU, skipping" gpu=1 total="512.0 MiB"
time=2024-04-25T10:12:33.367+09:00 level=INFO source=amd_linux.go:286 msg="no compatible amdgpu devices detected"

Specifically, it looks like the list of supported GPUs is: gfx1030 gfx1100 gfx1101 gfx1102 gfx900 gfx906 gfx908 gfx90a gfx940 gfx941 gfx942

I on the other hand have: gfx1032 and gfx1035.

Is there anything I can do, or should I just give up and run the model on the CPU?

<!-- gh-comment-id:2076133193 --> @arael commented on GitHub (Apr 25, 2024): I tried running the current version after building it up. I don't think that any of my GPUs are used even by the latest commit. Would you kindly confirm? Here is the output: ``` (ins)[ollama] (main) ./ollama serve time=2024-04-25T10:12:33.130+09:00 level=INFO source=images.go:814 msg="total blobs: 0" time=2024-04-25T10:12:33.131+09:00 level=INFO source=images.go:821 msg="total unused blobs removed: 0" [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached. [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production. - using env: export GIN_MODE=release - using code: gin.SetMode(gin.ReleaseMode) [GIN-debug] POST /api/pull --> github.com/ollama/ollama/server.(*Server).PullModelHandler-fm (5 handlers) [GIN-debug] POST /api/generate --> github.com/ollama/ollama/server.(*Server).GenerateHandler-fm (5 handlers) [GIN-debug] POST /api/chat --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (5 handlers) [GIN-debug] POST /api/embeddings --> github.com/ollama/ollama/server.(*Server).EmbeddingsHandler-fm (5 handlers) [GIN-debug] POST /api/create --> github.com/ollama/ollama/server.(*Server).CreateModelHandler-fm (5 handlers) [GIN-debug] POST /api/push --> github.com/ollama/ollama/server.(*Server).PushModelHandler-fm (5 handlers) [GIN-debug] POST /api/copy --> github.com/ollama/ollama/server.(*Server).CopyModelHandler-fm (5 handlers) [GIN-debug] DELETE /api/delete --> github.com/ollama/ollama/server.(*Server).DeleteModelHandler-fm (5 handlers) [GIN-debug] POST /api/show --> github.com/ollama/ollama/server.(*Server).ShowModelHandler-fm (5 handlers) [GIN-debug] POST /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).CreateBlobHandler-fm (5 handlers) [GIN-debug] HEAD /api/blobs/:digest --> github.com/ollama/ollama/server.(*Server).HeadBlobHandler-fm (5 handlers) [GIN-debug] POST /v1/chat/completions --> github.com/ollama/ollama/server.(*Server).ChatHandler-fm (6 handlers) [GIN-debug] GET / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] GET /api/tags --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers) [GIN-debug] GET /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) [GIN-debug] HEAD / --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func1 (5 handlers) [GIN-debug] HEAD /api/tags --> github.com/ollama/ollama/server.(*Server).ListModelsHandler-fm (5 handlers) [GIN-debug] HEAD /api/version --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers) time=2024-04-25T10:12:33.131+09:00 level=INFO source=routes.go:1064 msg="Listening on 127.0.0.1:11434 (version 0.0.0)" time=2024-04-25T10:12:33.131+09:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama462407758/runners time=2024-04-25T10:12:33.338+09:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 rocm]" time=2024-04-25T10:12:33.338+09:00 level=INFO source=gpu.go:96 msg="Detecting GPUs" time=2024-04-25T10:12:33.347+09:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2" time=2024-04-25T10:12:33.347+09:00 level=WARN source=amd_linux.go:49 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" time=2024-04-25T10:12:33.347+09:00 level=INFO source=amd_linux.go:217 msg="amdgpu memory" gpu=0 total="8176.0 MiB" time=2024-04-25T10:12:33.347+09:00 level=INFO source=amd_linux.go:218 msg="amdgpu memory" gpu=0 available="8176.0 MiB" time=2024-04-25T10:12:33.366+09:00 level=WARN source=amd_linux.go:271 msg="amdgpu is not supported" gpu=0 gpu_type=gfx1032 library=/opt/rocm/lib supported_types="[gfx1030 gfx1100 gfx1101 gfx1102 gfx900 gfx906 gfx908 gfx90a gfx940 gfx941 gfx942]" time=2024-04-25T10:12:33.366+09:00 level=WARN source=amd_linux.go:273 msg="See https://github.com/ollama/ollama/blob/main/docs/gpu.md#overrides for HSA_OVERRIDE_GFX_VERSION usage" time=2024-04-25T10:12:33.367+09:00 level=INFO source=amd_linux.go:213 msg="amdgpu appears to be an iGPU, skipping" gpu=1 total="512.0 MiB" time=2024-04-25T10:12:33.367+09:00 level=INFO source=amd_linux.go:286 msg="no compatible amdgpu devices detected" ``` Specifically, it looks like the list of supported GPUs is: gfx1030 gfx1100 gfx1101 gfx1102 gfx900 gfx906 gfx908 gfx90a gfx940 gfx941 gfx942 I on the other hand have: gfx1032 and gfx1035. Is there anything I can do, or should I just give up and run the model on the CPU?
GiteaMirror commented 2026-05-03 17:34:24 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (May 1, 2024):

@arael see https://github.com/ollama/ollama/blob/main/docs/gpu.md#overrides

<!-- gh-comment-id:2088849717 --> @dhiltgen commented on GitHub (May 1, 2024): @arael see https://github.com/ollama/ollama/blob/main/docs/gpu.md#overrides
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label amd bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#64414
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?