[GH-ISSUE #6304] Latest version (0.3.4) not detecting AMD GPUs (Instinct MI210) #3953

New Issue

GiteaMirror · 2026-04-12T14:49:43-05:00

GiteaMirror commented

2026-04-12 14:49:43 -05:00

Originally created by @aimanyounises1 on GitHub (Aug 10, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6304

Originally assigned to: @dhiltgen on GitHub.

The latest version of Ollama (0.3.4) is not detecting or utilizing AMD GPUs (Instinct MI210) when run in a Docker container.

I tried to run it with docker because I didn't want to update the Ollama working version for me 0.2.8.

I followed the instruction here: https://hub.docker.com/r/ollama/ollama.

command docker run -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocm

Please see below the error (I tried as well before to update the Ollama with the latest version on my server but encountered the same issue)

Any insights or suggestions would be greatly appreciated!

[genai1~]$ docker exec 92ebd80dcef1 /opt/rocm/bin/rocm-smi
========================================== ROCm System Management Interface ==========================================
==================================================== Concise Info ====================================================
Device  Node  IDs              Temp    Power   Partitions          SCLK     MCLK     Fan  Perf  PwrCap  VRAM%  GPU%  
              (DID,     GUID)  (Edge)  (Avg)   (Mem, Compute, ID)                                                    
======================================================================================================================
Traceback (most recent call last):
  File "/opt/rocm/bin/rocm-smi", line 4074, in <module>
    showAllConcise(deviceList)
  File "/opt/rocm/bin/rocm-smi", line 2033, in showAllConcise
    zip(range(len(max_widths)), values['card%s' % (str(device))])), None)
  File "/opt/rocm/bin/rocm-smi", line 872, in printLog
    print(logstr + '\n', end='')
UnicodeEncodeError: 'ascii' codec can't encode character '\xb0' in position 35: ordinal not in range(128)

[genai1~]$ docker exec 92ebd80dcef1 ls -la /root/.ollama
total 20
drwxr-xr-x 3 root root 4096 Aug 10 19:17 .
dr-xr-x--- 1 root root 4096 Aug 10 19:46 ..
-rw------- 1 root root  387 Aug 10 19:17 id_ed25519
-rw-r--r-- 1 root root   81 Aug 10 19:17 id_ed25519.pub
drwxr-xr-x 4 root root 4096 Aug 10 19:55 models
[genai1~]$ docker logs 92ebd80dcef1
2docker logs 92ebd80dcef1
2024/08/10 19:46:51 routes.go:1108: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR: OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]"
time=2024-08-10T19:46:51.654Z level=INFO source=images.go:781 msg="total blobs: 0"
time=2024-08-10T19:46:51.654Z level=INFO source=images.go:788 msg="total unused blobs removed: 0"
time=2024-08-10T19:46:51.654Z level=INFO source=routes.go:1155 msg="Listening on [::]:11434 (version 0.3.4)"
time=2024-08-10T19:46:51.655Z level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama2612956243/runners
time=2024-08-10T19:46:54.488Z level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cuda_v11 rocm_v60102 cpu cpu_avx cpu_avx2]"
time=2024-08-10T19:46:54.488Z level=INFO source=gpu.go:204 msg="looking for compatible GPUs"
time=2024-08-10T19:46:54.491Z level=WARN source=amd_linux.go:59 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
time=2024-08-10T19:46:54.491Z level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=0 total="0 B"
time=2024-08-10T19:46:54.492Z level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=1 total="0 B"
time=2024-08-10T19:46:54.492Z level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=2 total="0 B"
time=2024-08-10T19:46:54.492Z level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=3 total="0 B"
time=2024-08-10T19:46:54.492Z level=INFO source=amd_linux.go:360 msg="no compatible amdgpu devices detected"
time=2024-08-10T19:46:54.492Z level=INFO source=gpu.go:347 msg="no compatible GPUs were discovered"
time=2024-08-10T19:46:54.492Z level=INFO source=types.go:105 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="250.9 GiB" available="230.3 GiB"

Error: something went wrong, please see the ollama server logs for details
[genai1~]$ docker exec 92ebd80dcef1 env | grep OLLAMA
OLLAMA_HOST=0.0.0.0
[genai1~]$ docker exec 92ebd80dcef1 ls -l /opt/rocm/bin
total 78928
-rwxr-xr-x 1 root root  3993744 May 28 21:10 MIOpenDriver
lrwxrwxrwx 1 root root       35 Jun 10 14:56 amd-smi -> ../libexec/amdsmi_cli/amdsmi_cli.py
lrwxrwxrwx 1 root root       24 Jun 10 14:51 amdclang -> ../lib/llvm/bin/amdclang
lrwxrwxrwx 1 root root       26 Jun 10 14:51 amdclang++ -> ../lib/llvm/bin/amdclang++
lrwxrwxrwx 1 root root       27 Jun 10 14:51 amdclang-cl -> ../lib/llvm/bin/amdclang-cl
lrwxrwxrwx 1 root root       28 Jun 10 14:51 amdclang-cpp -> ../lib/llvm/bin/amdclang-cpp
lrwxrwxrwx 1 root root       24 Jun 10 14:51 amdflang -> ../lib/llvm/bin/amdflang
lrwxrwxrwx 1 root root       22 Jun 10 14:51 amdlld -> ../lib/llvm/bin/amdlld
lrwxrwxrwx 1 root root       22 Jun 10 14:53 aompcc -> ../lib/llvm/bin/aompcc
-rwxr-xr-x 1 root root   107592 May 28 19:14 clinfo
-rwxrwxr-x 1 root root     2551 May 28 18:08 hipcc
-rwxr-xr-x 1 root root   575472 May 28 18:11 hipcc.bin
-rwxrwxr-x 1 root root    21889 May 28 18:08 hipcc.pl
-rwxrwxr-x 1 root root     1508 May 28 18:08 hipcc_cmake_linker_helper
-rwxrwxr-x 1 root root     2453 May 28 18:08 hipconfig
-rwxr-xr-x 1 root root   575480 May 28 18:11 hipconfig.bin
-rwxrwxr-x 1 root root     7539 May 28 18:08 hipconfig.pl
-rwxr-xr-x 1 root root      784 May 28 18:08 hipconvertinplace-perl.sh
-rwxr-xr-x 1 root root      674 May 28 18:08 hipconvertinplace.sh
-rwxrwxr-x 1 root root     1857 May 28 18:08 hipdemangleatp
-rwxr-xr-x 1 root root      407 May 28 18:08 hipexamine-perl.sh
-rwxr-xr-x 1 root root      557 May 28 18:08 hipexamine.sh
-rwxr-xr-x 1 root root 62142024 May 28 19:14 hipify-clang
-rwxr-xr-x 1 root root   659191 May 28 18:08 hipify-perl
-rw-rw-r-- 1 root root     6486 May 28 18:08 hipvars.pm
-rwxr-xr-x 1 root root     1332 May 28 19:30 install_precompiled_kernels.sh
lrwxrwxrwx 1 root root       21 Jun 10 14:53 mygpu -> ../lib/llvm/bin/mygpu
lrwxrwxrwx 1 root root       22 Jun 10 14:53 mymcpu -> ../lib/llvm/bin/mymcpu
lrwxrwxrwx 1 root root       28 Jun 10 14:51 offload-arch -> ../lib/llvm/bin/offload-arch
-rwxrwxr-x 1 root root    10047 May 28 18:08 roc-obj
-rwxrwxr-x 1 root root     8457 May 28 18:08 roc-obj-extract
-rwxrwxr-x 1 root root       71 May 28 18:08 roc-obj-extract.bat
-rwxrwxr-x 1 root root     6973 May 28 18:08 roc-obj-ls
-rwxrwxr-x 1 root root       56 May 28 18:08 roc-obj-ls.bat
-rwxr-xr-x 1 root root 12314744 May 28 18:57 rocgdb
lrwxrwxrwx 1 root root       31 Jun 10 14:52 rocm-smi -> ../libexec/rocm_smi/rocm_smi.py
-rwxr-xr-x 1 root root    10057 May 28 18:50 rocm_agent_enumerator
-rwxr-xr-x 1 root root   159560 May 28 18:50 rocminfo
-r-xr-xr-x 1 root root    22019 May 28 18:08 rocprof
-r-xr-xr-x 1 root root    11523 May 28 18:08 rocprofv2
-rwxr-xr-x 1 root root    96000 May 28 19:17 rocsys
-rwxr-xr-x 1 root root     9224 May 28 19:30 test_perf.py
[genai1@~]$ docker exec 92ebd80dcef1 ls -l /opt/rocm/lib
total 10486408
drwxr-xr-x 32 root root       4096 Jun 10 14:56 cmake
drwxr-xr-x  3 root root       4096 Jun 10 14:55 hipblaslt
drwxr-xr-x  3 root root       4096 Jun 10 14:54 hipsparselt
drwxr-xr-x  2 root root       4096 Jun 10 14:52 hsa-amd-aqlprofile
lrwxrwxrwx  1 root root         14 Jun 10 14:55 libMIOpen.so -> libMIOpen.so.1
lrwxrwxrwx  1 root root         22 Jun 10 14:54 libMIOpen.so.1 -> libMIOpen.so.1.0.60102
-rwxr-xr-x  1 root root 1120623856 May 28 21:10 libMIOpen.so.1.0.60102
lrwxrwxrwx  1 root root         14 Jun 10 14:52 libOpenCL.so -> libOpenCL.so.1
lrwxrwxrwx  1 root root         16 Jun 10 14:52 libOpenCL.so.1 -> libOpenCL.so.1.2
-rwxr-xr-x  1 root root      36968 May 28 19:13 libOpenCL.so.1.2
lrwxrwxrwx  1 root root         17 Jun 10 14:51 libamd_comgr.so -> libamd_comgr.so.2
lrwxrwxrwx  1 root root         25 Jun 10 14:51 libamd_comgr.so.2 -> libamd_comgr.so.2.7.60102
-rwxr-xr-x  1 root root  149865832 May 28 18:50 libamd_comgr.so.2.7.60102
lrwxrwxrwx  1 root root         16 Jun 10 14:56 libamd_smi.so -> libamd_smi.so.24
lrwxrwxrwx  1 root root         24 Jun 10 14:56 libamd_smi.so.24 -> libamd_smi.so.24.5.60102
-rwxr-xr-x  1 root root    1664088 May 28 18:11 libamd_smi.so.24.5.60102
lrwxrwxrwx  1 root root         16 Jun 10 14:53 libamdhip64.so -> libamdhip64.so.6
-rwxr-xr-x  1 root root   23232120 Oct  6  2023 libamdhip64.so.5
lrwxrwxrwx  1 root root         24 Jun 10 14:53 libamdhip64.so.6 -> libamdhip64.so.6.1.60102
-rwxr-xr-x  1 root root   27904952 May 28 19:13 libamdhip64.so.6.1.60102
-rwxr-xr-x  1 root root    1531472 May 28 19:14 libamdocl64.so
-rw-r--r--  1 root root  295230498 May 28 19:46 libdevice_contraction_operations.a
-rw-r--r--  1 root root 1367985582 May 28 19:46 libdevice_conv_operations.a
-rw-r--r--  1 root root 1908049166 May 28 19:46 libdevice_gemm_operations.a
-rw-r--r--  1 root root  376043220 May 28 19:46 libdevice_other_operations.a
-rw-r--r--  1 root root 1074021278 May 28 19:47 libdevice_reduction_operations.a
lrwxrwxrwx  1 root root         15 Jun 10 14:56 libhipblas.so -> libhipblas.so.2
lrwxrwxrwx  1 root root         23 Jun 10 14:54 libhipblas.so.2 -> libhipblas.so.2.1.60102
-rwxr-xr-x  1 root root    1073688 May 28 21:09 libhipblas.so.2.1.60102
lrwxrwxrwx  1 root root         17 Jun 10 14:56 libhipblaslt.so -> libhipblaslt.so.0
lrwxrwxrwx  1 root root         25 Jun 10 14:54 libhipblaslt.so.0 -> libhipblaslt.so.0.7.60102
-rwxr-xr-x  1 root root    9946264 May 28 22:46 libhipblaslt.so.0.7.60102
lrwxrwxrwx  1 root root         14 Jun 10 14:52 libhipfft.so -> libhipfft.so.0
lrwxrwxrwx  1 root root         22 Jun 10 14:52 libhipfft.so.0 -> libhipfft.so.0.1.60102
-rwxr-xr-x  1 root root      61832 May 28 21:51 libhipfft.so.0.1.60102
lrwxrwxrwx  1 root root         15 Jun 10 14:52 libhiprand.so -> libhiprand.so.1
lrwxrwxrwx  1 root root         23 Jun 10 14:52 libhiprand.so.1 -> libhiprand.so.1.1.60102
-rwxr-xr-x  1 root root      18440 May 28 19:51 libhiprand.so.1.1.60102
lrwxrwxrwx  1 root root         23 Jun 10 14:53 libhiprtc-builtins.so -> libhiprtc-builtins.so.6
lrwxrwxrwx  1 root root         31 Jun 10 14:53 libhiprtc-builtins.so.6 -> libhiprtc-builtins.so.6.1.60102
-rwxr-xr-x  1 root root     559736 May 28 19:13 libhiprtc-builtins.so.6.1.60102
lrwxrwxrwx  1 root root         14 Jun 10 14:53 libhiprtc.so -> libhiprtc.so.6
lrwxrwxrwx  1 root root         22 Jun 10 14:53 libhiprtc.so.6 -> libhiprtc.so.6.1.60102
-rwxr-xr-x  1 root root     918128 May 28 19:13 libhiprtc.so.6.1.60102
lrwxrwxrwx  1 root root         17 Jun 10 14:56 libhipsolver.so -> libhipsolver.so.0
lrwxrwxrwx  1 root root         25 Jun 10 14:56 libhipsolver.so.0 -> libhipsolver.so.0.1.60102
-rwxr-xr-x  1 root root     307056 May 28 21:09 libhipsolver.so.0.1.60102
lrwxrwxrwx  1 root root         17 Jun 10 14:56 libhipsparse.so -> libhipsparse.so.1
lrwxrwxrwx  1 root root         27 Jun 10 14:54 libhipsparse.so.1 -> libhipsparse.so.1.0.0.60102
-rwxr-xr-x  1 root root     312176 May 28 20:01 libhipsparse.so.1.0.0.60102
lrwxrwxrwx  1 root root         19 Jun 10 14:56 libhipsparselt.so -> libhipsparselt.so.0
lrwxrwxrwx  1 root root         27 Jun 10 14:54 libhipsparselt.so.0 -> libhipsparselt.so.0.2.60102
-rwxr-xr-x  1 root root    8293856 May 28 20:08 libhipsparselt.so.0.2.60102
lrwxrwxrwx  1 root root         17 Jun 10 14:52 libhiptensor.so -> libhiptensor.so.0
lrwxrwxrwx  1 root root         25 Jun 10 14:52 libhiptensor.so.0 -> libhiptensor.so.0.1.60102
-rwxr-xr-x  1 root root  334544520 May 28 20:09 libhiptensor.so.0.1.60102
lrwxrwxrwx  1 root root         28 Jun 10 14:52 libhsa-amd-aqlprofile64.so -> libhsa-amd-aqlprofile64.so.1
lrwxrwxrwx  1 root root         36 Jun 10 14:52 libhsa-amd-aqlprofile64.so.1 -> libhsa-amd-aqlprofile64.so.1.0.60102
-rwxr-xr-x  1 root root     583088 May 28 18:50 libhsa-amd-aqlprofile64.so.1.0.60102
lrwxrwxrwx  1 root root         21 Jun 10 14:52 libhsa-runtime64.so -> libhsa-runtime64.so.1
lrwxrwxrwx  1 root root         30 Jun 10 14:52 libhsa-runtime64.so.1 -> libhsa-runtime64.so.1.13.60102
-rwxr-xr-x  1 root root    3089448 May 28 18:50 libhsa-runtime64.so.1.13.60102
-rwxr-xr-x  1 root root     275208 May 28 18:10 libhsakmt.a
lrwxrwxrwx  1 root root         11 Jun 10 14:52 liboam.so -> liboam.so.1
lrwxrwxrwx  1 root root         19 Jun 10 14:52 liboam.so.1 -> liboam.so.1.0.60102
-rwxr-xr-x  1 root root    1383936 May 28 18:11 liboam.so.1.0.60102
lrwxrwxrwx  1 root root         12 Jun 10 14:55 librccl.so -> librccl.so.1
lrwxrwxrwx  1 root root         20 Jun 10 14:55 librccl.so.1 -> librccl.so.1.0.60102
-rwxr-xr-x  1 root root  231504192 May 28 19:52 librccl.so.1.0.60102
lrwxrwxrwx  1 root root         18 Jun 10 14:55 librocalution.so -> librocalution.so.1
lrwxrwxrwx  1 root root         26 Jun 10 14:54 librocalution.so.1 -> librocalution.so.1.0.60102
-rwxr-xr-x  1 root root   11570680 May 28 21:07 librocalution.so.1.0.60102
lrwxrwxrwx  1 root root         22 Jun 10 14:55 librocalution_hip.so -> librocalution_hip.so.1
lrwxrwxrwx  1 root root         32 Jun 10 14:54 librocalution_hip.so.1 -> librocalution_hip.so.1.0.0.60102
-rwxr-xr-x  1 root root   75440704 May 28 21:07 librocalution_hip.so.1.0.0.60102
lrwxrwxrwx  1 root root         15 Jun 10 14:56 librocblas.so -> librocblas.so.4
lrwxrwxrwx  1 root root         23 Jun 10 14:53 librocblas.so.4 -> librocblas.so.4.1.60102
-rwxr-xr-x  1 root root  881735968 May 28 21:00 librocblas.so.4.1.60102
lrwxrwxrwx  1 root root         14 Jun 10 14:52 librocfft.so -> librocfft.so.0
lrwxrwxrwx  1 root root         22 Jun 10 14:51 librocfft.so.0 -> librocfft.so.0.1.60102
-rwxr-xr-x  1 root root   10736160 May 28 19:55 librocfft.so.0.1.60102
lrwxrwxrwx  1 root root         17 Jun 10 14:51 librocm-core.so -> librocm-core.so.1
lrwxrwxrwx  1 root root         25 Jun 10 14:51 librocm-core.so.1 -> librocm-core.so.1.0.60102
-rwxr-xr-x  1 root root      17000 May 28 19:21 librocm-core.so.1.0.60102
lrwxrwxrwx  1 root root         19 Jun 10 14:51 librocm-dbgapi.so -> librocm-dbgapi.so.0
lrwxrwxrwx  1 root root         24 Jun 10 14:51 librocm-dbgapi.so.0 -> librocm-dbgapi.so.0.71.0
-rwxr-xr-x  1 root root    1185632 May 28 18:51 librocm-dbgapi.so.0.71.0
lrwxrwxrwx  1 root root         28 Jun 10 14:52 librocm-debug-agent.so.2 -> librocm-debug-agent.so.2.0.3
-rwxr-xr-x  1 root root     151368 May 28 19:13 librocm-debug-agent.so.2.0.3
lrwxrwxrwx  1 root root         18 Jun 10 14:52 librocm_smi64.so -> librocm_smi64.so.7
lrwxrwxrwx  1 root root         26 Jun 10 14:52 librocm_smi64.so.7 -> librocm_smi64.so.7.2.60102
-rwxr-xr-x  1 root root    1363256 May 28 18:11 librocm_smi64.so.7.2.60102
lrwxrwxrwx  1 root root         28 Jun 10 14:52 librocprofiler-register.so -> librocprofiler-register.so.0
lrwxrwxrwx  1 root root         32 Jun 10 14:52 librocprofiler-register.so.0 -> librocprofiler-register.so.0.3.0
-rwxr-xr-x  1 root root    1049792 May 28 18:11 librocprofiler-register.so.0.3.0
lrwxrwxrwx  1 root root         21 Jun 10 14:56 librocprofiler64.so -> librocprofiler64.so.1
lrwxrwxrwx  1 root root         29 Jun 10 14:56 librocprofiler64.so.1 -> librocprofiler64.so.1.0.60102
-rwxr-xr-x  1 root root     334944 May 28 19:17 librocprofiler64.so.1.0.60102
lrwxrwxrwx  1 root root         29 Jun 10 14:56 librocprofiler64.so.2 -> librocprofiler64.so.2.0.60102
-rwxr-xr-x  1 root root    1129944 May 28 19:17 librocprofiler64.so.2.0.60102
lrwxrwxrwx  1 root root         21 Jun 10 14:56 librocprofiler64v2.so -> librocprofiler64.so.2
lrwxrwxrwx  1 root root         15 Jun 10 14:52 librocrand.so -> librocrand.so.1
lrwxrwxrwx  1 root root         23 Jun 10 14:52 librocrand.so.1 -> librocrand.so.1.1.60102
-rwxr-xr-x  1 root root   62393304 May 28 19:45 librocrand.so.1.1.60102
lrwxrwxrwx  1 root root         17 Jun 10 14:56 librocsolver.so -> librocsolver.so.0
lrwxrwxrwx  1 root root         25 Jun 10 14:54 librocsolver.so.0 -> librocsolver.so.0.1.60102
-rwxr-xr-x  1 root root 1384333400 May 28 21:05 librocsolver.so.0.1.60102
lrwxrwxrwx  1 root root         17 Jun 10 14:56 librocsparse.so -> librocsparse.so.1
lrwxrwxrwx  1 root root         25 Jun 10 14:54 librocsparse.so.1 -> librocsparse.so.1.0.60102
-rwxr-xr-x  1 root root 1367043336 May 28 19:52 librocsparse.so.1.0.60102
lrwxrwxrwx  1 root root         19 Jun 10 14:52 libroctracer64.so -> libroctracer64.so.4
lrwxrwxrwx  1 root root         27 Jun 10 14:52 libroctracer64.so.4 -> libroctracer64.so.4.1.60102
-rwxr-xr-x  1 root root     321512 May 28 19:14 libroctracer64.so.4.1.60102
lrwxrwxrwx  1 root root         15 Jun 10 14:52 libroctx64.so -> libroctx64.so.4
lrwxrwxrwx  1 root root         23 Jun 10 14:52 libroctx64.so.4 -> libroctx64.so.4.1.60102
-rwxr-xr-x  1 root root      15360 May 28 19:14 libroctx64.so.4.1.60102
-rw-r--r--  1 root root      38898 May 28 19:46 libutility.a
drwxr-xr-x  7 root root       4096 Jun 10 14:53 llvm
drwxr-xr-x  2 root root       4096 Jun 10 14:53 pkgconfig
drwxr-xr-x  3 root root       4096 Jun 10 14:53 rocblas
drwxr-xr-x  3 root root       4096 Jun 10 14:52 rocfft
-rw-r--r--  1 root root        512 May 28 19:21 rocmmod
drwxr-xr-x  2 root root       4096 Jun 10 14:56 rocprofiler
drwxr-xr-x  2 root root       4096 Jun 10 14:52 roctracer
[genai1~]$ docker exec 92ebd80dcef1 /opt/rocm/bin/rocminfo
ROCk module is loaded
=====================    
HSA System Attributes    
=====================    
Runtime Version:         1.13
Runtime Ext Version:     1.4
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:  18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE                              
System Endianness:       LITTLE                             
Mwaitx:                  DISABLED
DMAbuf Support:          NO

==========               
HSA Agents               
==========               
*******                  
Agent 1                  
*******                  
  Name:                    Intel(R) Xeon(R) Gold 5416S        
  Uuid:                    CPU-XX                             
  Marketing Name:          Intel(R) Xeon(R) Gold 5416S        
  Vendor Name:             CPU                                
  Feature:                 None specified                     
  Profile:                 FULL_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        0(0x0)                             
  Queue Min Size:          0(0x0)                             
  Queue Max Size:          0(0x0)                             
  Queue Type:              MULTI                              
  Node:                    0                                  
  Device Type:             CPU                                
  Cache Info:              
    L1:                      49152(0xc000) KB                   
  Chip ID:                 0(0x0)                             
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   4000                               
  BDFID:                   0                                  
  Internal Node ID:        0                                  
  Compute Unit:            32                                 
  SIMDs per CU:            0                                  
  Shader Engines:          0                                  
  Shader Arrs. per Eng.:   0                                  
  WatchPts on Addr. Ranges:1                                  
  Features:                None
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    130981244(0x7ce9d7c) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    130981244(0x7ce9d7c) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    130981244(0x7ce9d7c) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
  ISA Info:                
*******                  
Agent 2                  
*******                  
  Name:                    Intel(R) Xeon(R) Gold 5416S        
  Uuid:                    CPU-XX                             
  Marketing Name:          Intel(R) Xeon(R) Gold 5416S        
  Vendor Name:             CPU                                
  Feature:                 None specified                     
  Profile:                 FULL_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        0(0x0)                             
  Queue Min Size:          0(0x0)                             
  Queue Max Size:          0(0x0)                             
  Queue Type:              MULTI                              
  Node:                    1                                  
  Device Type:             CPU                                
  Cache Info:              
    L1:                      49152(0xc000) KB                   
  Chip ID:                 0(0x0)                             
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   4000                               
  BDFID:                   0                                  
  Internal Node ID:        1                                  
  Compute Unit:            32                                 
  SIMDs per CU:            0                                  
  Shader Engines:          0                                  
  Shader Arrs. per Eng.:   0                                  
  WatchPts on Addr. Ranges:1                                  
  Features:                None
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: FINE GRAINED        
      Size:                    132109384(0x7dfd448) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    132109384(0x7dfd448) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    132109384(0x7dfd448) KB            
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
  ISA Info:                
*******                  
Agent 3                  
*******                  
  Name:                    gfx90a                             
  Uuid:                    GPU-12f7490bc7b8aaf0               
  Marketing Name:          AMD Instinct MI210                 
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    2                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      16(0x10) KB                        
    L2:                      8192(0x2000) KB                    
  Chip ID:                 29711(0x740f)                      
  ASIC Revision:           1(0x1)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   1700                               
  BDFID:                   19456                              
  Internal Node ID:        2                                  
  Compute Unit:            104                                
  SIMDs per CU:            4                                  
  Shader Engines:          8                                  
  Shader Arrs. per Eng.:   1                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    FALSE                              
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          64(0x40)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    2048(0x800)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        4294967295(0xffffffff)             
    y                        4294967295(0xffffffff)             
    z                        4294967295(0xffffffff)             
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 82                                 
  SDMA engine uCode::      8                                  
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
*******                  
Agent 4                  
*******                  
  Name:                    gfx90a                             
  Uuid:                    GPU-6c2be151a7afe1e5               
  Marketing Name:          AMD Instinct MI210                 
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    3                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      16(0x10) KB                        
    L2:                      8192(0x2000) KB                    
  Chip ID:                 29711(0x740f)                      
  ASIC Revision:           1(0x1)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   1700                               
  BDFID:                   25344                              
  Internal Node ID:        3                                  
  Compute Unit:            104                                
  SIMDs per CU:            4                                  
  Shader Engines:          8                                  
  Shader Arrs. per Eng.:   1                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    FALSE                              
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          64(0x40)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    2048(0x800)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        4294967295(0xffffffff)             
    y                        4294967295(0xffffffff)             
    z                        4294967295(0xffffffff)             
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 82                                 
  SDMA engine uCode::      8                                  
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
*******                  
Agent 5                  
*******                  
  Name:                    gfx90a                             
  Uuid:                    GPU-692c10cf88a75cd5               
  Marketing Name:          AMD Instinct MI210                 
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    4                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      16(0x10) KB                        
    L2:                      8192(0x2000) KB                    
  Chip ID:                 29711(0x740f)                      
  ASIC Revision:           1(0x1)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   1700                               
  BDFID:                   52224                              
  Internal Node ID:        4                                  
  Compute Unit:            104                                
  SIMDs per CU:            4                                  
  Shader Engines:          8                                  
  Shader Arrs. per Eng.:   1                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    FALSE                              
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          64(0x40)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    2048(0x800)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        4294967295(0xffffffff)             
    y                        4294967295(0xffffffff)             
    z                        4294967295(0xffffffff)             
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 82                                 
  SDMA engine uCode::      8                                  
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
*******                  
Agent 6                  
*******                  
  Name:                    gfx90a                             
  Uuid:                    GPU-9d1a65bc110f6055               
  Marketing Name:          AMD Instinct MI210                 
  Vendor Name:             AMD                                
  Feature:                 KERNEL_DISPATCH                    
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        128(0x80)                          
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          131072(0x20000)                    
  Queue Type:              MULTI                              
  Node:                    5                                  
  Device Type:             GPU                                
  Cache Info:              
    L1:                      16(0x10) KB                        
    L2:                      8192(0x2000) KB                    
  Chip ID:                 29711(0x740f)                      
  ASIC Revision:           1(0x1)                             
  Cacheline Size:          64(0x40)                           
  Max Clock Freq. (MHz):   1700                               
  BDFID:                   58112                              
  Internal Node ID:        5                                  
  Compute Unit:            104                                
  SIMDs per CU:            4                                  
  Shader Engines:          8                                  
  Shader Arrs. per Eng.:   1                                  
  WatchPts on Addr. Ranges:4                                  
  Coherent Host Access:    FALSE                              
  Features:                KERNEL_DISPATCH 
  Fast F16 Operation:      TRUE                               
  Wavefront Size:          64(0x40)                           
  Workgroup Max Size:      1024(0x400)                        
  Workgroup Max Size per Dimension:
    x                        1024(0x400)                        
    y                        1024(0x400)                        
    z                        1024(0x400)                        
  Max Waves Per CU:        32(0x20)                           
  Max Work-item Per CU:    2048(0x800)                        
  Grid Max Size:           4294967295(0xffffffff)             
  Grid Max Size per Dimension:
    x                        4294967295(0xffffffff)             
    y                        4294967295(0xffffffff)             
    z                        4294967295(0xffffffff)             
  Max fbarriers/Workgrp:   32                                 
  Packet Processor uCode:: 82                                 
  SDMA engine uCode::      8                                  
  IOMMU Support::          None                               
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: EXTENDED FINE GRAINED
      Size:                    67092480(0x3ffc000) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:2048KB                             
      Alloc Alignment:         4KB                                
      Accessible by all:       FALSE                              
    Pool 3                   
      Segment:                 GROUP                              
      Size:                    64(0x40) KB                        
      Allocatable:             FALSE                              
      Alloc Granule:           0KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         0KB                                
      Accessible by all:       FALSE                              
  ISA Info:                
    ISA 1                    
      Name:                    amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-
      Machine Models:          HSA_MACHINE_MODEL_LARGE            
      Profiles:                HSA_PROFILE_BASE                   
      Default Rounding Mode:   NEAR                               
      Default Rounding Mode:   NEAR                               
      Fast f16:                TRUE                               
      Workgroup Max Size:      1024(0x400)                        
      Workgroup Max Size per Dimension:
        x                        1024(0x400)                        
        y                        1024(0x400)                        
        z                        1024(0x400)                        
      Grid Max Size:           4294967295(0xffffffff)             
      Grid Max Size per Dimension:
        x                        4294967295(0xffffffff)             
        y                        4294967295(0xffffffff)             
        z                        4294967295(0xffffffff)             
      FBarrier Max Size:       32                                 
*** Done ***

OS

Linux

GPU

AMD

CPU

No response

Ollama version

0.3.4

Originally created by @aimanyounises1 on GitHub (Aug 10, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6304 Originally assigned to: @dhiltgen on GitHub. The latest version of Ollama (0.3.4) is not detecting or utilizing AMD GPUs (Instinct MI210) when run in a Docker container. I tried to run it with docker because I didn't want to update the Ollama working version for me 0.2.8. I followed the instruction here: https://hub.docker.com/r/ollama/ollama. command ```docker run -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocm``` Please see below the error (I tried as well before to update the Ollama with the latest version on my server but encountered the same issue) Any insights or suggestions would be greatly appreciated! ``` [genai1~]$ docker exec 92ebd80dcef1 /opt/rocm/bin/rocm-smi ========================================== ROCm System Management Interface ========================================== ==================================================== Concise Info ==================================================== Device Node IDs Temp Power Partitions SCLK MCLK Fan Perf PwrCap VRAM% GPU% (DID, GUID) (Edge) (Avg) (Mem, Compute, ID) ====================================================================================================================== Traceback (most recent call last): File "/opt/rocm/bin/rocm-smi", line 4074, in <module> showAllConcise(deviceList) File "/opt/rocm/bin/rocm-smi", line 2033, in showAllConcise zip(range(len(max_widths)), values['card%s' % (str(device))])), None) File "/opt/rocm/bin/rocm-smi", line 872, in printLog print(logstr + '\n', end='') UnicodeEncodeError: 'ascii' codec can't encode character '\xb0' in position 35: ordinal not in range(128) ``` ``` [genai1~]$ docker exec 92ebd80dcef1 ls -la /root/.ollama total 20 drwxr-xr-x 3 root root 4096 Aug 10 19:17 . dr-xr-x--- 1 root root 4096 Aug 10 19:46 .. -rw------- 1 root root 387 Aug 10 19:17 id_ed25519 -rw-r--r-- 1 root root 81 Aug 10 19:17 id_ed25519.pub drwxr-xr-x 4 root root 4096 Aug 10 19:55 models [genai1~]$ docker logs 92ebd80dcef1 2docker logs 92ebd80dcef1 2024/08/10 19:46:51 routes.go:1108: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR: OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]" time=2024-08-10T19:46:51.654Z level=INFO source=images.go:781 msg="total blobs: 0" time=2024-08-10T19:46:51.654Z level=INFO source=images.go:788 msg="total unused blobs removed: 0" time=2024-08-10T19:46:51.654Z level=INFO source=routes.go:1155 msg="Listening on [::]:11434 (version 0.3.4)" time=2024-08-10T19:46:51.655Z level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama2612956243/runners time=2024-08-10T19:46:54.488Z level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cuda_v11 rocm_v60102 cpu cpu_avx cpu_avx2]" time=2024-08-10T19:46:54.488Z level=INFO source=gpu.go:204 msg="looking for compatible GPUs" time=2024-08-10T19:46:54.491Z level=WARN source=amd_linux.go:59 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" time=2024-08-10T19:46:54.491Z level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=0 total="0 B" time=2024-08-10T19:46:54.492Z level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=1 total="0 B" time=2024-08-10T19:46:54.492Z level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=2 total="0 B" time=2024-08-10T19:46:54.492Z level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=3 total="0 B" time=2024-08-10T19:46:54.492Z level=INFO source=amd_linux.go:360 msg="no compatible amdgpu devices detected" time=2024-08-10T19:46:54.492Z level=INFO source=gpu.go:347 msg="no compatible GPUs were discovered" time=2024-08-10T19:46:54.492Z level=INFO source=types.go:105 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="250.9 GiB" available="230.3 GiB" Error: something went wrong, please see the ollama server logs for details [genai1~]$ docker exec 92ebd80dcef1 env | grep OLLAMA OLLAMA_HOST=0.0.0.0 [genai1~]$ docker exec 92ebd80dcef1 ls -l /opt/rocm/bin total 78928 -rwxr-xr-x 1 root root 3993744 May 28 21:10 MIOpenDriver lrwxrwxrwx 1 root root 35 Jun 10 14:56 amd-smi -> ../libexec/amdsmi_cli/amdsmi_cli.py lrwxrwxrwx 1 root root 24 Jun 10 14:51 amdclang -> ../lib/llvm/bin/amdclang lrwxrwxrwx 1 root root 26 Jun 10 14:51 amdclang++ -> ../lib/llvm/bin/amdclang++ lrwxrwxrwx 1 root root 27 Jun 10 14:51 amdclang-cl -> ../lib/llvm/bin/amdclang-cl lrwxrwxrwx 1 root root 28 Jun 10 14:51 amdclang-cpp -> ../lib/llvm/bin/amdclang-cpp lrwxrwxrwx 1 root root 24 Jun 10 14:51 amdflang -> ../lib/llvm/bin/amdflang lrwxrwxrwx 1 root root 22 Jun 10 14:51 amdlld -> ../lib/llvm/bin/amdlld lrwxrwxrwx 1 root root 22 Jun 10 14:53 aompcc -> ../lib/llvm/bin/aompcc -rwxr-xr-x 1 root root 107592 May 28 19:14 clinfo -rwxrwxr-x 1 root root 2551 May 28 18:08 hipcc -rwxr-xr-x 1 root root 575472 May 28 18:11 hipcc.bin -rwxrwxr-x 1 root root 21889 May 28 18:08 hipcc.pl -rwxrwxr-x 1 root root 1508 May 28 18:08 hipcc_cmake_linker_helper -rwxrwxr-x 1 root root 2453 May 28 18:08 hipconfig -rwxr-xr-x 1 root root 575480 May 28 18:11 hipconfig.bin -rwxrwxr-x 1 root root 7539 May 28 18:08 hipconfig.pl -rwxr-xr-x 1 root root 784 May 28 18:08 hipconvertinplace-perl.sh -rwxr-xr-x 1 root root 674 May 28 18:08 hipconvertinplace.sh -rwxrwxr-x 1 root root 1857 May 28 18:08 hipdemangleatp -rwxr-xr-x 1 root root 407 May 28 18:08 hipexamine-perl.sh -rwxr-xr-x 1 root root 557 May 28 18:08 hipexamine.sh -rwxr-xr-x 1 root root 62142024 May 28 19:14 hipify-clang -rwxr-xr-x 1 root root 659191 May 28 18:08 hipify-perl -rw-rw-r-- 1 root root 6486 May 28 18:08 hipvars.pm -rwxr-xr-x 1 root root 1332 May 28 19:30 install_precompiled_kernels.sh lrwxrwxrwx 1 root root 21 Jun 10 14:53 mygpu -> ../lib/llvm/bin/mygpu lrwxrwxrwx 1 root root 22 Jun 10 14:53 mymcpu -> ../lib/llvm/bin/mymcpu lrwxrwxrwx 1 root root 28 Jun 10 14:51 offload-arch -> ../lib/llvm/bin/offload-arch -rwxrwxr-x 1 root root 10047 May 28 18:08 roc-obj -rwxrwxr-x 1 root root 8457 May 28 18:08 roc-obj-extract -rwxrwxr-x 1 root root 71 May 28 18:08 roc-obj-extract.bat -rwxrwxr-x 1 root root 6973 May 28 18:08 roc-obj-ls -rwxrwxr-x 1 root root 56 May 28 18:08 roc-obj-ls.bat -rwxr-xr-x 1 root root 12314744 May 28 18:57 rocgdb lrwxrwxrwx 1 root root 31 Jun 10 14:52 rocm-smi -> ../libexec/rocm_smi/rocm_smi.py -rwxr-xr-x 1 root root 10057 May 28 18:50 rocm_agent_enumerator -rwxr-xr-x 1 root root 159560 May 28 18:50 rocminfo -r-xr-xr-x 1 root root 22019 May 28 18:08 rocprof -r-xr-xr-x 1 root root 11523 May 28 18:08 rocprofv2 -rwxr-xr-x 1 root root 96000 May 28 19:17 rocsys -rwxr-xr-x 1 root root 9224 May 28 19:30 test_perf.py [genai1@~]$ docker exec 92ebd80dcef1 ls -l /opt/rocm/lib total 10486408 drwxr-xr-x 32 root root 4096 Jun 10 14:56 cmake drwxr-xr-x 3 root root 4096 Jun 10 14:55 hipblaslt drwxr-xr-x 3 root root 4096 Jun 10 14:54 hipsparselt drwxr-xr-x 2 root root 4096 Jun 10 14:52 hsa-amd-aqlprofile lrwxrwxrwx 1 root root 14 Jun 10 14:55 libMIOpen.so -> libMIOpen.so.1 lrwxrwxrwx 1 root root 22 Jun 10 14:54 libMIOpen.so.1 -> libMIOpen.so.1.0.60102 -rwxr-xr-x 1 root root 1120623856 May 28 21:10 libMIOpen.so.1.0.60102 lrwxrwxrwx 1 root root 14 Jun 10 14:52 libOpenCL.so -> libOpenCL.so.1 lrwxrwxrwx 1 root root 16 Jun 10 14:52 libOpenCL.so.1 -> libOpenCL.so.1.2 -rwxr-xr-x 1 root root 36968 May 28 19:13 libOpenCL.so.1.2 lrwxrwxrwx 1 root root 17 Jun 10 14:51 libamd_comgr.so -> libamd_comgr.so.2 lrwxrwxrwx 1 root root 25 Jun 10 14:51 libamd_comgr.so.2 -> libamd_comgr.so.2.7.60102 -rwxr-xr-x 1 root root 149865832 May 28 18:50 libamd_comgr.so.2.7.60102 lrwxrwxrwx 1 root root 16 Jun 10 14:56 libamd_smi.so -> libamd_smi.so.24 lrwxrwxrwx 1 root root 24 Jun 10 14:56 libamd_smi.so.24 -> libamd_smi.so.24.5.60102 -rwxr-xr-x 1 root root 1664088 May 28 18:11 libamd_smi.so.24.5.60102 lrwxrwxrwx 1 root root 16 Jun 10 14:53 libamdhip64.so -> libamdhip64.so.6 -rwxr-xr-x 1 root root 23232120 Oct 6 2023 libamdhip64.so.5 lrwxrwxrwx 1 root root 24 Jun 10 14:53 libamdhip64.so.6 -> libamdhip64.so.6.1.60102 -rwxr-xr-x 1 root root 27904952 May 28 19:13 libamdhip64.so.6.1.60102 -rwxr-xr-x 1 root root 1531472 May 28 19:14 libamdocl64.so -rw-r--r-- 1 root root 295230498 May 28 19:46 libdevice_contraction_operations.a -rw-r--r-- 1 root root 1367985582 May 28 19:46 libdevice_conv_operations.a -rw-r--r-- 1 root root 1908049166 May 28 19:46 libdevice_gemm_operations.a -rw-r--r-- 1 root root 376043220 May 28 19:46 libdevice_other_operations.a -rw-r--r-- 1 root root 1074021278 May 28 19:47 libdevice_reduction_operations.a lrwxrwxrwx 1 root root 15 Jun 10 14:56 libhipblas.so -> libhipblas.so.2 lrwxrwxrwx 1 root root 23 Jun 10 14:54 libhipblas.so.2 -> libhipblas.so.2.1.60102 -rwxr-xr-x 1 root root 1073688 May 28 21:09 libhipblas.so.2.1.60102 lrwxrwxrwx 1 root root 17 Jun 10 14:56 libhipblaslt.so -> libhipblaslt.so.0 lrwxrwxrwx 1 root root 25 Jun 10 14:54 libhipblaslt.so.0 -> libhipblaslt.so.0.7.60102 -rwxr-xr-x 1 root root 9946264 May 28 22:46 libhipblaslt.so.0.7.60102 lrwxrwxrwx 1 root root 14 Jun 10 14:52 libhipfft.so -> libhipfft.so.0 lrwxrwxrwx 1 root root 22 Jun 10 14:52 libhipfft.so.0 -> libhipfft.so.0.1.60102 -rwxr-xr-x 1 root root 61832 May 28 21:51 libhipfft.so.0.1.60102 lrwxrwxrwx 1 root root 15 Jun 10 14:52 libhiprand.so -> libhiprand.so.1 lrwxrwxrwx 1 root root 23 Jun 10 14:52 libhiprand.so.1 -> libhiprand.so.1.1.60102 -rwxr-xr-x 1 root root 18440 May 28 19:51 libhiprand.so.1.1.60102 lrwxrwxrwx 1 root root 23 Jun 10 14:53 libhiprtc-builtins.so -> libhiprtc-builtins.so.6 lrwxrwxrwx 1 root root 31 Jun 10 14:53 libhiprtc-builtins.so.6 -> libhiprtc-builtins.so.6.1.60102 -rwxr-xr-x 1 root root 559736 May 28 19:13 libhiprtc-builtins.so.6.1.60102 lrwxrwxrwx 1 root root 14 Jun 10 14:53 libhiprtc.so -> libhiprtc.so.6 lrwxrwxrwx 1 root root 22 Jun 10 14:53 libhiprtc.so.6 -> libhiprtc.so.6.1.60102 -rwxr-xr-x 1 root root 918128 May 28 19:13 libhiprtc.so.6.1.60102 lrwxrwxrwx 1 root root 17 Jun 10 14:56 libhipsolver.so -> libhipsolver.so.0 lrwxrwxrwx 1 root root 25 Jun 10 14:56 libhipsolver.so.0 -> libhipsolver.so.0.1.60102 -rwxr-xr-x 1 root root 307056 May 28 21:09 libhipsolver.so.0.1.60102 lrwxrwxrwx 1 root root 17 Jun 10 14:56 libhipsparse.so -> libhipsparse.so.1 lrwxrwxrwx 1 root root 27 Jun 10 14:54 libhipsparse.so.1 -> libhipsparse.so.1.0.0.60102 -rwxr-xr-x 1 root root 312176 May 28 20:01 libhipsparse.so.1.0.0.60102 lrwxrwxrwx 1 root root 19 Jun 10 14:56 libhipsparselt.so -> libhipsparselt.so.0 lrwxrwxrwx 1 root root 27 Jun 10 14:54 libhipsparselt.so.0 -> libhipsparselt.so.0.2.60102 -rwxr-xr-x 1 root root 8293856 May 28 20:08 libhipsparselt.so.0.2.60102 lrwxrwxrwx 1 root root 17 Jun 10 14:52 libhiptensor.so -> libhiptensor.so.0 lrwxrwxrwx 1 root root 25 Jun 10 14:52 libhiptensor.so.0 -> libhiptensor.so.0.1.60102 -rwxr-xr-x 1 root root 334544520 May 28 20:09 libhiptensor.so.0.1.60102 lrwxrwxrwx 1 root root 28 Jun 10 14:52 libhsa-amd-aqlprofile64.so -> libhsa-amd-aqlprofile64.so.1 lrwxrwxrwx 1 root root 36 Jun 10 14:52 libhsa-amd-aqlprofile64.so.1 -> libhsa-amd-aqlprofile64.so.1.0.60102 -rwxr-xr-x 1 root root 583088 May 28 18:50 libhsa-amd-aqlprofile64.so.1.0.60102 lrwxrwxrwx 1 root root 21 Jun 10 14:52 libhsa-runtime64.so -> libhsa-runtime64.so.1 lrwxrwxrwx 1 root root 30 Jun 10 14:52 libhsa-runtime64.so.1 -> libhsa-runtime64.so.1.13.60102 -rwxr-xr-x 1 root root 3089448 May 28 18:50 libhsa-runtime64.so.1.13.60102 -rwxr-xr-x 1 root root 275208 May 28 18:10 libhsakmt.a lrwxrwxrwx 1 root root 11 Jun 10 14:52 liboam.so -> liboam.so.1 lrwxrwxrwx 1 root root 19 Jun 10 14:52 liboam.so.1 -> liboam.so.1.0.60102 -rwxr-xr-x 1 root root 1383936 May 28 18:11 liboam.so.1.0.60102 lrwxrwxrwx 1 root root 12 Jun 10 14:55 librccl.so -> librccl.so.1 lrwxrwxrwx 1 root root 20 Jun 10 14:55 librccl.so.1 -> librccl.so.1.0.60102 -rwxr-xr-x 1 root root 231504192 May 28 19:52 librccl.so.1.0.60102 lrwxrwxrwx 1 root root 18 Jun 10 14:55 librocalution.so -> librocalution.so.1 lrwxrwxrwx 1 root root 26 Jun 10 14:54 librocalution.so.1 -> librocalution.so.1.0.60102 -rwxr-xr-x 1 root root 11570680 May 28 21:07 librocalution.so.1.0.60102 lrwxrwxrwx 1 root root 22 Jun 10 14:55 librocalution_hip.so -> librocalution_hip.so.1 lrwxrwxrwx 1 root root 32 Jun 10 14:54 librocalution_hip.so.1 -> librocalution_hip.so.1.0.0.60102 -rwxr-xr-x 1 root root 75440704 May 28 21:07 librocalution_hip.so.1.0.0.60102 lrwxrwxrwx 1 root root 15 Jun 10 14:56 librocblas.so -> librocblas.so.4 lrwxrwxrwx 1 root root 23 Jun 10 14:53 librocblas.so.4 -> librocblas.so.4.1.60102 -rwxr-xr-x 1 root root 881735968 May 28 21:00 librocblas.so.4.1.60102 lrwxrwxrwx 1 root root 14 Jun 10 14:52 librocfft.so -> librocfft.so.0 lrwxrwxrwx 1 root root 22 Jun 10 14:51 librocfft.so.0 -> librocfft.so.0.1.60102 -rwxr-xr-x 1 root root 10736160 May 28 19:55 librocfft.so.0.1.60102 lrwxrwxrwx 1 root root 17 Jun 10 14:51 librocm-core.so -> librocm-core.so.1 lrwxrwxrwx 1 root root 25 Jun 10 14:51 librocm-core.so.1 -> librocm-core.so.1.0.60102 -rwxr-xr-x 1 root root 17000 May 28 19:21 librocm-core.so.1.0.60102 lrwxrwxrwx 1 root root 19 Jun 10 14:51 librocm-dbgapi.so -> librocm-dbgapi.so.0 lrwxrwxrwx 1 root root 24 Jun 10 14:51 librocm-dbgapi.so.0 -> librocm-dbgapi.so.0.71.0 -rwxr-xr-x 1 root root 1185632 May 28 18:51 librocm-dbgapi.so.0.71.0 lrwxrwxrwx 1 root root 28 Jun 10 14:52 librocm-debug-agent.so.2 -> librocm-debug-agent.so.2.0.3 -rwxr-xr-x 1 root root 151368 May 28 19:13 librocm-debug-agent.so.2.0.3 lrwxrwxrwx 1 root root 18 Jun 10 14:52 librocm_smi64.so -> librocm_smi64.so.7 lrwxrwxrwx 1 root root 26 Jun 10 14:52 librocm_smi64.so.7 -> librocm_smi64.so.7.2.60102 -rwxr-xr-x 1 root root 1363256 May 28 18:11 librocm_smi64.so.7.2.60102 lrwxrwxrwx 1 root root 28 Jun 10 14:52 librocprofiler-register.so -> librocprofiler-register.so.0 lrwxrwxrwx 1 root root 32 Jun 10 14:52 librocprofiler-register.so.0 -> librocprofiler-register.so.0.3.0 -rwxr-xr-x 1 root root 1049792 May 28 18:11 librocprofiler-register.so.0.3.0 lrwxrwxrwx 1 root root 21 Jun 10 14:56 librocprofiler64.so -> librocprofiler64.so.1 lrwxrwxrwx 1 root root 29 Jun 10 14:56 librocprofiler64.so.1 -> librocprofiler64.so.1.0.60102 -rwxr-xr-x 1 root root 334944 May 28 19:17 librocprofiler64.so.1.0.60102 lrwxrwxrwx 1 root root 29 Jun 10 14:56 librocprofiler64.so.2 -> librocprofiler64.so.2.0.60102 -rwxr-xr-x 1 root root 1129944 May 28 19:17 librocprofiler64.so.2.0.60102 lrwxrwxrwx 1 root root 21 Jun 10 14:56 librocprofiler64v2.so -> librocprofiler64.so.2 lrwxrwxrwx 1 root root 15 Jun 10 14:52 librocrand.so -> librocrand.so.1 lrwxrwxrwx 1 root root 23 Jun 10 14:52 librocrand.so.1 -> librocrand.so.1.1.60102 -rwxr-xr-x 1 root root 62393304 May 28 19:45 librocrand.so.1.1.60102 lrwxrwxrwx 1 root root 17 Jun 10 14:56 librocsolver.so -> librocsolver.so.0 lrwxrwxrwx 1 root root 25 Jun 10 14:54 librocsolver.so.0 -> librocsolver.so.0.1.60102 -rwxr-xr-x 1 root root 1384333400 May 28 21:05 librocsolver.so.0.1.60102 lrwxrwxrwx 1 root root 17 Jun 10 14:56 librocsparse.so -> librocsparse.so.1 lrwxrwxrwx 1 root root 25 Jun 10 14:54 librocsparse.so.1 -> librocsparse.so.1.0.60102 -rwxr-xr-x 1 root root 1367043336 May 28 19:52 librocsparse.so.1.0.60102 lrwxrwxrwx 1 root root 19 Jun 10 14:52 libroctracer64.so -> libroctracer64.so.4 lrwxrwxrwx 1 root root 27 Jun 10 14:52 libroctracer64.so.4 -> libroctracer64.so.4.1.60102 -rwxr-xr-x 1 root root 321512 May 28 19:14 libroctracer64.so.4.1.60102 lrwxrwxrwx 1 root root 15 Jun 10 14:52 libroctx64.so -> libroctx64.so.4 lrwxrwxrwx 1 root root 23 Jun 10 14:52 libroctx64.so.4 -> libroctx64.so.4.1.60102 -rwxr-xr-x 1 root root 15360 May 28 19:14 libroctx64.so.4.1.60102 -rw-r--r-- 1 root root 38898 May 28 19:46 libutility.a drwxr-xr-x 7 root root 4096 Jun 10 14:53 llvm drwxr-xr-x 2 root root 4096 Jun 10 14:53 pkgconfig drwxr-xr-x 3 root root 4096 Jun 10 14:53 rocblas drwxr-xr-x 3 root root 4096 Jun 10 14:52 rocfft -rw-r--r-- 1 root root 512 May 28 19:21 rocmmod drwxr-xr-x 2 root root 4096 Jun 10 14:56 rocprofiler drwxr-xr-x 2 root root 4096 Jun 10 14:52 roctracer [genai1~]$ docker exec 92ebd80dcef1 /opt/rocm/bin/rocminfo ROCk module is loaded ===================== HSA System Attributes ===================== Runtime Version: 1.13 Runtime Ext Version: 1.4 System Timestamp Freq.: 1000.000000MHz Sig. Max Wait Duration: 18446744073709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count) Machine Model: LARGE System Endianness: LITTLE Mwaitx: DISABLED DMAbuf Support: NO ========== HSA Agents ========== ******* Agent 1 ******* Name: Intel(R) Xeon(R) Gold 5416S Uuid: CPU-XX Marketing Name: Intel(R) Xeon(R) Gold 5416S Vendor Name: CPU Feature: None specified Profile: FULL_PROFILE Float Round Mode: NEAR Max Queue Number: 0(0x0) Queue Min Size: 0(0x0) Queue Max Size: 0(0x0) Queue Type: MULTI Node: 0 Device Type: CPU Cache Info: L1: 49152(0xc000) KB Chip ID: 0(0x0) ASIC Revision: 0(0x0) Cacheline Size: 64(0x40) Max Clock Freq. (MHz): 4000 BDFID: 0 Internal Node ID: 0 Compute Unit: 32 SIMDs per CU: 0 Shader Engines: 0 Shader Arrs. per Eng.: 0 WatchPts on Addr. Ranges:1 Features: None Pool Info: Pool 1 Segment: GLOBAL; FLAGS: FINE GRAINED Size: 130981244(0x7ce9d7c) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:4KB Alloc Alignment: 4KB Accessible by all: TRUE Pool 2 Segment: GLOBAL; FLAGS: KERNARG, FINE GRAINED Size: 130981244(0x7ce9d7c) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:4KB Alloc Alignment: 4KB Accessible by all: TRUE Pool 3 Segment: GLOBAL; FLAGS: COARSE GRAINED Size: 130981244(0x7ce9d7c) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:4KB Alloc Alignment: 4KB Accessible by all: TRUE ISA Info: ******* Agent 2 ******* Name: Intel(R) Xeon(R) Gold 5416S Uuid: CPU-XX Marketing Name: Intel(R) Xeon(R) Gold 5416S Vendor Name: CPU Feature: None specified Profile: FULL_PROFILE Float Round Mode: NEAR Max Queue Number: 0(0x0) Queue Min Size: 0(0x0) Queue Max Size: 0(0x0) Queue Type: MULTI Node: 1 Device Type: CPU Cache Info: L1: 49152(0xc000) KB Chip ID: 0(0x0) ASIC Revision: 0(0x0) Cacheline Size: 64(0x40) Max Clock Freq. (MHz): 4000 BDFID: 0 Internal Node ID: 1 Compute Unit: 32 SIMDs per CU: 0 Shader Engines: 0 Shader Arrs. per Eng.: 0 WatchPts on Addr. Ranges:1 Features: None Pool Info: Pool 1 Segment: GLOBAL; FLAGS: FINE GRAINED Size: 132109384(0x7dfd448) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:4KB Alloc Alignment: 4KB Accessible by all: TRUE Pool 2 Segment: GLOBAL; FLAGS: KERNARG, FINE GRAINED Size: 132109384(0x7dfd448) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:4KB Alloc Alignment: 4KB Accessible by all: TRUE Pool 3 Segment: GLOBAL; FLAGS: COARSE GRAINED Size: 132109384(0x7dfd448) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:4KB Alloc Alignment: 4KB Accessible by all: TRUE ISA Info: ******* Agent 3 ******* Name: gfx90a Uuid: GPU-12f7490bc7b8aaf0 Marketing Name: AMD Instinct MI210 Vendor Name: AMD Feature: KERNEL_DISPATCH Profile: BASE_PROFILE Float Round Mode: NEAR Max Queue Number: 128(0x80) Queue Min Size: 64(0x40) Queue Max Size: 131072(0x20000) Queue Type: MULTI Node: 2 Device Type: GPU Cache Info: L1: 16(0x10) KB L2: 8192(0x2000) KB Chip ID: 29711(0x740f) ASIC Revision: 1(0x1) Cacheline Size: 64(0x40) Max Clock Freq. (MHz): 1700 BDFID: 19456 Internal Node ID: 2 Compute Unit: 104 SIMDs per CU: 4 Shader Engines: 8 Shader Arrs. per Eng.: 1 WatchPts on Addr. Ranges:4 Coherent Host Access: FALSE Features: KERNEL_DISPATCH Fast F16 Operation: TRUE Wavefront Size: 64(0x40) Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Max Waves Per CU: 32(0x20) Max Work-item Per CU: 2048(0x800) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) Max fbarriers/Workgrp: 32 Packet Processor uCode:: 82 SDMA engine uCode:: 8 IOMMU Support:: None Pool Info: Pool 1 Segment: GLOBAL; FLAGS: COARSE GRAINED Size: 67092480(0x3ffc000) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:2048KB Alloc Alignment: 4KB Accessible by all: FALSE Pool 2 Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED Size: 67092480(0x3ffc000) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:2048KB Alloc Alignment: 4KB Accessible by all: FALSE Pool 3 Segment: GROUP Size: 64(0x40) KB Allocatable: FALSE Alloc Granule: 0KB Alloc Recommended Granule:0KB Alloc Alignment: 0KB Accessible by all: FALSE ISA Info: ISA 1 Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- Machine Models: HSA_MACHINE_MODEL_LARGE Profiles: HSA_PROFILE_BASE Default Rounding Mode: NEAR Default Rounding Mode: NEAR Fast f16: TRUE Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) FBarrier Max Size: 32 ******* Agent 4 ******* Name: gfx90a Uuid: GPU-6c2be151a7afe1e5 Marketing Name: AMD Instinct MI210 Vendor Name: AMD Feature: KERNEL_DISPATCH Profile: BASE_PROFILE Float Round Mode: NEAR Max Queue Number: 128(0x80) Queue Min Size: 64(0x40) Queue Max Size: 131072(0x20000) Queue Type: MULTI Node: 3 Device Type: GPU Cache Info: L1: 16(0x10) KB L2: 8192(0x2000) KB Chip ID: 29711(0x740f) ASIC Revision: 1(0x1) Cacheline Size: 64(0x40) Max Clock Freq. (MHz): 1700 BDFID: 25344 Internal Node ID: 3 Compute Unit: 104 SIMDs per CU: 4 Shader Engines: 8 Shader Arrs. per Eng.: 1 WatchPts on Addr. Ranges:4 Coherent Host Access: FALSE Features: KERNEL_DISPATCH Fast F16 Operation: TRUE Wavefront Size: 64(0x40) Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Max Waves Per CU: 32(0x20) Max Work-item Per CU: 2048(0x800) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) Max fbarriers/Workgrp: 32 Packet Processor uCode:: 82 SDMA engine uCode:: 8 IOMMU Support:: None Pool Info: Pool 1 Segment: GLOBAL; FLAGS: COARSE GRAINED Size: 67092480(0x3ffc000) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:2048KB Alloc Alignment: 4KB Accessible by all: FALSE Pool 2 Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED Size: 67092480(0x3ffc000) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:2048KB Alloc Alignment: 4KB Accessible by all: FALSE Pool 3 Segment: GROUP Size: 64(0x40) KB Allocatable: FALSE Alloc Granule: 0KB Alloc Recommended Granule:0KB Alloc Alignment: 0KB Accessible by all: FALSE ISA Info: ISA 1 Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- Machine Models: HSA_MACHINE_MODEL_LARGE Profiles: HSA_PROFILE_BASE Default Rounding Mode: NEAR Default Rounding Mode: NEAR Fast f16: TRUE Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) FBarrier Max Size: 32 ******* Agent 5 ******* Name: gfx90a Uuid: GPU-692c10cf88a75cd5 Marketing Name: AMD Instinct MI210 Vendor Name: AMD Feature: KERNEL_DISPATCH Profile: BASE_PROFILE Float Round Mode: NEAR Max Queue Number: 128(0x80) Queue Min Size: 64(0x40) Queue Max Size: 131072(0x20000) Queue Type: MULTI Node: 4 Device Type: GPU Cache Info: L1: 16(0x10) KB L2: 8192(0x2000) KB Chip ID: 29711(0x740f) ASIC Revision: 1(0x1) Cacheline Size: 64(0x40) Max Clock Freq. (MHz): 1700 BDFID: 52224 Internal Node ID: 4 Compute Unit: 104 SIMDs per CU: 4 Shader Engines: 8 Shader Arrs. per Eng.: 1 WatchPts on Addr. Ranges:4 Coherent Host Access: FALSE Features: KERNEL_DISPATCH Fast F16 Operation: TRUE Wavefront Size: 64(0x40) Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Max Waves Per CU: 32(0x20) Max Work-item Per CU: 2048(0x800) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) Max fbarriers/Workgrp: 32 Packet Processor uCode:: 82 SDMA engine uCode:: 8 IOMMU Support:: None Pool Info: Pool 1 Segment: GLOBAL; FLAGS: COARSE GRAINED Size: 67092480(0x3ffc000) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:2048KB Alloc Alignment: 4KB Accessible by all: FALSE Pool 2 Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED Size: 67092480(0x3ffc000) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:2048KB Alloc Alignment: 4KB Accessible by all: FALSE Pool 3 Segment: GROUP Size: 64(0x40) KB Allocatable: FALSE Alloc Granule: 0KB Alloc Recommended Granule:0KB Alloc Alignment: 0KB Accessible by all: FALSE ISA Info: ISA 1 Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- Machine Models: HSA_MACHINE_MODEL_LARGE Profiles: HSA_PROFILE_BASE Default Rounding Mode: NEAR Default Rounding Mode: NEAR Fast f16: TRUE Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) FBarrier Max Size: 32 ******* Agent 6 ******* Name: gfx90a Uuid: GPU-9d1a65bc110f6055 Marketing Name: AMD Instinct MI210 Vendor Name: AMD Feature: KERNEL_DISPATCH Profile: BASE_PROFILE Float Round Mode: NEAR Max Queue Number: 128(0x80) Queue Min Size: 64(0x40) Queue Max Size: 131072(0x20000) Queue Type: MULTI Node: 5 Device Type: GPU Cache Info: L1: 16(0x10) KB L2: 8192(0x2000) KB Chip ID: 29711(0x740f) ASIC Revision: 1(0x1) Cacheline Size: 64(0x40) Max Clock Freq. (MHz): 1700 BDFID: 58112 Internal Node ID: 5 Compute Unit: 104 SIMDs per CU: 4 Shader Engines: 8 Shader Arrs. per Eng.: 1 WatchPts on Addr. Ranges:4 Coherent Host Access: FALSE Features: KERNEL_DISPATCH Fast F16 Operation: TRUE Wavefront Size: 64(0x40) Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Max Waves Per CU: 32(0x20) Max Work-item Per CU: 2048(0x800) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) Max fbarriers/Workgrp: 32 Packet Processor uCode:: 82 SDMA engine uCode:: 8 IOMMU Support:: None Pool Info: Pool 1 Segment: GLOBAL; FLAGS: COARSE GRAINED Size: 67092480(0x3ffc000) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:2048KB Alloc Alignment: 4KB Accessible by all: FALSE Pool 2 Segment: GLOBAL; FLAGS: EXTENDED FINE GRAINED Size: 67092480(0x3ffc000) KB Allocatable: TRUE Alloc Granule: 4KB Alloc Recommended Granule:2048KB Alloc Alignment: 4KB Accessible by all: FALSE Pool 3 Segment: GROUP Size: 64(0x40) KB Allocatable: FALSE Alloc Granule: 0KB Alloc Recommended Granule:0KB Alloc Alignment: 0KB Accessible by all: FALSE ISA Info: ISA 1 Name: amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack- Machine Models: HSA_MACHINE_MODEL_LARGE Profiles: HSA_PROFILE_BASE Default Rounding Mode: NEAR Default Rounding Mode: NEAR Fast f16: TRUE Workgroup Max Size: 1024(0x400) Workgroup Max Size per Dimension: x 1024(0x400) y 1024(0x400) z 1024(0x400) Grid Max Size: 4294967295(0xffffffff) Grid Max Size per Dimension: x 4294967295(0xffffffff) y 4294967295(0xffffffff) z 4294967295(0xffffffff) FBarrier Max Size: 32 *** Done *** ``` ### OS Linux ### GPU AMD ### CPU _No response_ ### Ollama version 0.3.4

GiteaMirror added the linux amd bug labels 2026-04-12 14:49:43 -05:00

GiteaMirror closed this issue

2026-04-12 14:49:43 -05:00

GiteaMirror commented

2026-04-12 14:49:44 -05:00

@GKeppler commented on GitHub (Aug 12, 2024):

I have a similar situation, Mi300As are not usable. Ollama Version is 0.3.5.
Running ./ollama-linux-amd64 serve yields:

2024/08/12 14:55:33 routes.go:1123: INFO server config env="map[CUDA_VISIBLE_DEVICES:0,1 GPU_DEVICE_ORDINAL:0,1 HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/hk-project-test-p0020932/sb7059/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR: OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:0,1]"
time=2024-08-12T14:55:34.541+02:00 level=INFO source=images.go:782 msg="total blobs: 19"
time=2024-08-12T14:55:34.545+02:00 level=INFO source=images.go:790 msg="total unused blobs removed: 0"
time=2024-08-12T14:55:34.545+02:00 level=INFO source=routes.go:1170 msg="Listening on [::]:11434 (version 0.3.5)"
time=2024-08-12T14:55:34.546+02:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/scratch/slurm_tmpdir/job_1139/ollama1144170004/runners
time=2024-08-12T14:55:41.044+02:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11 rocm_v60102]"
time=2024-08-12T14:55:41.044+02:00 level=INFO source=gpu.go:204 msg="looking for compatible GPUs"
time=2024-08-12T14:55:41.053+02:00 level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=0 total="0 B"
time=2024-08-12T14:55:41.053+02:00 level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=1 total="0 B"
time=2024-08-12T14:55:41.054+02:00 level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=2 total="0 B"
time=2024-08-12T14:55:41.055+02:00 level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=3 total="0 B"
time=2024-08-12T14:55:41.055+02:00 level=INFO source=amd_linux.go:360 msg="no compatible amdgpu devices detected"
time=2024-08-12T14:55:41.055+02:00 level=INFO source=gpu.go:350 msg="no compatible GPUs were discovered"
time=2024-08-12T14:55:41.055+02:00 level=INFO source=types.go:105 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="502.2 GiB" available="469.6 GiB"

@GKeppler commented on GitHub (Aug 12, 2024): I have a similar situation, Mi300As are not usable. Ollama Version is 0.3.5. Running `./ollama-linux-amd64 serve` yields: ``` 2024/08/12 14:55:33 routes.go:1123: INFO server config env="map[CUDA_VISIBLE_DEVICES:0,1 GPU_DEVICE_ORDINAL:0,1 HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/home/hk-project-test-p0020932/sb7059/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR: OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:0,1]" time=2024-08-12T14:55:34.541+02:00 level=INFO source=images.go:782 msg="total blobs: 19" time=2024-08-12T14:55:34.545+02:00 level=INFO source=images.go:790 msg="total unused blobs removed: 0" time=2024-08-12T14:55:34.545+02:00 level=INFO source=routes.go:1170 msg="Listening on [::]:11434 (version 0.3.5)" time=2024-08-12T14:55:34.546+02:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/scratch/slurm_tmpdir/job_1139/ollama1144170004/runners time=2024-08-12T14:55:41.044+02:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11 rocm_v60102]" time=2024-08-12T14:55:41.044+02:00 level=INFO source=gpu.go:204 msg="looking for compatible GPUs" time=2024-08-12T14:55:41.053+02:00 level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=0 total="0 B" time=2024-08-12T14:55:41.053+02:00 level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=1 total="0 B" time=2024-08-12T14:55:41.054+02:00 level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=2 total="0 B" time=2024-08-12T14:55:41.055+02:00 level=INFO source=amd_linux.go:274 msg="unsupported Radeon iGPU detected skipping" id=3 total="0 B" time=2024-08-12T14:55:41.055+02:00 level=INFO source=amd_linux.go:360 msg="no compatible amdgpu devices detected" time=2024-08-12T14:55:41.055+02:00 level=INFO source=gpu.go:350 msg="no compatible GPUs were discovered" time=2024-08-12T14:55:41.055+02:00 level=INFO source=types.go:105 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="502.2 GiB" available="469.6 GiB" ```

GiteaMirror commented

2026-04-12 14:49:44 -05:00

@aimanyounises1 commented on GitHub (Aug 13, 2024):

Hi,

anyone can please help?

@aimanyounises1 commented on GitHub (Aug 13, 2024): Hi, anyone can please help?

GiteaMirror commented

2026-04-12 14:49:45 -05:00

@dhiltgen commented on GitHub (Aug 18, 2024):

Could you try running the container with -e OLLAMA_DEBUG=1 and share the startup logs? That will report more details on GPU discovery. From the non-debug logs, it seems like we may be misidentifying the GPUS as if they were iGPUs due to zero VRAM being reported.

@dhiltgen commented on GitHub (Aug 18, 2024): Could you try running the container with `-e OLLAMA_DEBUG=1` and share the startup logs? That will report more details on GPU discovery. From the non-debug logs, it seems like we may be misidentifying the GPUS as if they were iGPUs due to zero VRAM being reported.

GiteaMirror commented

2026-04-12 14:49:45 -05:00

@aimanyounises1 commented on GitHub (Sep 1, 2024):

hi @dhiltgen PSB logs:

Please note that in prev version it was working I also have 4 gpus of MI210 Instinct

OLLAMA_DEBUG=1 ollama serve
2024/09/02 03:20:23 routes.go:1100: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/genaiusers/genaiusr1/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR: OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]"
time=2024-09-02T03:20:23.599+05:30 level=INFO source=images.go:784 msg="total blobs: 45"
time=2024-09-02T03:20:23.600+05:30 level=INFO source=images.go:791 msg="total unused blobs removed: 0"
time=2024-09-02T03:20:23.600+05:30 level=INFO source=routes.go:1147 msg="Listening on 127.0.0.1:11434 (version 0.2.8)"
time=2024-09-02T03:20:23.600+05:30 level=WARN source=assets.go:81 msg="failed to read ollama.pid" path=/tmp/ollama1513572561 error="open /tmp/ollama1513572561/ollama.pid: permission denied"
time=2024-09-02T03:20:23.600+05:30 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama3114635962/runners
time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=cpu file=build/linux/x86_64/cpu/bin/ollama_llama_server.gz
time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=cpu_avx file=build/linux/x86_64/cpu_avx/bin/ollama_llama_server.gz
time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=cpu_avx2 file=build/linux/x86_64/cpu_avx2/bin/ollama_llama_server.gz
time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcublas.so.11.gz
time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcublasLt.so.11.gz
time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcudart.so.11.0.gz
time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/ollama_llama_server.gz
time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=rocm_v60102 file=build/linux/x86_64/rocm_v60102/bin/deps.txt.gz
time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=rocm_v60102 file=build/linux/x86_64/rocm_v60102/bin/ollama_llama_server.gz
time=2024-09-02T03:20:26.396+05:30 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3114635962/runners/cpu/ollama_llama_server
time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3114635962/runners/cpu_avx/ollama_llama_server
time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3114635962/runners/cpu_avx2/ollama_llama_server
time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3114635962/runners/cuda_v11/ollama_llama_server
time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3114635962/runners/rocm_v60102/ollama_llama_server
time=2024-09-02T03:20:26.397+05:30 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11 rocm_v60102]"
time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=payload.go:45 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY"
time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=sched.go:102 msg="starting llm scheduler"
time=2024-09-02T03:20:26.397+05:30 level=INFO source=gpu.go:205 msg="looking for compatible GPUs"
time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=gpu.go:91 msg="searching for GPU discovery libraries for NVIDIA"
time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=gpu.go:468 msg="Searching for GPU library" name=libcuda.so*
time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=gpu.go:487 msg="gpu library search" globs="[/opt/rocm-6.1.2/lib/libcuda.so** /genaiusers/genaiusr1/CLBlast/build/lib64/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=gpu.go:521 msg="discovered GPU libraries" paths=[]
time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=gpu.go:468 msg="Searching for GPU library" name=libcudart.so*
time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=gpu.go:487 msg="gpu library search" globs="[/opt/rocm-6.1.2/lib/libcudart.so** /genaiusers/genaiusr1/CLBlast/build/lib64/libcudart.so** /tmp/ollama3114635962/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
time=2024-09-02T03:20:26.398+05:30 level=DEBUG source=gpu.go:521 msg="discovered GPU libraries" paths=[/tmp/ollama3114635962/runners/cuda_v11/libcudart.so.11.0]
cudaSetDevice err: 35
time=2024-09-02T03:20:26.398+05:30 level=DEBUG source=gpu.go:533 msg="Unable to load cudart" library=/tmp/ollama3114635962/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
time=2024-09-02T03:20:26.398+05:30 level=WARN source=amd_linux.go:58 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:87 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:112 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:87 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:112 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/1/properties"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:87 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:202 msg="mapping amdgpu to drm sysfs nodes" amdgpu=/sys/class/kfd/kfd/topology/nodes/2/properties vendor=4098 device=29711 unique_id=1366641326874471152
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:215 msg="failed to read sysfs node" file=/sys/class/drm/card0-VGA-1/device/vendor error="open /sys/class/drm/card0-VGA-1/device/vendor: no such file or directory"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:236 msg=matched amdgpu=/sys/class/kfd/kfd/topology/nodes/2/properties drm=/sys/class/drm/card1/device
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:240 msg="failed to read sysfs node" file=/sys/class/drm/card1/device/mem_info_vram_total error="open /sys/class/drm/card1/device/mem_info_vram_total: no such file or directory"
time=2024-09-02T03:20:26.399+05:30 level=INFO source=amd_linux.go:259 msg="unsupported Radeon iGPU detected skipping" id=0 total="0 B"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:87 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/3/properties"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:202 msg="mapping amdgpu to drm sysfs nodes" amdgpu=/sys/class/kfd/kfd/topology/nodes/3/properties vendor=4098 device=29711 unique_id=7794571320916697573
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:215 msg="failed to read sysfs node" file=/sys/class/drm/card0-VGA-1/device/vendor error="open /sys/class/drm/card0-VGA-1/device/vendor: no such file or directory"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:236 msg=matched amdgpu=/sys/class/kfd/kfd/topology/nodes/3/properties drm=/sys/class/drm/card2/device
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:240 msg="failed to read sysfs node" file=/sys/class/drm/card2/device/mem_info_vram_total error="open /sys/class/drm/card2/device/mem_info_vram_total: no such file or directory"
time=2024-09-02T03:20:26.399+05:30 level=INFO source=amd_linux.go:259 msg="unsupported Radeon iGPU detected skipping" id=1 total="0 B"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:87 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/4/properties"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:202 msg="mapping amdgpu to drm sysfs nodes" amdgpu=/sys/class/kfd/kfd/topology/nodes/4/properties vendor=4098 device=29711 unique_id=7578450756494646485
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:215 msg="failed to read sysfs node" file=/sys/class/drm/card0-VGA-1/device/vendor error="open /sys/class/drm/card0-VGA-1/device/vendor: no such file or directory"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:236 msg=matched amdgpu=/sys/class/kfd/kfd/topology/nodes/4/properties drm=/sys/class/drm/card3/device
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:240 msg="failed to read sysfs node" file=/sys/class/drm/card3/device/mem_info_vram_total error="open /sys/class/drm/card3/device/mem_info_vram_total: no such file or directory"
time=2024-09-02T03:20:26.399+05:30 level=INFO source=amd_linux.go:259 msg="unsupported Radeon iGPU detected skipping" id=2 total="0 B"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:87 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/5/properties"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:202 msg="mapping amdgpu to drm sysfs nodes" amdgpu=/sys/class/kfd/kfd/topology/nodes/5/properties vendor=4098 device=29711 unique_id=11320472471763640405
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:215 msg="failed to read sysfs node" file=/sys/class/drm/card0-VGA-1/device/vendor error="open /sys/class/drm/card0-VGA-1/device/vendor: no such file or directory"
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:236 msg=matched amdgpu=/sys/class/kfd/kfd/topology/nodes/5/properties drm=/sys/class/drm/card4/device
time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:240 msg="failed to read sysfs node" file=/sys/class/drm/card4/device/mem_info_vram_total error="open /sys/class/drm/card4/device/mem_info_vram_total: no such file or directory"
time=2024-09-02T03:20:26.399+05:30 level=INFO source=amd_linux.go:259 msg="unsupported Radeon iGPU detected skipping" id=3 total="0 B"
time=2024-09-02T03:20:26.399+05:30 level=INFO source=amd_linux.go:345 msg="no compatible amdgpu devices detected"
time=2024-09-02T03:20:26.399+05:30 level=INFO source=gpu.go:346 msg="no compatible GPUs were discovered"
time=2024-09-02T03:20:26.399+05:30 level=INFO source=types.go:105 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="250.9 GiB" available="205.4 GiB"

@aimanyounises1 commented on GitHub (Sep 1, 2024): hi @dhiltgen PSB logs: Please note that in prev version it was working I also have 4 gpus of MI210 Instinct ``` OLLAMA_DEBUG=1 ollama serve 2024/09/02 03:20:23 routes.go:1100: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST:http://127.0.0.1:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/genaiusers/genaiusr1/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR: OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES:]" time=2024-09-02T03:20:23.599+05:30 level=INFO source=images.go:784 msg="total blobs: 45" time=2024-09-02T03:20:23.600+05:30 level=INFO source=images.go:791 msg="total unused blobs removed: 0" time=2024-09-02T03:20:23.600+05:30 level=INFO source=routes.go:1147 msg="Listening on 127.0.0.1:11434 (version 0.2.8)" time=2024-09-02T03:20:23.600+05:30 level=WARN source=assets.go:81 msg="failed to read ollama.pid" path=/tmp/ollama1513572561 error="open /tmp/ollama1513572561/ollama.pid: permission denied" time=2024-09-02T03:20:23.600+05:30 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama3114635962/runners time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=cpu file=build/linux/x86_64/cpu/bin/ollama_llama_server.gz time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=cpu_avx file=build/linux/x86_64/cpu_avx/bin/ollama_llama_server.gz time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=cpu_avx2 file=build/linux/x86_64/cpu_avx2/bin/ollama_llama_server.gz time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcublas.so.11.gz time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcublasLt.so.11.gz time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcudart.so.11.0.gz time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/ollama_llama_server.gz time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=rocm_v60102 file=build/linux/x86_64/rocm_v60102/bin/deps.txt.gz time=2024-09-02T03:20:23.600+05:30 level=DEBUG source=payload.go:182 msg=extracting variant=rocm_v60102 file=build/linux/x86_64/rocm_v60102/bin/ollama_llama_server.gz time=2024-09-02T03:20:26.396+05:30 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3114635962/runners/cpu/ollama_llama_server time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3114635962/runners/cpu_avx/ollama_llama_server time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3114635962/runners/cpu_avx2/ollama_llama_server time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3114635962/runners/cuda_v11/ollama_llama_server time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3114635962/runners/rocm_v60102/ollama_llama_server time=2024-09-02T03:20:26.397+05:30 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11 rocm_v60102]" time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=payload.go:45 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY" time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=sched.go:102 msg="starting llm scheduler" time=2024-09-02T03:20:26.397+05:30 level=INFO source=gpu.go:205 msg="looking for compatible GPUs" time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=gpu.go:91 msg="searching for GPU discovery libraries for NVIDIA" time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=gpu.go:468 msg="Searching for GPU library" name=libcuda.so* time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=gpu.go:487 msg="gpu library search" globs="[/opt/rocm-6.1.2/lib/libcuda.so** /genaiusers/genaiusr1/CLBlast/build/lib64/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=gpu.go:521 msg="discovered GPU libraries" paths=[] time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=gpu.go:468 msg="Searching for GPU library" name=libcudart.so* time=2024-09-02T03:20:26.397+05:30 level=DEBUG source=gpu.go:487 msg="gpu library search" globs="[/opt/rocm-6.1.2/lib/libcudart.so** /genaiusers/genaiusr1/CLBlast/build/lib64/libcudart.so** /tmp/ollama3114635962/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" time=2024-09-02T03:20:26.398+05:30 level=DEBUG source=gpu.go:521 msg="discovered GPU libraries" paths=[/tmp/ollama3114635962/runners/cuda_v11/libcudart.so.11.0] cudaSetDevice err: 35 time=2024-09-02T03:20:26.398+05:30 level=DEBUG source=gpu.go:533 msg="Unable to load cudart" library=/tmp/ollama3114635962/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" time=2024-09-02T03:20:26.398+05:30 level=WARN source=amd_linux.go:58 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:87 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:112 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:87 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:112 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/1/properties" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:87 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:202 msg="mapping amdgpu to drm sysfs nodes" amdgpu=/sys/class/kfd/kfd/topology/nodes/2/properties vendor=4098 device=29711 unique_id=1366641326874471152 time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:215 msg="failed to read sysfs node" file=/sys/class/drm/card0-VGA-1/device/vendor error="open /sys/class/drm/card0-VGA-1/device/vendor: no such file or directory" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:236 msg=matched amdgpu=/sys/class/kfd/kfd/topology/nodes/2/properties drm=/sys/class/drm/card1/device time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:240 msg="failed to read sysfs node" file=/sys/class/drm/card1/device/mem_info_vram_total error="open /sys/class/drm/card1/device/mem_info_vram_total: no such file or directory" time=2024-09-02T03:20:26.399+05:30 level=INFO source=amd_linux.go:259 msg="unsupported Radeon iGPU detected skipping" id=0 total="0 B" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:87 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/3/properties" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:202 msg="mapping amdgpu to drm sysfs nodes" amdgpu=/sys/class/kfd/kfd/topology/nodes/3/properties vendor=4098 device=29711 unique_id=7794571320916697573 time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:215 msg="failed to read sysfs node" file=/sys/class/drm/card0-VGA-1/device/vendor error="open /sys/class/drm/card0-VGA-1/device/vendor: no such file or directory" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:236 msg=matched amdgpu=/sys/class/kfd/kfd/topology/nodes/3/properties drm=/sys/class/drm/card2/device time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:240 msg="failed to read sysfs node" file=/sys/class/drm/card2/device/mem_info_vram_total error="open /sys/class/drm/card2/device/mem_info_vram_total: no such file or directory" time=2024-09-02T03:20:26.399+05:30 level=INFO source=amd_linux.go:259 msg="unsupported Radeon iGPU detected skipping" id=1 total="0 B" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:87 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/4/properties" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:202 msg="mapping amdgpu to drm sysfs nodes" amdgpu=/sys/class/kfd/kfd/topology/nodes/4/properties vendor=4098 device=29711 unique_id=7578450756494646485 time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:215 msg="failed to read sysfs node" file=/sys/class/drm/card0-VGA-1/device/vendor error="open /sys/class/drm/card0-VGA-1/device/vendor: no such file or directory" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:236 msg=matched amdgpu=/sys/class/kfd/kfd/topology/nodes/4/properties drm=/sys/class/drm/card3/device time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:240 msg="failed to read sysfs node" file=/sys/class/drm/card3/device/mem_info_vram_total error="open /sys/class/drm/card3/device/mem_info_vram_total: no such file or directory" time=2024-09-02T03:20:26.399+05:30 level=INFO source=amd_linux.go:259 msg="unsupported Radeon iGPU detected skipping" id=2 total="0 B" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:87 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/5/properties" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:202 msg="mapping amdgpu to drm sysfs nodes" amdgpu=/sys/class/kfd/kfd/topology/nodes/5/properties vendor=4098 device=29711 unique_id=11320472471763640405 time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:215 msg="failed to read sysfs node" file=/sys/class/drm/card0-VGA-1/device/vendor error="open /sys/class/drm/card0-VGA-1/device/vendor: no such file or directory" time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:236 msg=matched amdgpu=/sys/class/kfd/kfd/topology/nodes/5/properties drm=/sys/class/drm/card4/device time=2024-09-02T03:20:26.399+05:30 level=DEBUG source=amd_linux.go:240 msg="failed to read sysfs node" file=/sys/class/drm/card4/device/mem_info_vram_total error="open /sys/class/drm/card4/device/mem_info_vram_total: no such file or directory" time=2024-09-02T03:20:26.399+05:30 level=INFO source=amd_linux.go:259 msg="unsupported Radeon iGPU detected skipping" id=3 total="0 B" time=2024-09-02T03:20:26.399+05:30 level=INFO source=amd_linux.go:345 msg="no compatible amdgpu devices detected" time=2024-09-02T03:20:26.399+05:30 level=INFO source=gpu.go:346 msg="no compatible GPUs were discovered" time=2024-09-02T03:20:26.399+05:30 level=INFO source=types.go:105 msg="inference compute" id=0 library=cpu compute="" driver=0.0 name="" total="250.9 GiB" available="205.4 GiB" ```

GiteaMirror commented

2026-04-12 14:49:45 -05:00

@aimanyounises1 commented on GitHub (Sep 2, 2024):

@dhiltgen what can be the reason of this please?

also version 0.1.43 is working fine with Ollama

@aimanyounises1 commented on GitHub (Sep 2, 2024): @dhiltgen what can be the reason of this please? also version 0.1.43 is working fine with Ollama

GiteaMirror commented

2026-04-12 14:49:45 -05:00

@dhiltgen commented on GitHub (Sep 3, 2024):

@aimanyounises1 the 0.2.8 logs you pasted above show it was unsuccessful in discovering the AMD GPU and was using cpu for inference.

Between 0.1.43 and 0.2.8+ we switched to using the Direct Rendering Manager (DRM) sysfs nodes which report VRAM usage across multiple apps on the GPU so we can accurately predict if a model will fit while scheduling. For some reason in your system it seems these DRM sysfs nodes are not behaving as we expect.

If you can exec into one of these containers with the AMD device nodes passed in and run the following, that may help shed some light on where the algorithm is going wrong. We don't have any Instinct cards in our test bed, so maybe there's a unique pattern in how they're modeled in the drm tree that's tripping things up.

ls /sys/class/kfd/kfd/topology/nodes/*/properties
cat /sys/class/kfd/kfd/topology/nodes/*/properties
ls /sys/class/drm/card*/device/
ls /sys/class/drm/card*/device/vendor
ls /sys/class/drm/card*/device/unique_id

@dhiltgen commented on GitHub (Sep 3, 2024): @aimanyounises1 the 0.2.8 logs you pasted above show it was unsuccessful in discovering the AMD GPU and was using cpu for inference. Between 0.1.43 and 0.2.8+ we switched to using the Direct Rendering Manager (DRM) sysfs nodes which report VRAM usage across multiple apps on the GPU so we can accurately predict if a model will fit while scheduling. For some reason in your system it seems these DRM sysfs nodes are not behaving as we expect. If you can exec into one of these containers with the AMD device nodes passed in and run the following, that may help shed some light on where the algorithm is going wrong. We don't have any Instinct cards in our test bed, so maybe there's a unique pattern in how they're modeled in the drm tree that's tripping things up. ``` ls /sys/class/kfd/kfd/topology/nodes/*/properties cat /sys/class/kfd/kfd/topology/nodes/*/properties ls /sys/class/drm/card*/device/ ls /sys/class/drm/card*/device/vendor ls /sys/class/drm/card*/device/unique_id ```

GiteaMirror referenced this issue

2026-04-22 06:06:48 -05:00

[GH-ISSUE #3953] Support VLLM as a backend #28210

GiteaMirror referenced this issue

2026-04-28 10:18:23 -05:00

[GH-ISSUE #3953] Support VLLM as a backend #48962

GiteaMirror referenced this issue

2026-05-03 17:49:52 -05:00

[GH-ISSUE #3953] Support VLLM as a backend #64488

Sign in to join this conversation.

Branches Tags

main

dhiltgen/ci

parth-launch-plan-gating

hoyyeva/anthropic-reference-images-path

parth-anthropic-reference-images-path

brucemacd/download-before-remove

hoyyeva/editor-config-repair

parth-mlx-decode-checkpoints

parth-launch-codex-app

hoyyeva/fix-codex-model-metadata-warning

hoyyeva/qwen

parth/hide-claude-desktop-till-release

hoyyeva/opencode-image-modality

parth-add-claude-code-autoinstall

release_v0.22.0

pdevine/manifest-list

codex/fix-codex-model-metadata-warning

pdevine/addressable-manifest

brucemacd/launch-fetch-reccomended

jmorganca/llama-compat

launch-copilot-cli

hoyyeva/opencode-thinking

release_v0.20.7

parth-auto-save-backup

parth-test

jmorganca/gemma4-audio-replacements

fix-manifest-digest-on-pull

hoyyeva/vscode-improve

brucemacd/install-server-wait

parth/update-claude-docs

brucemac/start-ap-install

pdevine/mlx-update

pdevine/qwen35_vision

drifkin/api-show-fallback

mintlify/image-generation-1773352582

hoyyeva/server-context-length-local-config

jmorganca/faster-reptition-penalties

jmorganca/convert-nemotron

parth-pi-thinking

pdevine/sampling-penalties

jmorganca/fix-create-quantization-memory

dongchen/resumable_transfer_fix

pdevine/sampling-cache-error

jessegross/mlx-usage

hoyyeva/openclaw-config

hoyyeva/app-html

pdevine/qwen3next

brucemacd/sign-sh-install

brucemacd/tui-update

brucemacd/usage-api

jmorganca/launch-empty

fix-app-dist-embed

mxyng/mlx-compile

mxyng/mlx-quant

mxyng/mlx-glm4.7

mxyng/mlx

brucemacd/simplify-model-picker

jmorganca/qwen3-concurrent

fix-glm-4.7-flash-mla-config

drifkin/qwen3-coder-opening-tag

brucemacd/usage-cli

fix-cuda12-fattn-shmem

ollama-imagegen-docs

parth/fix-multiline-inputs

brucemacd/config-docs

mxyng/model-files

mxyng/simple-execute

fix-imagegen-ollama-models

mxyng/async-upload

jmorganca/lazy-no-dtype-changes

imagegen-auto-detect-create

parth/decrease-concurrent-download-hf

fix-mlx-quantize-init

jmorganca/x-cleanup

usage

imagegen-readme

jmorganca/glm-image

mlx-gpu-cd

jmorganca/imagegen-modelfile

parth/agent-skills

parth/agent-allowlist

parth/signed-in-offline

parth/agents

parth/fix-context-chopping

improve-cloud-flow

parth/add-models-websearch

parth/prompt-renderer-mcp

jmorganca/native-settings

jmorganca/download-stream-hash

jmorganca/client2-rebased

brucemacd/oai-chat-req-multipart

jessegross/multi_chunk_reserve

grace/additional-omit-empty

grace/mistral-3-large

mxyng/tokenizer2

mxyng/tokenizer

jessegross/flash

hoyyeva/windows-nacked-app

mxyng/cleanup-attention

grace/deepseek-parser

hoyyeva/remember-unsent-prompt

parth/add-lfs-pointer-error-conversion

parth/olmo2-test2

hoyyeva/ollama-launchagent-plist

nicole/olmo-model

parth/olmo-test

mxyng/remove-embedded

parth/render-template

jmorganca/intellect-3

parth/remove-prealloc-linter

jmorganca/cmd-eval

nicole/nomic-embed-text-fix

mxyng/lint-2

hoyyeva/add-gemini-3-pro-preview

hoyyeva/load-model-list

mxyng/expand-path

mxyng/environ-2

hoyyeva/deeplink-json-encoding

parth/improve-tool-calling-tests

hoyyeva/conversation

hoyyeva/assistant-edit-response

hoyyeva/thinking

origin/brucemacd/invalid-char-i-err

parth/improve-tool-calling

jmorganca/required-omitempty

grace/qwen3-vl-tests

mxyng/iter-client

parth/docs-readme

nicole/embed-test

pdevine/integration-benchstat

parth/remove-generate-cmd

parth/add-toolcall-id

mxyng/server-tests

jmorganca/glm-4.6

jmorganca/gin-h-compat

drifkin/stable-tool-args

pdevine/qwen3-more-thinking

parth/add-websearch-client

nicole/websearch_local

jmorganca/qwen3-coder-updates

grace/deepseek-v3-migration-tests

mxyng/fix-create

jmorganca/cloud-errors

pdevine/parser-tidy

revert-12233-parth/simplify-entrypoints-runner

parth/enable-so-gpt-oss

brucemacd/qwen3vl

jmorganca/readme-simplify

parth/gpt-oss-structured-outputs

revert-12039-jmorganca/tools-braces

mxyng/embeddings

mxyng/gguf

mxyng/benchmark

mxyng/types-null

parth/move-parsing

mxyng/gemma2

jmorganca/docs

mxyng/16-bit

mxyng/create-stdin

pdevine/authorizedkeys

mxyng/quant

parth/opt-in-error-context-window

brucemacd/cache-models

brucemacd/runner-completion

jmorganca/llama-update-6

brucemacd/benchmark-list

brucemacd/partial-read-caps

parth/deepseek-r1-tools

mxyng/omit-array

parth/tool-prefix-temp

brucemacd/runner-test

jmorganca/qwen25vl

brucemacd/model-forward-test-ext

parth/python-function-parsing

jmorganca/cuda-compression-none

drifkin/num-parallel

drifkin/chat-truncation-fix

jmorganca/sync

parth/python-tools-calling

drifkin/array-head-count

brucemacd/create-no-loop

parth/server-enable-content-stream-with-tools

qwen25omni

mxyng/v3

brucemacd/ropeconfig

jmorganca/silence-tokenizer

parth/sample-so-test

parth/sampling-structured-outputs

brucemacd/doc-go-engine

parth/constrained-sampling-json

jmorganca/mistral-wip

brucemacd/mistral-small-convert

parth/sample-unmarshal-json-for-params

brucemacd/jomorganca/mistral

pdevine/bfloat16

jmorganca/mistral

brucemacd/mistral

pdevine/logging

parth/sample-correctness-fix

parth/sample-fix-sorting

jmorgan/sample-fix-sorting-extras

jmorganca/temp-0-images

brucemacd/parallel-embed-models

brucemacd/shim-grammar

jmorganca/fix-gguf-error

bmizerany/nameswork

jmorganca/faster-releases

bmizerany/validatenames

brucemacd/err-no-vocab

brucemacd/rope-config

brucemacd/err-hint

brucemacd/qwen2_5

brucemacd/logprobs

brucemacd/new_runner_graph_bench

progress-flicker

brucemacd/forward-test

brucemacd/go_qwen2

pdevine/gemma2

jmorganca/add-missing-symlink-eval

mxyng/next-debug

parth/set-context-size-openai

brucemacd/next-bpe-bench

brucemacd/next-bpe-test

brucemacd/new_runner_e2e

brucemacd/new_runner_qwen2

pdevine/convert-cohere2

brucemacd/convert-cli

parth/log-probs

mxyng/next-mlx

mxyng/cmd-history

parth/templating

parth/tokenize-detokenize

brucemacd/check-key-register

bmizerany/grammar

jmorganca/vendor-081b29bd

mxyng/func-checks

jmorganca/fix-null-format

parth/fix-default-to-warn-json

jmorganca/qwen2vl

jmorganca/no-concat

parth/cmd-cleanup-SO

brucemacd/check-key-register-structured-err

parth/openai-stream-usage

parth/fix-referencing-so

stream-tools-stop

jmorganca/degin-1

brucemacd/install-path-clean

brucemacd/push-name-validation

brucemacd/browser-key-register

jmorganca/openai-fix-first-message

jmorganca/fix-proxy

jessegross/sample

parth/disallow-streaming-tools

dhiltgen/remove_submodule

jmorganca/ga

jmorganca/mllama

pdevine/newlines

pdevine/geems-2b

jmorganca/llama-bump

mxyng/modelname-7

mxyng/gin-slog

mxyng/modelname-6

jyan/convert-prog

jyan/quant5

paligemma-support

pdevine/import-docs

jmorganca/openai-context

jyan/paligemma

jyan/p2

jyan/palitest

bmizerany/embedspeedup

jmorganca/llama-vit

brucemacd/allow-ollama

royh/ep-methods

royh/whisper

mxyng/api-models

mxyng/fix-memory

jyan/q4_4/8

jyan/ollama-v

royh/stream-tools

roy-embed-parallel

bmizerany/hrm

revert-5963-revert-5924-mxyng/llama3.1-rope

royh/embed-viz

jyan/local2

jyan/auth

jyan/local

jyan/parse-temp

jmorganca/template-mistral

jyan/reord-g

royh-openai-suffixdocs

royh-imgembed

royh-embed-parallel

jyan/quant4

royh-precision

jyan/progress

pdevine/fix-template

jyan/quant3

pdevine/ggla

mxyng/update-registry-domain

jmorganca/ggml-static

mxyng/create-context

jyan/v0.146

mxyng/layers-from-files

build_dist

bmizerany/noseek

royh-ls

royh-name

timeout

mxyng/server-timestamp

bmizerany/nosillyggufslurps

royh-params

jmorganca/llama-cpp-7c26775

royh-openai-delete

royh-show-rigid

jmorganca/enable-fa

jmorganca/no-error-template

jyan/format

royh-testdelete

bmizerany/fastverify

language_support

pdevine/ps-glitches

brucemacd/tokenize

bruce/iq-quants

bmizerany/filepathwithcoloninhost

mxyng/split-bin

bmizerany/client-registry

jmorganca/if-none-match

native

jmorganca/native

jmorganca/batch-embeddings

jmorganca/initcmake

jmorganca/mm

pdevine/showggmlinfo

modenameenforcealphanum

bmizerany/modenameenforcealphanum

jmorganca/done-reason

jmorganca/llama-cpp-8960fe8

ollama.com

bmizerany/filepathnobuild

bmizerany/types/model/defaultfix

rmdisplaylong

nogogen

bmizerany/x

modelfile-readme

bmizerany/replacecolon

jmorganca/limit

jmorganca/execstack

jmorganca/replace-assets

mxyng/tune-concurrency

jmorganca/testing

whitespace-detection

jmorganca/options

upgrade-all

scratch

cuda-search

mattw/airenamer

mattw/allmodelsonhuggingface

mattw/quantcontext

mattw/whatneedstorun

brucemacd/llama-mem-calc

mattw/faq-context

mattw/communitylinks

mattw/noprune

mattw/python-functioncalling

rename

mxyng/install

pulse

remove-first

editor

mattw/selfqueryingretrieval

cgo

mattw/howtoquant

api

matt/streamingapi

format-config

mxyng/extra-args

shell

update-nous-hermes

cp-model

upload-progress

fix-unknown-model

fix-model-names

delete-fix

insecure-registry

ls

deletemodels

progressbar

readme-updates

license-layers

skip-list

list-models

modelpath

matt/examplemodelfiles

distribution

go-opts

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/ollama#3953