github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 16:40:08 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #5000] Regression on ollama docker images >=0.1.33-rocm: rocBLAS does not find secondary GPU for Tensile #49674

New Issue
Closed
opened 2026-04-28 12:36:46 -05:00 by GiteaMirror · 8 comments
GiteaMirror commented 2026-04-28 12:36:46 -05:00
Owner
Copy Link

Originally created by @ic4-y on GitHub (Jun 12, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/5000

Originally assigned to: @dhiltgen on GitHub.

What is the issue?

Updating my docker stack from the ollama 0.1.32-rocm docker image to 0.1.33-rocm and above (tested up to 0.1.43-rocm) leads to my secondary GPU becoming unusable for ollama with the following

ollama-rocm | rocBLAS error: Could not initialize Tensile host: No devices found

The secondary GPU is a Radeon Pro W6600 and the primary GPU is a Radeon Pro W6800. The configuration that I am using is isolating the secondary GPU for use with ollama via the ROCR_VISIBLE_DEVICES=1 environment variable. This used to work just fine with the 0.1.32-rocm image from DockerHub (and still does).

Since I am running an identical second Radeon Pro W6600 as sole GPU in a server, neither the GPU nor the docker image itself should be the issue. In that machine everything works fine.

Most likely there is some problem with the dual GPU configuration and the way I am trying to isolate the second GPU for use with ollama. But I don't really know what it might be at this point.

Any help is very much appreciated!

Here is my compose.yaml file:

services:
    ollama:
        privileged: true
        devices:
            - /dev/kfd:/dev/kfd
            - /dev/dri:/dev/dri
        volumes:
            - ollama:/root/.ollama
        ports:
            - 11434:11434
        environment:
            - OLLAMA_DEBUG=1
            - HSA_OVERRIDE_GFX_VERSION=10.3.0
            - ROCR_VISIBLE_DEVICES=1
#- GPU_MAX_HW_QUEUES=1
        container_name: ollama-rocm
        restart: always
        image: ollama/ollama:0.1.33-rocm
    open-webui:
        ports:
            - 3000:8080
        volumes:
            - open-webui:/app/backend/data
        environment:
            - OLLAMA_BASE_URLS=http://ollama-rocm:11434
        container_name: open-webui
        restart: always
        image: ghcr.io/open-webui/open-webui:main

volumes:
    ollama:
        external: false
        name: ollama
    open-webui:
        external: false
        name: open-webui

System info:

  • Linux 6.6.32 NixOS
  • Ryzen 3900X
  • Dual GPU, Radeon Pro W6800 and Radeon Pro W6600

OS

Linux

GPU

AMD

CPU

AMD

Ollama version

0.1.33 and above

Originally created by @ic4-y on GitHub (Jun 12, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/5000 Originally assigned to: @dhiltgen on GitHub. ### What is the issue? Updating my docker stack from the ollama `0.1.32-rocm` docker image to `0.1.33-rocm` and above (tested up to `0.1.43-rocm`) leads to my secondary GPU becoming unusable for ollama with the following > ollama-rocm | rocBLAS error: Could not initialize Tensile host: No devices found The secondary GPU is a Radeon Pro W6600 and the primary GPU is a Radeon Pro W6800. The configuration that I am using is isolating the secondary GPU for use with ollama via the `ROCR_VISIBLE_DEVICES=1` environment variable. This used to work just fine with the `0.1.32-rocm` image from DockerHub (and still does). Since I am running an identical second Radeon Pro W6600 as sole GPU in a server, neither the GPU nor the docker image itself should be the issue. In that machine everything works fine. Most likely there is some problem with the dual GPU configuration and the way I am trying to isolate the second GPU for use with ollama. But I don't really know what it might be at this point. Any help is very much appreciated! Here is my `compose.yaml` file: ```yaml services: ollama: privileged: true devices: - /dev/kfd:/dev/kfd - /dev/dri:/dev/dri volumes: - ollama:/root/.ollama ports: - 11434:11434 environment: - OLLAMA_DEBUG=1 - HSA_OVERRIDE_GFX_VERSION=10.3.0 - ROCR_VISIBLE_DEVICES=1 #- GPU_MAX_HW_QUEUES=1 container_name: ollama-rocm restart: always image: ollama/ollama:0.1.33-rocm open-webui: ports: - 3000:8080 volumes: - open-webui:/app/backend/data environment: - OLLAMA_BASE_URLS=http://ollama-rocm:11434 container_name: open-webui restart: always image: ghcr.io/open-webui/open-webui:main volumes: ollama: external: false name: ollama open-webui: external: false name: open-webui ``` System info: - Linux 6.6.32 NixOS - Ryzen 3900X - Dual GPU, Radeon Pro W6800 and Radeon Pro W6600 ### OS Linux ### GPU AMD ### CPU AMD ### Ollama version 0.1.33 and above
GiteaMirror added the bugamd labels 2026-04-28 12:36:47 -05:00
GiteaMirror commented 2026-04-28 12:36:50 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Jun 13, 2024):

If you drop the ROCR_VISIBLE_DEVICES setting, do we correctly detect both GPUs?
Can you share the server log with and without that setting so we can see why the GPU detection logic may be having trouble? (may sure to include a model load so we can see the env vars passed to the subprocess.) There may be a logic error in the detection of the variable, then what we pass to the subprocess.

<!-- gh-comment-id:2166386762 --> @dhiltgen commented on GitHub (Jun 13, 2024): If you drop the `ROCR_VISIBLE_DEVICES` setting, do we correctly detect both GPUs? Can you share the server log with and without that setting so we can see why the GPU detection logic may be having trouble? (may sure to include a model load so we can see the env vars passed to the subprocess.) There may be a logic error in the detection of the variable, then what we pass to the subprocess.
GiteaMirror commented 2026-04-28 12:36:50 -05:00
Author
Owner
Copy Link

@ic4-y commented on GitHub (Jun 13, 2024):

I do believe that both GPUs get detected correctly even if I do NOT drop that variable, and the first one gets filtered out correctly. At least they both show up in the log with their respective amount of VRAM.

Let me post the entire log WITH ROCR_VISIBLE_DEVICES being set.

It is a bit of a long one because on ollama 0.1.43 it seems to be stuck in a loop trying to restart the server multiple times after the error message referring to Tensile.

Attaching to ollama-rocm
ollama-rocm  | 2024/06/13 17:39:53 routes.go:1011: INFO server config env="map[OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST: OLLAMA_KEEP_ALIVE: OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_MODELS: OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR: OLLAMA_TMPDIR:]"
ollama-rocm  | time=2024-06-13T17:39:53.285Z level=INFO source=images.go:740 msg="total blobs: 34"
ollama-rocm  | time=2024-06-13T17:39:53.286Z level=INFO source=images.go:747 msg="total unused blobs removed: 0"
ollama-rocm  | time=2024-06-13T17:39:53.286Z level=INFO source=routes.go:1057 msg="Listening on [::]:11434 (version 0.1.43)"
ollama-rocm  | time=2024-06-13T17:39:53.287Z level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama3388558046/runners
ollama-rocm  | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=cpu file=build/linux/x86_64/cpu/bin/ollama_llama_server.gz
ollama-rocm  | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=cpu_avx file=build/linux/x86_64/cpu_avx/bin/ollama_llama_server.gz
ollama-rocm  | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=cpu_avx2 file=build/linux/x86_64/cpu_avx2/bin/ollama_llama_server.gz
ollama-rocm  | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcublas.so.11.gz
ollama-rocm  | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcublasLt.so.11.gz
ollama-rocm  | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcudart.so.11.0.gz
ollama-rocm  | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/ollama_llama_server.gz
ollama-rocm  | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=rocm_v60002 file=build/linux/x86_64/rocm_v60002/bin/deps.txt.gz
ollama-rocm  | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=rocm_v60002 file=build/linux/x86_64/rocm_v60002/bin/ollama_llama_server.gz
ollama-rocm  | time=2024-06-13T17:39:56.070Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu
ollama-rocm  | time=2024-06-13T17:39:56.070Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu_avx
ollama-rocm  | time=2024-06-13T17:39:56.070Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu_avx2
ollama-rocm  | time=2024-06-13T17:39:56.070Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cuda_v11
ollama-rocm  | time=2024-06-13T17:39:56.070Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/rocm_v60002
ollama-rocm  | time=2024-06-13T17:39:56.070Z level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cuda_v11 rocm_v60002 cpu cpu_avx cpu_avx2]"
ollama-rocm  | time=2024-06-13T17:39:56.070Z level=DEBUG source=payload.go:45 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY"
ollama-rocm  | time=2024-06-13T17:39:56.070Z level=DEBUG source=sched.go:90 msg="starting llm scheduler"
ollama-rocm  | time=2024-06-13T17:39:56.070Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:39:56.070Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:39:56.070Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:39:56.072Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:39:56.072Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:39:56.072Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:39:56.073Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:39:56.073Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:39:56.073Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:39:56.073Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:39:56.074Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:39:56.074Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:39:56.075Z level=INFO source=types.go:71 msg="inference compute" id=1 library=rocm compute=gfx1032 driver=0.0 name=1002:73e3 total="8.0 GiB" available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:39.054Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:39.055Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:39.055Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:39.055Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:39.055Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:39.055Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:39.056Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:39.056Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:39.056Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:39.056Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:39.056Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:39.056Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:39.056Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:39.057Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:39.057Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:39.057Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:39.057Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:39.057Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:39.057Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:39.057Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:39.057Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:39.057Z level=DEBUG source=gguf.go:70 msg="model = &llm.gguf{containerGGUF:(*llm.containerGGUF)(0xc0002c7ac0), kv:llm.KV{}, tensors:[]*llm.Tensor(nil), parameters:0x0}"
ollama-rocm  | time=2024-06-13T17:40:40.900Z level=DEBUG source=sched.go:153 msg="loading first model" model=/root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29
ollama-rocm  | time=2024-06-13T17:40:40.900Z level=DEBUG source=memory.go:44 msg=evaluating library=rocm gpu_count=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=33 memory.available="8.0 GiB" memory.required.full="5.0 GiB" memory.required.partial="5.0 GiB" memory.required.kv="256.0 MiB" memory.weights.total="4.1 GiB" memory.weights.repeating="3.7 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="164.0 MiB" memory.graph.partial="677.5 MiB"
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=DEBUG source=sched.go:563 msg="new model will fit in available VRAM in single GPU, loading" model=/root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29 gpu=1 available=8573157376 required="5.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=DEBUG source=memory.go:44 msg=evaluating library=rocm gpu_count=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=33 memory.available="8.0 GiB" memory.required.full="5.0 GiB" memory.required.partial="5.0 GiB" memory.required.kv="256.0 MiB" memory.weights.total="4.1 GiB" memory.weights.repeating="3.7 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="164.0 MiB" memory.graph.partial="677.5 MiB"
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu_avx
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu_avx2
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cuda_v11
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/rocm_v60002
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu_avx
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu_avx2
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cuda_v11
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/rocm_v60002
ollama-rocm  | time=2024-06-13T17:40:40.901Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:40.913Z level=INFO source=server.go:341 msg="starting llama server" cmd="/tmp/ollama3388558046/runners/rocm_v60002/ollama_llama_server --model /root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 33 --verbose --parallel 1 --port 32889"
ollama-rocm  | time=2024-06-13T17:40:40.913Z level=DEBUG source=server.go:356 msg=subprocess environment="[PATH=/opt/rh/devtoolset-7/root/usr/bin:/opt/rocm/hcc/bin:/opt/rocm/hip/bin:/opt/rocm/bin:/opt/rocm/hcc/bin::/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HSA_OVERRIDE_GFX_VERSION=10.3.0 LD_LIBRARY_PATH=/opt/rocm/lib:/tmp/ollama3388558046/runners/rocm_v60002:/opt/rocm/lib:/usr/local/lib:/opt/rh/devtoolset-7/root HIP_VISIBLE_DEVICES=1]"
ollama-rocm  | time=2024-06-13T17:40:40.914Z level=INFO source=sched.go:338 msg="loaded runners" count=1
ollama-rocm  | time=2024-06-13T17:40:40.914Z level=INFO source=server.go:529 msg="waiting for llama runner to start responding"
ollama-rocm  | time=2024-06-13T17:40:40.914Z level=INFO source=server.go:567 msg="waiting for server to become available" status="llm server error"
ollama-rocm  | INFO [main] build info | build=1 commit="5921b8f" tid="140344978488384" timestamp=1718300441
ollama-rocm  | INFO [main] system info | n_threads=12 n_threads_batch=-1 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="140344978488384" timestamp=1718300441 total_threads=24
ollama-rocm  | INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="23" port="32889" tid="140344978488384" timestamp=1718300441
ollama-rocm  | llama_model_loader: loaded meta data with 21 key-value pairs and 291 tensors from /root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29 (version GGUF V3 (latest))
ollama-rocm  | llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
ollama-rocm  | llama_model_loader: - kv   0:                       general.architecture str              = llama
ollama-rocm  | llama_model_loader: - kv   1:                               general.name str              = Meta-Llama-3-8B-Instruct
ollama-rocm  | llama_model_loader: - kv   2:                          llama.block_count u32              = 32
ollama-rocm  | llama_model_loader: - kv   3:                       llama.context_length u32              = 8192
ollama-rocm  | llama_model_loader: - kv   4:                     llama.embedding_length u32              = 4096
ollama-rocm  | llama_model_loader: - kv   5:                  llama.feed_forward_length u32              = 14336
ollama-rocm  | llama_model_loader: - kv   6:                 llama.attention.head_count u32              = 32
ollama-rocm  | llama_model_loader: - kv   7:              llama.attention.head_count_kv u32              = 8
ollama-rocm  | llama_model_loader: - kv   8:                       llama.rope.freq_base f32              = 500000.000000
ollama-rocm  | llama_model_loader: - kv   9:     llama.attention.layer_norm_rms_epsilon f32              = 0.000010
ollama-rocm  | llama_model_loader: - kv  10:                          general.file_type u32              = 2
ollama-rocm  | llama_model_loader: - kv  11:                           llama.vocab_size u32              = 128256
ollama-rocm  | llama_model_loader: - kv  12:                 llama.rope.dimension_count u32              = 128
ollama-rocm  | llama_model_loader: - kv  13:                       tokenizer.ggml.model str              = gpt2
ollama-rocm  | time=2024-06-13T17:40:41.166Z level=INFO source=server.go:567 msg="waiting for server to become available" status="llm server loading model"
ollama-rocm  | llama_model_loader: - kv  14:                      tokenizer.ggml.tokens arr[str,128256]  = ["!", "\"", "#", "$", "%", "&", "'", ...
ollama-rocm  | llama_model_loader: - kv  15:                  tokenizer.ggml.token_type arr[i32,128256]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
ollama-rocm  | llama_model_loader: - kv  16:                      tokenizer.ggml.merges arr[str,280147]  = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "...
ollama-rocm  | llama_model_loader: - kv  17:                tokenizer.ggml.bos_token_id u32              = 128000
ollama-rocm  | llama_model_loader: - kv  18:                tokenizer.ggml.eos_token_id u32              = 128001
ollama-rocm  | llama_model_loader: - kv  19:                    tokenizer.chat_template str              = {% set loop_messages = messages %}{% ...
ollama-rocm  | llama_model_loader: - kv  20:               general.quantization_version u32              = 2
ollama-rocm  | llama_model_loader: - type  f32:   65 tensors
ollama-rocm  | llama_model_loader: - type q4_0:  225 tensors
ollama-rocm  | llama_model_loader: - type q6_K:    1 tensors
ollama-rocm  | llm_load_vocab: missing or unrecognized pre-tokenizer type, using: 'default'
ollama-rocm  | llm_load_vocab: special tokens cache size = 256
ollama-rocm  | llm_load_vocab: token to piece cache size = 1.5928 MB
ollama-rocm  | llm_load_print_meta: format           = GGUF V3 (latest)
ollama-rocm  | llm_load_print_meta: arch             = llama
ollama-rocm  | llm_load_print_meta: vocab type       = BPE
ollama-rocm  | llm_load_print_meta: n_vocab          = 128256
ollama-rocm  | llm_load_print_meta: n_merges         = 280147
ollama-rocm  | llm_load_print_meta: n_ctx_train      = 8192
ollama-rocm  | llm_load_print_meta: n_embd           = 4096
ollama-rocm  | llm_load_print_meta: n_head           = 32
ollama-rocm  | llm_load_print_meta: n_head_kv        = 8
ollama-rocm  | llm_load_print_meta: n_layer          = 32
ollama-rocm  | llm_load_print_meta: n_rot            = 128
ollama-rocm  | llm_load_print_meta: n_embd_head_k    = 128
ollama-rocm  | llm_load_print_meta: n_embd_head_v    = 128
ollama-rocm  | llm_load_print_meta: n_gqa            = 4
ollama-rocm  | llm_load_print_meta: n_embd_k_gqa     = 1024
ollama-rocm  | llm_load_print_meta: n_embd_v_gqa     = 1024
ollama-rocm  | llm_load_print_meta: f_norm_eps       = 0.0e+00
ollama-rocm  | llm_load_print_meta: f_norm_rms_eps   = 1.0e-05
ollama-rocm  | llm_load_print_meta: f_clamp_kqv      = 0.0e+00
ollama-rocm  | llm_load_print_meta: f_max_alibi_bias = 0.0e+00
ollama-rocm  | llm_load_print_meta: f_logit_scale    = 0.0e+00
ollama-rocm  | llm_load_print_meta: n_ff             = 14336
ollama-rocm  | llm_load_print_meta: n_expert         = 0
ollama-rocm  | llm_load_print_meta: n_expert_used    = 0
ollama-rocm  | llm_load_print_meta: causal attn      = 1
ollama-rocm  | llm_load_print_meta: pooling type     = 0
ollama-rocm  | llm_load_print_meta: rope type        = 0
ollama-rocm  | llm_load_print_meta: rope scaling     = linear
ollama-rocm  | llm_load_print_meta: freq_base_train  = 500000.0
ollama-rocm  | llm_load_print_meta: freq_scale_train = 1
ollama-rocm  | llm_load_print_meta: n_yarn_orig_ctx  = 8192
ollama-rocm  | llm_load_print_meta: rope_finetuned   = unknown
ollama-rocm  | llm_load_print_meta: ssm_d_conv       = 0
ollama-rocm  | llm_load_print_meta: ssm_d_inner      = 0
ollama-rocm  | llm_load_print_meta: ssm_d_state      = 0
ollama-rocm  | llm_load_print_meta: ssm_dt_rank      = 0
ollama-rocm  | llm_load_print_meta: model type       = 8B
ollama-rocm  | llm_load_print_meta: model ftype      = Q4_0
ollama-rocm  | llm_load_print_meta: model params     = 8.03 B
ollama-rocm  | llm_load_print_meta: model size       = 4.33 GiB (4.64 BPW) 
ollama-rocm  | llm_load_print_meta: general.name     = Meta-Llama-3-8B-Instruct
ollama-rocm  | llm_load_print_meta: BOS token        = 128000 '<|begin_of_text|>'
ollama-rocm  | llm_load_print_meta: EOS token        = 128001 '<|end_of_text|>'
ollama-rocm  | llm_load_print_meta: LF token         = 128 'Ä'
ollama-rocm  | llm_load_print_meta: EOT token        = 128009 '<|eot_id|>'
ollama-rocm  | 
ollama-rocm  | rocBLAS error: Could not initialize Tensile host: No devices found
ollama-rocm  | time=2024-06-13T17:40:42.118Z level=INFO source=server.go:567 msg="waiting for server to become available" status="llm server not responding"
ollama-rocm  | time=2024-06-13T17:40:42.997Z level=INFO source=server.go:567 msg="waiting for server to become available" status="llm server error"
ollama-rocm  | time=2024-06-13T17:40:43.248Z level=ERROR source=sched.go:344 msg="error loading llama server" error="llama runner process has terminated: signal: aborted (core dumped) error:Could not initialize Tensile host: No devices found"
ollama-rocm  | time=2024-06-13T17:40:43.248Z level=DEBUG source=sched.go:347 msg="triggering expiration for failed load" model=/root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29
ollama-rocm  | time=2024-06-13T17:40:43.248Z level=DEBUG source=sched.go:258 msg="runner expired event received" modelPath=/root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29
ollama-rocm  | time=2024-06-13T17:40:43.248Z level=DEBUG source=sched.go:274 msg="got lock to unload" modelPath=/root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29
ollama-rocm  | time=2024-06-13T17:40:43.248Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:43.248Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:43.248Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | [GIN] 2024/06/13 - 17:40:43 | 500 |  4.193868835s |      172.23.0.1 | POST     "/api/chat"
ollama-rocm  | time=2024-06-13T17:40:43.249Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:43.249Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:43.249Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:43.251Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:43.251Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:43.251Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:43.251Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:43.251Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:43.251Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:43.251Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:43.252Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:43.252Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:43.252Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:43.252Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:43.252Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:43.252Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:43.252Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:43.252Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:43.252Z level=DEBUG source=server.go:990 msg="stopping llama server"
ollama-rocm  | time=2024-06-13T17:40:43.252Z level=DEBUG source=sched.go:279 msg="runner released" modelPath=/root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29
ollama-rocm  | time=2024-06-13T17:40:43.503Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:43.503Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:43.503Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:43.504Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:43.504Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:43.504Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:43.506Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:43.506Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:43.506Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:43.506Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:43.507Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:43.507Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:43.753Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:43.753Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:43.753Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:43.755Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:43.755Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:43.755Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:43.756Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:43.756Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:43.756Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:43.756Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:43.757Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:43.757Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:44.003Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:44.003Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:44.003Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:44.005Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:44.005Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:44.005Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:44.006Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:44.006Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:44.006Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:44.006Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:44.006Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.007Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:44.007Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:44.253Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:44.253Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:44.253Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:44.255Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:44.255Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:44.255Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:44.256Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:44.256Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:44.256Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:44.256Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.257Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:44.257Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:44.503Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:44.503Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:44.503Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:44.505Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:44.505Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:44.505Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:44.506Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:44.506Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:44.506Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:44.506Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:44.506Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.507Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:44.507Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:44.753Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:44.753Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:44.753Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:44.755Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:44.755Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:44.755Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:44.756Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:44.756Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:44.756Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:44.756Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.757Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:44.757Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:45.003Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:45.003Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:45.003Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:45.005Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:45.005Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:45.005Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:45.006Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:45.006Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:45.006Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:45.006Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.007Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:45.007Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:45.253Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:45.253Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:45.253Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:45.255Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:45.255Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:45.255Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:45.256Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:45.256Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:45.256Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:45.256Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.257Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:45.257Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:45.503Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:45.503Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:45.503Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:45.505Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:45.505Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:45.505Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:45.506Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:45.506Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:45.506Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:45.506Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:45.506Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.507Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:45.507Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:45.752Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:45.752Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:45.752Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:45.754Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:45.754Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:45.754Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:45.755Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:45.756Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:45.756Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:45.756Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:45.756Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:45.756Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:45.756Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:45.756Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.756Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.756Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:45.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:45.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:45.757Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:45.757Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:46.003Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:46.003Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:46.003Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:46.005Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:46.005Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:46.005Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:46.006Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:46.007Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:46.007Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:46.007Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:46.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:46.007Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:46.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:46.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.007Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:46.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:46.008Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.008Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.008Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:46.008Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:46.253Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:46.253Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:46.253Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:46.254Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:46.255Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:46.255Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:46.256Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:46.256Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:46.256Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:46.256Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:46.256Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.257Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:46.257Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:46.503Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:46.503Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:46.503Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:46.504Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:46.505Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:46.505Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:46.506Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:46.506Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:46.506Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:46.506Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.507Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:46.507Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:46.752Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:46.752Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:46.752Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:46.754Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:46.754Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:46.754Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:46.755Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:46.756Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:46.756Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:46.756Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:46.756Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:46.756Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:46.756Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:46.756Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.756Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.756Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:46.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:46.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:46.757Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:46.757Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:47.002Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:47.003Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:47.003Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:47.004Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:47.004Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:47.004Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:47.006Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:47.006Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:47.006Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:47.006Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:47.006Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:47.006Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:47.006Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:47.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.007Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:47.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:47.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.007Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:47.007Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:47.253Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:47.253Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:47.253Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:47.254Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:47.255Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:47.255Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:47.256Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:47.256Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:47.256Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:47.256Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:47.256Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.257Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:47.257Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:47.503Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:47.503Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:47.503Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:47.505Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:47.505Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:47.505Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:47.506Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:47.506Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:47.506Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:47.506Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.507Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:47.508Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:47.753Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:47.753Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:47.753Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:47.754Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:47.754Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:47.754Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:47.756Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:47.756Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:47.756Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:47.756Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:47.756Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:47.756Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:47.756Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:47.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.757Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:47.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:47.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:47.757Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:47.757Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:48.003Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:48.003Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:48.003Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:48.005Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:48.005Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:48.005Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:48.006Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:48.007Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:48.007Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:48.007Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:48.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:48.007Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:48.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:48.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:48.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:48.007Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:48.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:48.008Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:48.008Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:48.008Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:48.008Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:48.252Z level=WARN source=sched.go:511 msg="gpu VRAM usage didn't recover within timeout" seconds=5.004719725
ollama-rocm  | time=2024-06-13T17:40:48.252Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:48.252Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:48.252Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:48.253Z level=DEBUG source=sched.go:283 msg="sending an unloaded event" modelPath=/root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29
ollama-rocm  | time=2024-06-13T17:40:48.253Z level=DEBUG source=sched.go:206 msg="ignoring unload event with no pending requests"
ollama-rocm  | time=2024-06-13T17:40:48.254Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:48.254Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:48.254Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:48.255Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:48.256Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:48.256Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:48.256Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:48.256Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:48.256Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:48.256Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:48.256Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:48.256Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:48.256Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:48.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:48.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:48.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:48.257Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:48.257Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:48.502Z level=WARN source=sched.go:511 msg="gpu VRAM usage didn't recover within timeout" seconds=5.25481111
ollama-rocm  | time=2024-06-13T17:40:48.503Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs"
ollama-rocm  | time=2024-06-13T17:40:48.503Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so*
ollama-rocm  | time=2024-06-13T17:40:48.503Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]"
ollama-rocm  | time=2024-06-13T17:40:48.504Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[]
ollama-rocm  | time=2024-06-13T17:40:48.504Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so*
ollama-rocm  | time=2024-06-13T17:40:48.504Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]"
ollama-rocm  | time=2024-06-13T17:40:48.506Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0]
ollama-rocm  | cudaSetDevice err: 35
ollama-rocm  | time=2024-06-13T17:40:48.506Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing.  If you have a CUDA GPU please upgrade to run ollama"
ollama-rocm  | time=2024-06-13T17:40:48.506Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2"
ollama-rocm  | time=2024-06-13T17:40:48.506Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory"
ollama-rocm  | time=2024-06-13T17:40:48.506Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:48.506Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties"
ollama-rocm  | time=2024-06-13T17:40:48.506Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties"
ollama-rocm  | time=2024-06-13T17:40:48.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:48.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:48.507Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1]
ollama-rocm  | time=2024-06-13T17:40:48.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties"
ollama-rocm  | time=2024-06-13T17:40:48.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:48.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB"
ollama-rocm  | time=2024-06-13T17:40:48.507Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib"
ollama-rocm  | time=2024-06-13T17:40:48.507Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0
ollama-rocm  | time=2024-06-13T17:40:48.753Z level=WARN source=sched.go:511 msg="gpu VRAM usage didn't recover within timeout" seconds=5.505507595
<!-- gh-comment-id:2166435491 --> @ic4-y commented on GitHub (Jun 13, 2024): I do believe that both GPUs get detected correctly even if I do NOT drop that variable, and the first one gets filtered out correctly. At least they both show up in the log with their respective amount of VRAM. Let me post the entire log WITH `ROCR_VISIBLE_DEVICES` being set. It is a bit of a long one because on ollama 0.1.43 it seems to be stuck in a loop trying to restart the server multiple times after the error message referring to Tensile. ``` Attaching to ollama-rocm ollama-rocm | 2024/06/13 17:39:53 routes.go:1011: INFO server config env="map[OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_HOST: OLLAMA_KEEP_ALIVE: OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_MODELS: OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_RUNNERS_DIR: OLLAMA_TMPDIR:]" ollama-rocm | time=2024-06-13T17:39:53.285Z level=INFO source=images.go:740 msg="total blobs: 34" ollama-rocm | time=2024-06-13T17:39:53.286Z level=INFO source=images.go:747 msg="total unused blobs removed: 0" ollama-rocm | time=2024-06-13T17:39:53.286Z level=INFO source=routes.go:1057 msg="Listening on [::]:11434 (version 0.1.43)" ollama-rocm | time=2024-06-13T17:39:53.287Z level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama3388558046/runners ollama-rocm | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=cpu file=build/linux/x86_64/cpu/bin/ollama_llama_server.gz ollama-rocm | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=cpu_avx file=build/linux/x86_64/cpu_avx/bin/ollama_llama_server.gz ollama-rocm | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=cpu_avx2 file=build/linux/x86_64/cpu_avx2/bin/ollama_llama_server.gz ollama-rocm | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcublas.so.11.gz ollama-rocm | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcublasLt.so.11.gz ollama-rocm | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/libcudart.so.11.0.gz ollama-rocm | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=cuda_v11 file=build/linux/x86_64/cuda_v11/bin/ollama_llama_server.gz ollama-rocm | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=rocm_v60002 file=build/linux/x86_64/rocm_v60002/bin/deps.txt.gz ollama-rocm | time=2024-06-13T17:39:53.287Z level=DEBUG source=payload.go:180 msg=extracting variant=rocm_v60002 file=build/linux/x86_64/rocm_v60002/bin/ollama_llama_server.gz ollama-rocm | time=2024-06-13T17:39:56.070Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu ollama-rocm | time=2024-06-13T17:39:56.070Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu_avx ollama-rocm | time=2024-06-13T17:39:56.070Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu_avx2 ollama-rocm | time=2024-06-13T17:39:56.070Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cuda_v11 ollama-rocm | time=2024-06-13T17:39:56.070Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/rocm_v60002 ollama-rocm | time=2024-06-13T17:39:56.070Z level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cuda_v11 rocm_v60002 cpu cpu_avx cpu_avx2]" ollama-rocm | time=2024-06-13T17:39:56.070Z level=DEBUG source=payload.go:45 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY" ollama-rocm | time=2024-06-13T17:39:56.070Z level=DEBUG source=sched.go:90 msg="starting llm scheduler" ollama-rocm | time=2024-06-13T17:39:56.070Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:39:56.070Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:39:56.070Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:39:56.072Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:39:56.072Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:39:56.072Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:39:56.073Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:39:56.073Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:39:56.073Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:39:56.073Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:39:56.074Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:39:56.074Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:39:56.074Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:39:56.075Z level=INFO source=types.go:71 msg="inference compute" id=1 library=rocm compute=gfx1032 driver=0.0 name=1002:73e3 total="8.0 GiB" available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:39.054Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:39.055Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:39.055Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:39.055Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:39.055Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:39.055Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:39.056Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:39.056Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:39.056Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:39.056Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:39.056Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:39.056Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:39.056Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:39.057Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:39.057Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:39.057Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:39.057Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:39.057Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:39.057Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:39.057Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:39.057Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:39.057Z level=DEBUG source=gguf.go:70 msg="model = &llm.gguf{containerGGUF:(*llm.containerGGUF)(0xc0002c7ac0), kv:llm.KV{}, tensors:[]*llm.Tensor(nil), parameters:0x0}" ollama-rocm | time=2024-06-13T17:40:40.900Z level=DEBUG source=sched.go:153 msg="loading first model" model=/root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29 ollama-rocm | time=2024-06-13T17:40:40.900Z level=DEBUG source=memory.go:44 msg=evaluating library=rocm gpu_count=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:40.901Z level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=33 memory.available="8.0 GiB" memory.required.full="5.0 GiB" memory.required.partial="5.0 GiB" memory.required.kv="256.0 MiB" memory.weights.total="4.1 GiB" memory.weights.repeating="3.7 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="164.0 MiB" memory.graph.partial="677.5 MiB" ollama-rocm | time=2024-06-13T17:40:40.901Z level=DEBUG source=sched.go:563 msg="new model will fit in available VRAM in single GPU, loading" model=/root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29 gpu=1 available=8573157376 required="5.0 GiB" ollama-rocm | time=2024-06-13T17:40:40.901Z level=DEBUG source=memory.go:44 msg=evaluating library=rocm gpu_count=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:40.901Z level=INFO source=memory.go:133 msg="offload to gpu" layers.requested=-1 layers.real=33 memory.available="8.0 GiB" memory.required.full="5.0 GiB" memory.required.partial="5.0 GiB" memory.required.kv="256.0 MiB" memory.weights.total="4.1 GiB" memory.weights.repeating="3.7 GiB" memory.weights.nonrepeating="411.0 MiB" memory.graph.full="164.0 MiB" memory.graph.partial="677.5 MiB" ollama-rocm | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu ollama-rocm | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu_avx ollama-rocm | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu_avx2 ollama-rocm | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cuda_v11 ollama-rocm | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/rocm_v60002 ollama-rocm | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu ollama-rocm | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu_avx ollama-rocm | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cpu_avx2 ollama-rocm | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/cuda_v11 ollama-rocm | time=2024-06-13T17:40:40.901Z level=DEBUG source=payload.go:71 msg="availableServers : found" file=/tmp/ollama3388558046/runners/rocm_v60002 ollama-rocm | time=2024-06-13T17:40:40.901Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:40.913Z level=INFO source=server.go:341 msg="starting llama server" cmd="/tmp/ollama3388558046/runners/rocm_v60002/ollama_llama_server --model /root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29 --ctx-size 2048 --batch-size 512 --embedding --log-disable --n-gpu-layers 33 --verbose --parallel 1 --port 32889" ollama-rocm | time=2024-06-13T17:40:40.913Z level=DEBUG source=server.go:356 msg=subprocess environment="[PATH=/opt/rh/devtoolset-7/root/usr/bin:/opt/rocm/hcc/bin:/opt/rocm/hip/bin:/opt/rocm/bin:/opt/rocm/hcc/bin::/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HSA_OVERRIDE_GFX_VERSION=10.3.0 LD_LIBRARY_PATH=/opt/rocm/lib:/tmp/ollama3388558046/runners/rocm_v60002:/opt/rocm/lib:/usr/local/lib:/opt/rh/devtoolset-7/root HIP_VISIBLE_DEVICES=1]" ollama-rocm | time=2024-06-13T17:40:40.914Z level=INFO source=sched.go:338 msg="loaded runners" count=1 ollama-rocm | time=2024-06-13T17:40:40.914Z level=INFO source=server.go:529 msg="waiting for llama runner to start responding" ollama-rocm | time=2024-06-13T17:40:40.914Z level=INFO source=server.go:567 msg="waiting for server to become available" status="llm server error" ollama-rocm | INFO [main] build info | build=1 commit="5921b8f" tid="140344978488384" timestamp=1718300441 ollama-rocm | INFO [main] system info | n_threads=12 n_threads_batch=-1 system_info="AVX = 1 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 0 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 0 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | " tid="140344978488384" timestamp=1718300441 total_threads=24 ollama-rocm | INFO [main] HTTP server listening | hostname="127.0.0.1" n_threads_http="23" port="32889" tid="140344978488384" timestamp=1718300441 ollama-rocm | llama_model_loader: loaded meta data with 21 key-value pairs and 291 tensors from /root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29 (version GGUF V3 (latest)) ollama-rocm | llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output. ollama-rocm | llama_model_loader: - kv 0: general.architecture str = llama ollama-rocm | llama_model_loader: - kv 1: general.name str = Meta-Llama-3-8B-Instruct ollama-rocm | llama_model_loader: - kv 2: llama.block_count u32 = 32 ollama-rocm | llama_model_loader: - kv 3: llama.context_length u32 = 8192 ollama-rocm | llama_model_loader: - kv 4: llama.embedding_length u32 = 4096 ollama-rocm | llama_model_loader: - kv 5: llama.feed_forward_length u32 = 14336 ollama-rocm | llama_model_loader: - kv 6: llama.attention.head_count u32 = 32 ollama-rocm | llama_model_loader: - kv 7: llama.attention.head_count_kv u32 = 8 ollama-rocm | llama_model_loader: - kv 8: llama.rope.freq_base f32 = 500000.000000 ollama-rocm | llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000010 ollama-rocm | llama_model_loader: - kv 10: general.file_type u32 = 2 ollama-rocm | llama_model_loader: - kv 11: llama.vocab_size u32 = 128256 ollama-rocm | llama_model_loader: - kv 12: llama.rope.dimension_count u32 = 128 ollama-rocm | llama_model_loader: - kv 13: tokenizer.ggml.model str = gpt2 ollama-rocm | time=2024-06-13T17:40:41.166Z level=INFO source=server.go:567 msg="waiting for server to become available" status="llm server loading model" ollama-rocm | llama_model_loader: - kv 14: tokenizer.ggml.tokens arr[str,128256] = ["!", "\"", "#", "$", "%", "&", "'", ... ollama-rocm | llama_model_loader: - kv 15: tokenizer.ggml.token_type arr[i32,128256] = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ... ollama-rocm | llama_model_loader: - kv 16: tokenizer.ggml.merges arr[str,280147] = ["Ġ Ġ", "Ġ ĠĠĠ", "ĠĠ ĠĠ", "... ollama-rocm | llama_model_loader: - kv 17: tokenizer.ggml.bos_token_id u32 = 128000 ollama-rocm | llama_model_loader: - kv 18: tokenizer.ggml.eos_token_id u32 = 128001 ollama-rocm | llama_model_loader: - kv 19: tokenizer.chat_template str = {% set loop_messages = messages %}{% ... ollama-rocm | llama_model_loader: - kv 20: general.quantization_version u32 = 2 ollama-rocm | llama_model_loader: - type f32: 65 tensors ollama-rocm | llama_model_loader: - type q4_0: 225 tensors ollama-rocm | llama_model_loader: - type q6_K: 1 tensors ollama-rocm | llm_load_vocab: missing or unrecognized pre-tokenizer type, using: 'default' ollama-rocm | llm_load_vocab: special tokens cache size = 256 ollama-rocm | llm_load_vocab: token to piece cache size = 1.5928 MB ollama-rocm | llm_load_print_meta: format = GGUF V3 (latest) ollama-rocm | llm_load_print_meta: arch = llama ollama-rocm | llm_load_print_meta: vocab type = BPE ollama-rocm | llm_load_print_meta: n_vocab = 128256 ollama-rocm | llm_load_print_meta: n_merges = 280147 ollama-rocm | llm_load_print_meta: n_ctx_train = 8192 ollama-rocm | llm_load_print_meta: n_embd = 4096 ollama-rocm | llm_load_print_meta: n_head = 32 ollama-rocm | llm_load_print_meta: n_head_kv = 8 ollama-rocm | llm_load_print_meta: n_layer = 32 ollama-rocm | llm_load_print_meta: n_rot = 128 ollama-rocm | llm_load_print_meta: n_embd_head_k = 128 ollama-rocm | llm_load_print_meta: n_embd_head_v = 128 ollama-rocm | llm_load_print_meta: n_gqa = 4 ollama-rocm | llm_load_print_meta: n_embd_k_gqa = 1024 ollama-rocm | llm_load_print_meta: n_embd_v_gqa = 1024 ollama-rocm | llm_load_print_meta: f_norm_eps = 0.0e+00 ollama-rocm | llm_load_print_meta: f_norm_rms_eps = 1.0e-05 ollama-rocm | llm_load_print_meta: f_clamp_kqv = 0.0e+00 ollama-rocm | llm_load_print_meta: f_max_alibi_bias = 0.0e+00 ollama-rocm | llm_load_print_meta: f_logit_scale = 0.0e+00 ollama-rocm | llm_load_print_meta: n_ff = 14336 ollama-rocm | llm_load_print_meta: n_expert = 0 ollama-rocm | llm_load_print_meta: n_expert_used = 0 ollama-rocm | llm_load_print_meta: causal attn = 1 ollama-rocm | llm_load_print_meta: pooling type = 0 ollama-rocm | llm_load_print_meta: rope type = 0 ollama-rocm | llm_load_print_meta: rope scaling = linear ollama-rocm | llm_load_print_meta: freq_base_train = 500000.0 ollama-rocm | llm_load_print_meta: freq_scale_train = 1 ollama-rocm | llm_load_print_meta: n_yarn_orig_ctx = 8192 ollama-rocm | llm_load_print_meta: rope_finetuned = unknown ollama-rocm | llm_load_print_meta: ssm_d_conv = 0 ollama-rocm | llm_load_print_meta: ssm_d_inner = 0 ollama-rocm | llm_load_print_meta: ssm_d_state = 0 ollama-rocm | llm_load_print_meta: ssm_dt_rank = 0 ollama-rocm | llm_load_print_meta: model type = 8B ollama-rocm | llm_load_print_meta: model ftype = Q4_0 ollama-rocm | llm_load_print_meta: model params = 8.03 B ollama-rocm | llm_load_print_meta: model size = 4.33 GiB (4.64 BPW) ollama-rocm | llm_load_print_meta: general.name = Meta-Llama-3-8B-Instruct ollama-rocm | llm_load_print_meta: BOS token = 128000 '<|begin_of_text|>' ollama-rocm | llm_load_print_meta: EOS token = 128001 '<|end_of_text|>' ollama-rocm | llm_load_print_meta: LF token = 128 'Ä' ollama-rocm | llm_load_print_meta: EOT token = 128009 '<|eot_id|>' ollama-rocm | ollama-rocm | rocBLAS error: Could not initialize Tensile host: No devices found ollama-rocm | time=2024-06-13T17:40:42.118Z level=INFO source=server.go:567 msg="waiting for server to become available" status="llm server not responding" ollama-rocm | time=2024-06-13T17:40:42.997Z level=INFO source=server.go:567 msg="waiting for server to become available" status="llm server error" ollama-rocm | time=2024-06-13T17:40:43.248Z level=ERROR source=sched.go:344 msg="error loading llama server" error="llama runner process has terminated: signal: aborted (core dumped) error:Could not initialize Tensile host: No devices found" ollama-rocm | time=2024-06-13T17:40:43.248Z level=DEBUG source=sched.go:347 msg="triggering expiration for failed load" model=/root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29 ollama-rocm | time=2024-06-13T17:40:43.248Z level=DEBUG source=sched.go:258 msg="runner expired event received" modelPath=/root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29 ollama-rocm | time=2024-06-13T17:40:43.248Z level=DEBUG source=sched.go:274 msg="got lock to unload" modelPath=/root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29 ollama-rocm | time=2024-06-13T17:40:43.248Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:43.248Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:43.248Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | [GIN] 2024/06/13 - 17:40:43 | 500 | 4.193868835s | 172.23.0.1 | POST "/api/chat" ollama-rocm | time=2024-06-13T17:40:43.249Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:43.249Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:43.249Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:43.251Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:43.251Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:43.251Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:43.251Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:43.251Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:43.251Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:43.251Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:43.252Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:43.252Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:43.252Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:43.252Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:43.252Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:43.252Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:43.252Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:43.252Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:43.252Z level=DEBUG source=server.go:990 msg="stopping llama server" ollama-rocm | time=2024-06-13T17:40:43.252Z level=DEBUG source=sched.go:279 msg="runner released" modelPath=/root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29 ollama-rocm | time=2024-06-13T17:40:43.503Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:43.503Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:43.503Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:43.504Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:43.504Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:43.504Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:43.506Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:43.506Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:43.506Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:43.506Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:43.507Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:43.507Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:43.507Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:43.753Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:43.753Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:43.753Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:43.755Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:43.755Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:43.755Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:43.756Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:43.756Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:43.756Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:43.756Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:43.757Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:43.757Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:43.757Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:44.003Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:44.003Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:44.003Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:44.005Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:44.005Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:44.005Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:44.006Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:44.006Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:44.006Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:44.006Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:44.006Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.007Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.007Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:44.007Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:44.253Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:44.253Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:44.253Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:44.255Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:44.255Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:44.255Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:44.256Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:44.256Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:44.256Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:44.256Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.257Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.257Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:44.257Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:44.503Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:44.503Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:44.503Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:44.505Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:44.505Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:44.505Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:44.506Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:44.506Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:44.506Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:44.506Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:44.506Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.507Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.507Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:44.507Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:44.753Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:44.753Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:44.753Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:44.755Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:44.755Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:44.755Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:44.756Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:44.756Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:44.756Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:44.756Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.757Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:44.757Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:44.757Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:45.003Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:45.003Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:45.003Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:45.005Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:45.005Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:45.005Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:45.006Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:45.006Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:45.006Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:45.006Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.007Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.007Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:45.007Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:45.253Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:45.253Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:45.253Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:45.255Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:45.255Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:45.255Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:45.256Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:45.256Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:45.256Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:45.256Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.257Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.257Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:45.257Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:45.503Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:45.503Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:45.503Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:45.505Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:45.505Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:45.505Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:45.506Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:45.506Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:45.506Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:45.506Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:45.506Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.507Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.507Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:45.507Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:45.752Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:45.752Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:45.752Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:45.754Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:45.754Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:45.754Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:45.755Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:45.756Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:45.756Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:45.756Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:45.756Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:45.756Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:45.756Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:45.756Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.756Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.756Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:45.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:45.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:45.757Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:45.757Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:46.003Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:46.003Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:46.003Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:46.005Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:46.005Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:46.005Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:46.006Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:46.007Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:46.007Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:46.007Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:46.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:46.007Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:46.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:46.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.007Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:46.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:46.008Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.008Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.008Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:46.008Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:46.253Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:46.253Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:46.253Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:46.254Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:46.255Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:46.255Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:46.256Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:46.256Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:46.256Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:46.256Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:46.256Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.257Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.257Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:46.257Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:46.503Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:46.503Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:46.503Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:46.504Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:46.505Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:46.505Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:46.506Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:46.506Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:46.506Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:46.506Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.507Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.507Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:46.507Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:46.752Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:46.752Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:46.752Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:46.754Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:46.754Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:46.754Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:46.755Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:46.756Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:46.756Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:46.756Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:46.756Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:46.756Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:46.756Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:46.756Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.756Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.756Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:46.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:46.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:46.757Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:46.757Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:47.002Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:47.003Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:47.003Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:47.004Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:47.004Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:47.004Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:47.006Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:47.006Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:47.006Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:47.006Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:47.006Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:47.006Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:47.006Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:47.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.007Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:47.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:47.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.007Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:47.007Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:47.253Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:47.253Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:47.253Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:47.254Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:47.255Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:47.255Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:47.256Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:47.256Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:47.256Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:47.256Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:47.256Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.257Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.257Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:47.257Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:47.503Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:47.503Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:47.503Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:47.505Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:47.505Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:47.505Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:47.506Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:47.506Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:47.506Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:47.506Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.507Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.507Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:47.508Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:47.753Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:47.753Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:47.753Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:47.754Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:47.754Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:47.754Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:47.756Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:47.756Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:47.756Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:47.756Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:47.756Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:47.756Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:47.756Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:47.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.757Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:47.757Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:47.757Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.757Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:47.757Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:47.757Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:48.003Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:48.003Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:48.003Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:48.005Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:48.005Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:48.005Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:48.006Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:48.007Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:48.007Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:48.007Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:48.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:48.007Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:48.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:48.007Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:48.007Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:48.007Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:48.007Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:48.008Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:48.008Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:48.008Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:48.008Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:48.252Z level=WARN source=sched.go:511 msg="gpu VRAM usage didn't recover within timeout" seconds=5.004719725 ollama-rocm | time=2024-06-13T17:40:48.252Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:48.252Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:48.252Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:48.253Z level=DEBUG source=sched.go:283 msg="sending an unloaded event" modelPath=/root/.ollama/models/blobs/sha256-00e1317cbf74d901080d7100f57580ba8dd8de57203072dc6f668324ba545f29 ollama-rocm | time=2024-06-13T17:40:48.253Z level=DEBUG source=sched.go:206 msg="ignoring unload event with no pending requests" ollama-rocm | time=2024-06-13T17:40:48.254Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:48.254Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:48.254Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:48.255Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:48.256Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:48.256Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:48.256Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:48.256Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:48.256Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:48.256Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:48.256Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:48.256Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:48.256Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:48.257Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:48.257Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:48.257Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:48.257Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:48.257Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:48.502Z level=WARN source=sched.go:511 msg="gpu VRAM usage didn't recover within timeout" seconds=5.25481111 ollama-rocm | time=2024-06-13T17:40:48.503Z level=DEBUG source=gpu.go:132 msg="Detecting GPUs" ollama-rocm | time=2024-06-13T17:40:48.503Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcuda.so* ollama-rocm | time=2024-06-13T17:40:48.503Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcuda.so** /usr/local/lib/libcuda.so** /opt/rh/devtoolset-7/root/libcuda.so** /usr/local/cuda*/targets/*/lib/libcuda.so* /usr/lib/*-linux-gnu/nvidia/current/libcuda.so* /usr/lib/*-linux-gnu/libcuda.so* /usr/lib/wsl/lib/libcuda.so* /usr/lib/wsl/drivers/*/libcuda.so* /opt/cuda/lib*/libcuda.so* /usr/local/cuda/lib*/libcuda.so* /usr/lib*/libcuda.so* /usr/local/lib*/libcuda.so*]" ollama-rocm | time=2024-06-13T17:40:48.504Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[] ollama-rocm | time=2024-06-13T17:40:48.504Z level=DEBUG source=gpu.go:274 msg="Searching for GPU library" name=libcudart.so* ollama-rocm | time=2024-06-13T17:40:48.504Z level=DEBUG source=gpu.go:293 msg="gpu library search" globs="[/opt/rocm/lib/libcudart.so** /usr/local/lib/libcudart.so** /opt/rh/devtoolset-7/root/libcudart.so** /tmp/ollama3388558046/runners/cuda*/libcudart.so* /usr/local/cuda/lib64/libcudart.so* /usr/lib/x86_64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/x86_64-linux-gnu/libcudart.so* /usr/lib/wsl/lib/libcudart.so* /usr/lib/wsl/drivers/*/libcudart.so* /opt/cuda/lib64/libcudart.so* /usr/local/cuda*/targets/aarch64-linux/lib/libcudart.so* /usr/lib/aarch64-linux-gnu/nvidia/current/libcudart.so* /usr/lib/aarch64-linux-gnu/libcudart.so* /usr/local/cuda/lib*/libcudart.so* /usr/lib*/libcudart.so* /usr/local/lib*/libcudart.so*]" ollama-rocm | time=2024-06-13T17:40:48.506Z level=DEBUG source=gpu.go:326 msg="discovered GPU libraries" paths=[/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0] ollama-rocm | cudaSetDevice err: 35 ollama-rocm | time=2024-06-13T17:40:48.506Z level=DEBUG source=gpu.go:338 msg="Unable to load cudart" library=/tmp/ollama3388558046/runners/cuda_v11/libcudart.so.11.0 error="your nvidia driver is too old or missing. If you have a CUDA GPU please upgrade to run ollama" ollama-rocm | time=2024-06-13T17:40:48.506Z level=DEBUG source=cpu_common.go:11 msg="CPU has AVX2" ollama-rocm | time=2024-06-13T17:40:48.506Z level=WARN source=amd_linux.go:48 msg="ollama recommends running the https://www.amd.com/en/support/linux-drivers" error="amdgpu version file missing: /sys/module/amdgpu/version stat /sys/module/amdgpu/version: no such file or directory" ollama-rocm | time=2024-06-13T17:40:48.506Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:48.506Z level=DEBUG source=amd_linux.go:102 msg="detected CPU /sys/class/kfd/kfd/topology/nodes/0/properties" ollama-rocm | time=2024-06-13T17:40:48.506Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/1/properties" ollama-rocm | time=2024-06-13T17:40:48.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=0 total="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:48.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=0 available="30.0 GiB" ollama-rocm | time=2024-06-13T17:40:48.507Z level=INFO source=amd_linux.go:268 msg="filtering out device per user request" id=0 visible_devices=[1] ollama-rocm | time=2024-06-13T17:40:48.507Z level=DEBUG source=amd_linux.go:77 msg="evaluating amdgpu node /sys/class/kfd/kfd/topology/nodes/2/properties" ollama-rocm | time=2024-06-13T17:40:48.507Z level=DEBUG source=amd_linux.go:242 msg="amdgpu memory" gpu=1 total="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:48.507Z level=DEBUG source=amd_linux.go:243 msg="amdgpu memory" gpu=1 available="8.0 GiB" ollama-rocm | time=2024-06-13T17:40:48.507Z level=DEBUG source=amd_common.go:16 msg="evaluating potential rocm lib dir /opt/rocm/lib" ollama-rocm | time=2024-06-13T17:40:48.507Z level=INFO source=amd_linux.go:304 msg="skipping rocm gfx compatibility check" HSA_OVERRIDE_GFX_VERSION=10.3.0 ollama-rocm | time=2024-06-13T17:40:48.753Z level=WARN source=sched.go:511 msg="gpu VRAM usage didn't recover within timeout" seconds=5.505507595 ```
GiteaMirror commented 2026-04-28 12:36:52 -05:00
Author
Owner
Copy Link

@ic4-y commented on GitHub (Jun 13, 2024):

I noticed that these loops which you can see the monumental log at the end also happen when I select the first GPU. There ,generation works at first, then at some point the GPU stops working with the "gpu VRAM usage didn't recover within timeout" error and seems to get stuck in a bad state and restarting ollama doesn't bring it back to a working state sadly... idk if that is the same bug or a different one. I'll investigate a bit more.

So yeah maybe that is another one we might have to investigate. But I guess they might both be related to the somewhat rare dual AMD setup I am running here.

<!-- gh-comment-id:2166615117 --> @ic4-y commented on GitHub (Jun 13, 2024): I noticed that these loops which you can see the monumental log at the end also happen when I select the first GPU. There ,generation works at first, then at some point the GPU stops working with the "gpu VRAM usage didn't recover within timeout" error and seems to get stuck in a bad state and restarting ollama doesn't bring it back to a working state sadly... idk if that is the same bug or a different one. I'll investigate a bit more. So yeah maybe that is another one we might have to investigate. But I guess they might both be related to the somewhat rare dual AMD setup I am running here.
GiteaMirror commented 2026-04-28 12:36:54 -05:00
Author
Owner
Copy Link

@ic4-y commented on GitHub (Jun 13, 2024):

I also just found you have an open PR relating to multi gpu support on AMD and gpu discovery (which to me is what these loops seem to be): https://github.com/ollama/ollama/pull/4517

Maybe related?

<!-- gh-comment-id:2166620280 --> @ic4-y commented on GitHub (Jun 13, 2024): I also just found you have an open PR relating to multi gpu support on AMD and gpu discovery (which to me is what these loops seem to be): https://github.com/ollama/ollama/pull/4517 Maybe related?
GiteaMirror commented 2026-04-28 12:36:59 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Jun 13, 2024):

It's possible that PR might resolve this. I'm hoping to get it merged soon, then build an RC with that.

<!-- gh-comment-id:2166800743 --> @dhiltgen commented on GitHub (Jun 13, 2024): It's possible that PR might resolve this. I'm hoping to get it merged soon, then build an RC with that.
GiteaMirror commented 2026-04-28 12:37:03 -05:00
Author
Owner
Copy Link

@ic4-y commented on GitHub (Jun 14, 2024):

Yes, it seems best to wait until this PR is done. I will check back in a week or two (maybe?) and revert back to 0.1.32-rocm on my dual GPU machine in the meantime.

If there is any testing I could help with on the coming RC when it comes to AMD, let me know.

<!-- gh-comment-id:2167707848 --> @ic4-y commented on GitHub (Jun 14, 2024): Yes, it seems best to wait until this PR is done. I will check back in a week or two (maybe?) and revert back to 0.1.32-rocm on my dual GPU machine in the meantime. If there is any testing I could help with on the coming RC when it comes to AMD, let me know.
GiteaMirror commented 2026-04-28 12:37:09 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Jun 18, 2024):

Please give the latest RC for 0.1.45 a try and see if the problem is resolved.

https://github.com/ollama/ollama/releases

<!-- gh-comment-id:2176876759 --> @dhiltgen commented on GitHub (Jun 18, 2024): Please give the latest RC for 0.1.45 a try and see if the problem is resolved. https://github.com/ollama/ollama/releases
GiteaMirror commented 2026-04-28 12:37:12 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Jul 3, 2024):

If the latest release doesn't clear up the GPU discovery, please share an updated log and I'll reopen the issue.

<!-- gh-comment-id:2207476693 --> @dhiltgen commented on GitHub (Jul 3, 2024): If the latest release doesn't clear up the GPU discovery, please share an updated log and I'll reopen the issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label amd bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#49674
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?