github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #6626] unable to load cuda driver library . symbol lookup for cuCtxCreate_v3 failed #29930

New Issue
Closed
opened 2026-04-22 09:17:05 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-04-22 09:17:05 -05:00
Owner
Copy Link

Originally created by @Wangzg97 on GitHub (Sep 4, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/6626

What is the issue?

i installed ollama by the sh file (i will show it at the end), but i found an error that ollama cannot use gpu, and i cannot find any methods to solve this problem

error message is

Sep 04 14:28:32 ip ollama[22329]: time=2024-09-04T14:28:32.433+08:00 level=INFO source=images.go:753 msg="total blobs: 0"
Sep 04 14:28:32 ip ollama[22329]: time=2024-09-04T14:28:32.433+08:00 level=INFO source=images.go:760 msg="total unused blobs removed: 0"
Sep 04 14:28:32 ip ollama[22329]: time=2024-09-04T14:28:32.433+08:00 level=INFO source=routes.go:1172 msg="Listening on 127.0.0.1:11434 (version 0.3.9)"
Sep 04 14:28:32 ip ollama[22329]: time=2024-09-04T14:28:32.433+08:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama629948305/runners
Sep 04 14:28:41 ip ollama[22329]: time=2024-09-04T14:28:41.834+08:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cuda_v12 rocm_v60102 cpu cpu_avx cpu_avx2 cuda_v11]"
Sep 04 14:28:41 ip ollama[22329]: time=2024-09-04T14:28:41.834+08:00 level=INFO source=gpu.go:200 msg="looking for compatible GPUs"
Sep 04 14:28:41 ip ollama[22329]: time=2024-09-04T14:28:41.837+08:00 level=INFO source=gpu.go:568 msg="unable to load cuda driver library" library=/usr/lib64/libcuda.so.460.106.00 error="symbol lookup for cuCtxCreate_v3 failed: /usr/lib64/libcuda.so.460.106.00: undefined symbol: cuCtxCreate_v3"
Sep 04 14:28:42 ip ollama[22329]: time=2024-09-04T14:28:42.023+08:00 level=INFO source=types.go:107 msg="inference compute" id=GPU-ba7826fa-xxxx-xxxx-xxxx-xxxxxxxxxxxx library=cuda variant=v11 compute=8.6 driver=0.0 name="" total="22.2 GiB" available="22.0 GiB"
Sep 04 14:30:59 ip ollama[22329]: [GIN] 2024/09/04 - 14:30:59 | 200 | 68.809µs | 127.0.0.1 | HEAD "/"
Sep 04 14:30:59 ip ollama[22329]: [GIN] 2024/09/04 - 14:30:59 | 200 | 207.344µs | 127.0.0.1 | GET "/api/tags"

i have installed driver and cuda

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.106.00 Driver Version: 460.106.00 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 A10 Off | 00000000:86:00.0 Off | 0 |
| 0% 27C P8 15W / 150W | 2MiB / 22731MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+

and cuda version is

$nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Mon_Nov_30_19:08:53_PST_2020
Cuda compilation tools, release 11.2, V11.2.67
Build cuda_11.2.r11.2/compiler.29373293_0

ollama_install.sh is below

#!/bin/sh
# This script installs Ollama on Linux.
# It detects the current operating system architecture and installs the appropriate version of Ollama.

set -eu

status() { echo ">>> $*" >&2; }
error() { echo "ERROR $*"; exit 1; }
warning() { echo "WARNING: $*"; }

TEMP_DIR=$(mktemp -d)
cleanup() { rm -rf $TEMP_DIR; }
trap cleanup EXIT

available() { command -v $1 >/dev/null; }
require() {
    local MISSING=''
    for TOOL in $*; do
        if ! available $TOOL; then
            MISSING="$MISSING $TOOL"
        fi
    done

    echo $MISSING
}

[ "$(uname -s)" = "Linux" ] || error 'This script is intended to run on Linux only.'

ARCH=$(uname -m)
case "$ARCH" in
    x86_64) ARCH="amd64" ;;
    aarch64|arm64) ARCH="arm64" ;;
    *) error "Unsupported architecture: $ARCH" ;;
esac

IS_WSL2=false

KERN=$(uname -r)
case "$KERN" in
    *icrosoft*WSL2 | *icrosoft*wsl2) IS_WSL2=true;;
    *icrosoft) error "Microsoft WSL1 is not currently supported. Please upgrade to WSL2 with 'wsl --set-version <distro> 2'" ;;
    *) ;;
esac

VER_PARAM="${OLLAMA_VERSION:+?version=$OLLAMA_VERSION}"

SUDO=
if [ "$(id -u)" -ne 0 ]; then
    # Running as root, no need for sudo
    if ! available sudo; then
        error "This script requires superuser permissions. Please re-run as root."
    fi

    SUDO="sudo"
fi

NEEDS=$(require curl awk grep sed tee xargs)
if [ -n "$NEEDS" ]; then
    status "ERROR: The following tools are required but missing:"
    for NEED in $NEEDS; do
        echo "  - $NEED"
    done
    exit 1
fi

for BINDIR in /usr/local/bin /usr/bin /bin; do
    echo $PATH | grep -q $BINDIR && break || continue
done
OLLAMA_INSTALL_DIR=$(dirname ${BINDIR})

status "Installing ollama to $OLLAMA_INSTALL_DIR"
$SUDO install -o0 -g0 -m755 -d $BINDIR
$SUDO install -o0 -g0 -m755 -d "$OLLAMA_INSTALL_DIR"
if curl -I --silent --fail --location "https://ollama.com/download/ollama-linux-${ARCH}.tgz${VER_PARAM}" >/dev/null ; then
    status "Downloading Linux ${ARCH} bundle"
    curl --fail --show-error --location --progress-bar \
        "https://ollama.com/download/ollama-linux-${ARCH}.tgz${VER_PARAM}" | \
        $SUDO tar -xzf - -C "$OLLAMA_INSTALL_DIR"
    BUNDLE=1
    if [ "$OLLAMA_INSTALL_DIR/bin/ollama" != "$BINDIR/ollama" ] ; then
        status "Making ollama accessible in the PATH in $BINDIR"
        $SUDO ln -sf "$OLLAMA_INSTALL_DIR/ollama" "$BINDIR/ollama"
    fi
else
    status "Downloading Linux ${ARCH} CLI"
    curl --fail --show-error --location --progress-bar -o "$TEMP_DIR/ollama"\
    "https://ollama.com/download/ollama-linux-${ARCH}${VER_PARAM}"
    $SUDO install -o0 -g0 -m755 $TEMP_DIR/ollama $OLLAMA_INSTALL_DIR/ollama
    BUNDLE=0
    if [ "$OLLAMA_INSTALL_DIR/ollama" != "$BINDIR/ollama" ] ; then
        status "Making ollama accessible in the PATH in $BINDIR"
        $SUDO ln -sf "$OLLAMA_INSTALL_DIR/ollama" "$BINDIR/ollama"
    fi
fi


install_success() {
    status 'The Ollama API is now available at 127.0.0.1:11434.'
    status 'Install complete. Run "ollama" from the command line.'
}
trap install_success EXIT

# Everything from this point onwards is optional.

configure_systemd() {
    status "Creating ollama systemd service..."
    cat <<EOF | $SUDO tee /etc/systemd/system/ollama.service >/dev/null
[Unit]
Description=Ollama Service
After=network-online.target

[Service]
ExecStart=$BINDIR/ollama serve
Restart=always
RestartSec=3
Environment="PATH=$PATH"

[Install]
WantedBy=default.target
EOF
    SYSTEMCTL_RUNNING="$(systemctl is-system-running || true)"
    case $SYSTEMCTL_RUNNING in
        running|degraded)
            status "Enabling and starting ollama service..."
            $SUDO systemctl daemon-reload
            $SUDO systemctl enable ollama

            start_service() { $SUDO systemctl restart ollama; }
            trap start_service EXIT
            ;;
    esac
}

if available systemctl; then
    configure_systemd
fi

# WSL2 only supports GPUs via nvidia passthrough
# so check for nvidia-smi to determine if GPU is available
if [ "$IS_WSL2" = true ]; then
    if available nvidia-smi && [ -n "$(nvidia-smi | grep -o "CUDA Version: [0-9]*\.[0-9]*")" ]; then
        status "Nvidia GPU detected."
    fi
    install_success
    exit 0
fi

# Install GPU dependencies on Linux
if ! available lspci && ! available lshw; then
    warning "Unable to detect NVIDIA/AMD GPU. Install lspci or lshw to automatically detect and install GPU dependencies."
    exit 0
fi

check_gpu() {
    # Look for devices based on vendor ID for NVIDIA and AMD
    case $1 in
        lspci)
            case $2 in
                nvidia) available lspci && lspci -d '10de:' | grep -q 'NVIDIA' || return 1 ;;
                amdgpu) available lspci && lspci -d '1002:' | grep -q 'AMD' || return 1 ;;
            esac ;;
        lshw)
            case $2 in
                nvidia) available lshw && $SUDO lshw -c display -numeric -disable network | grep -q 'vendor: .* \[10DE\]' || return 1 ;;
                amdgpu) available lshw && $SUDO lshw -c display -numeric -disable network | grep -q 'vendor: .* \[1002\]' || return 1 ;;
            esac ;;
        nvidia-smi) available nvidia-smi || return 1 ;;
    esac
}

if check_gpu nvidia-smi; then
    status "NVIDIA GPU installed."
    exit 0
fi

if ! check_gpu lspci nvidia && ! check_gpu lshw nvidia && ! check_gpu lspci amdgpu && ! check_gpu lshw amdgpu; then
    install_success
    warning "No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode."
    exit 0
fi

if check_gpu lspci amdgpu || check_gpu lshw amdgpu; then
    if [ $BUNDLE -ne 0 ]; then
        status "Downloading Linux ROCm ${ARCH} bundle"
        curl --fail --show-error --location --progress-bar \
            "https://ollama.com/download/ollama-linux-${ARCH}-rocm.tgz${VER_PARAM}" | \
            $SUDO tar -xzf - -C "$OLLAMA_INSTALL_DIR"

        install_success
        status "AMD GPU ready."
        exit 0
    fi
    # Look for pre-existing ROCm v6 before downloading the dependencies
    for search in "${HIP_PATH:-''}" "${ROCM_PATH:-''}" "/opt/rocm" "/usr/lib64"; do
        if [ -n "${search}" ] && [ -e "${search}/libhipblas.so.2" -o -e "${search}/lib/libhipblas.so.2" ]; then
            status "Compatible AMD GPU ROCm library detected at ${search}"
            install_success
            exit 0
        fi
    done

    status "Downloading AMD GPU dependencies..."
    $SUDO rm -rf /usr/share/ollama/lib
    $SUDO chmod o+x /usr/share/ollama
    $SUDO install -o ollama -g ollama -m 755 -d /usr/share/ollama/lib/rocm
    curl --fail --show-error --location --progress-bar "https://ollama.com/download/ollama-linux-amd64-rocm.tgz${VER_PARAM}" \
        | $SUDO tar zx --owner ollama --group ollama -C /usr/share/ollama/lib/rocm .
    install_success
    status "AMD GPU ready."
    exit 0
fi

CUDA_REPO_ERR_MSG="NVIDIA GPU detected, but your OS and Architecture are not supported by NVIDIA.  Please install the CUDA driver manually https://docs.nvidia.com/cuda/cuda-installation-guide-linux/"
# ref: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#rhel-7-centos-7
# ref: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#rhel-8-rocky-8
# ref: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#rhel-9-rocky-9
# ref: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#fedora
install_cuda_driver_yum() {
    status 'Installing NVIDIA repository...'
    
    case $PACKAGE_MANAGER in
        yum)
            $SUDO $PACKAGE_MANAGER -y install yum-utils
            if curl -I --silent --fail --location "https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m | sed -e 's/aarch64/sbsa/')/cuda-$1$2.repo" >/dev/null ; then
                $SUDO $PACKAGE_MANAGER-config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m | sed -e 's/aarch64/sbsa/')/cuda-$1$2.repo
            else
                error $CUDA_REPO_ERR_MSG
            fi
            ;;
        dnf)
            if curl -I --silent --fail --location "https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m | sed -e 's/aarch64/sbsa/')/cuda-$1$2.repo" >/dev/null ; then
                $SUDO $PACKAGE_MANAGER config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m | sed -e 's/aarch64/sbsa/')/cuda-$1$2.repo
            else
                error $CUDA_REPO_ERR_MSG
            fi
            ;;
    esac

    case $1 in
        rhel)
            status 'Installing EPEL repository...'
            # EPEL is required for third-party dependencies such as dkms and libvdpau
            $SUDO $PACKAGE_MANAGER -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-$2.noarch.rpm || true
            ;;
    esac

    status 'Installing CUDA driver...'

    if [ "$1" = 'centos' ] || [ "$1$2" = 'rhel7' ]; then
        $SUDO $PACKAGE_MANAGER -y install nvidia-driver-latest-dkms
    fi

    $SUDO $PACKAGE_MANAGER -y install cuda-drivers
}

# ref: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#ubuntu
# ref: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#debian
install_cuda_driver_apt() {
    status 'Installing NVIDIA repository...'
    if curl -I --silent --fail --location "https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m | sed -e 's/aarch64/sbsa/')/cuda-keyring_1.1-1_all.deb" >/dev/null ; then
        curl -fsSL -o $TEMP_DIR/cuda-keyring.deb https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m | sed -e 's/aarch64/sbsa/')/cuda-keyring_1.1-1_all.deb
    else
        error $CUDA_REPO_ERR_MSG
    fi

    case $1 in
        debian)
            status 'Enabling contrib sources...'
            $SUDO sed 's/main/contrib/' < /etc/apt/sources.list | $SUDO tee /etc/apt/sources.list.d/contrib.list > /dev/null
            if [ -f "/etc/apt/sources.list.d/debian.sources" ]; then
                $SUDO sed 's/main/contrib/' < /etc/apt/sources.list.d/debian.sources | $SUDO tee /etc/apt/sources.list.d/contrib.sources > /dev/null
            fi
            ;;
    esac

    status 'Installing CUDA driver...'
    $SUDO dpkg -i $TEMP_DIR/cuda-keyring.deb
    $SUDO apt-get update

    [ -n "$SUDO" ] && SUDO_E="$SUDO -E" || SUDO_E=
    DEBIAN_FRONTEND=noninteractive $SUDO_E apt-get -y install cuda-drivers -q
}

if [ ! -f "/etc/os-release" ]; then
    error "Unknown distribution. Skipping CUDA installation."
fi

. /etc/os-release

OS_NAME=$ID
OS_VERSION=$VERSION_ID

PACKAGE_MANAGER=
for PACKAGE_MANAGER in dnf yum apt-get; do
    if available $PACKAGE_MANAGER; then
        break
    fi
done

if [ -z "$PACKAGE_MANAGER" ]; then
    error "Unknown package manager. Skipping CUDA installation."
fi

if ! check_gpu nvidia-smi || [ -z "$(nvidia-smi | grep -o "CUDA Version: [0-9]*\.[0-9]*")" ]; then
    case $OS_NAME in
        centos|rhel) install_cuda_driver_yum 'rhel' $(echo $OS_VERSION | cut -d '.' -f 1) ;;
        rocky) install_cuda_driver_yum 'rhel' $(echo $OS_VERSION | cut -c1) ;;
        fedora) [ $OS_VERSION -lt '39' ] && install_cuda_driver_yum $OS_NAME $OS_VERSION || install_cuda_driver_yum $OS_NAME '39';;
        amzn) install_cuda_driver_yum 'fedora' '37' ;;
        debian) install_cuda_driver_apt $OS_NAME $OS_VERSION ;;
        ubuntu) install_cuda_driver_apt $OS_NAME $(echo $OS_VERSION | sed 's/\.//') ;;
        *) exit ;;
    esac
fi

if ! lsmod | grep -q nvidia || ! lsmod | grep -q nvidia_uvm; then
    KERNEL_RELEASE="$(uname -r)"
    case $OS_NAME in
        rocky) $SUDO $PACKAGE_MANAGER -y install kernel-devel kernel-headers ;;
        centos|rhel|amzn) $SUDO $PACKAGE_MANAGER -y install kernel-devel-$KERNEL_RELEASE kernel-headers-$KERNEL_RELEASE ;;
        fedora) $SUDO $PACKAGE_MANAGER -y install kernel-devel-$KERNEL_RELEASE ;;
        debian|ubuntu) $SUDO apt-get -y install linux-headers-$KERNEL_RELEASE ;;
        *) exit ;;
    esac

    NVIDIA_CUDA_VERSION=$($SUDO dkms status | awk -F: '/added/ { print $1 }')
    if [ -n "$NVIDIA_CUDA_VERSION" ]; then
        $SUDO dkms install $NVIDIA_CUDA_VERSION
    fi

    if lsmod | grep -q nouveau; then
        status 'Reboot to complete NVIDIA CUDA driver install.'
        exit 0
    fi

    $SUDO modprobe nvidia
    $SUDO modprobe nvidia_uvm
fi

# make sure the NVIDIA modules are loaded on boot with nvidia-persistenced
if command -v nvidia-persistenced > /dev/null 2>&1; then
    $SUDO touch /etc/modules-load.d/nvidia.conf
    MODULES="nvidia nvidia-uvm"
    for MODULE in $MODULES; do
        if ! grep -qxF "$MODULE" /etc/modules-load.d/nvidia.conf; then
            echo "$MODULE" | sudo tee -a /etc/modules-load.d/nvidia.conf > /dev/null
        fi
    done
fi

status "NVIDIA GPU ready."
install_success

run result of the sh file is

$sudo sh ollama_install.sh

Installing ollama to /usr/local
Downloading Linux amd64 bundle
######################################################################## 100.0%
Creating ollama systemd service...
Enabling and starting ollama service...
Created symlink from /etc/systemd/system/default.target.wants/ollama.service to /etc/systemd/system/ollama.service.
NVIDIA GPU installed.

OS

Linux

GPU

Nvidia

CPU

Intel

Ollama version

0.3.39

Originally created by @Wangzg97 on GitHub (Sep 4, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/6626 ### What is the issue? i installed ollama by the sh file (i will show it at the end), but i found an error that ollama cannot use gpu, and i cannot find any methods to solve this problem error message is > Sep 04 14:28:32 ip ollama[22329]: time=2024-09-04T14:28:32.433+08:00 level=INFO source=images.go:753 msg="total blobs: 0" > Sep 04 14:28:32 ip ollama[22329]: time=2024-09-04T14:28:32.433+08:00 level=INFO source=images.go:760 msg="total unused blobs removed: 0" > Sep 04 14:28:32 ip ollama[22329]: time=2024-09-04T14:28:32.433+08:00 level=INFO source=routes.go:1172 msg="Listening on 127.0.0.1:11434 (version 0.3.9)" > Sep 04 14:28:32 ip ollama[22329]: time=2024-09-04T14:28:32.433+08:00 level=INFO source=payload.go:30 msg="extracting embedded files" dir=/tmp/ollama629948305/runners > Sep 04 14:28:41 ip ollama[22329]: time=2024-09-04T14:28:41.834+08:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cuda_v12 rocm_v60102 cpu cpu_avx cpu_avx2 cuda_v11]" > Sep 04 14:28:41 ip ollama[22329]: time=2024-09-04T14:28:41.834+08:00 level=INFO source=gpu.go:200 msg="looking for compatible GPUs" > Sep 04 14:28:41 ip ollama[22329]: time=2024-09-04T14:28:41.837+08:00 level=INFO source=gpu.go:568 msg="unable to load cuda driver library" library=/usr/lib64/libcuda.so.460.106.00 error="symbol lookup for cuCtxCreate_v3 failed: /usr/lib64/libcuda.so.460.106.00: undefined symbol: cuCtxCreate_v3" > Sep 04 14:28:42 ip ollama[22329]: time=2024-09-04T14:28:42.023+08:00 level=INFO source=types.go:107 msg="inference compute" id=GPU-ba7826fa-xxxx-xxxx-xxxx-xxxxxxxxxxxx library=cuda variant=v11 compute=8.6 driver=0.0 name="" total="22.2 GiB" available="22.0 GiB" > Sep 04 14:30:59 ip ollama[22329]: [GIN] 2024/09/04 - 14:30:59 | 200 | 68.809µs | 127.0.0.1 | HEAD "/" > Sep 04 14:30:59 ip ollama[22329]: [GIN] 2024/09/04 - 14:30:59 | 200 | 207.344µs | 127.0.0.1 | GET "/api/tags" i have installed driver and cuda > +-----------------------------------------------------------------------------+ > | NVIDIA-SMI 460.106.00 Driver Version: 460.106.00 CUDA Version: 11.2 | > |-------------------------------+----------------------+----------------------+ > | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | > | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | > | | | MIG M. | > |===============================+======================+======================| > | 0 A10 Off | 00000000:86:00.0 Off | 0 | > | 0% 27C P8 15W / 150W | 2MiB / 22731MiB | 0% Default | > | | | N/A | > +-------------------------------+----------------------+----------------------+ > > +-----------------------------------------------------------------------------+ > | Processes: | > | GPU GI CI PID Type Process name GPU Memory | > | ID ID Usage | > |=============================================================================| > | No running processes found | > +-----------------------------------------------------------------------------+ and cuda version is > $nvcc -V > nvcc: NVIDIA (R) Cuda compiler driver > Copyright (c) 2005-2020 NVIDIA Corporation > Built on Mon_Nov_30_19:08:53_PST_2020 > Cuda compilation tools, release 11.2, V11.2.67 > Build cuda_11.2.r11.2/compiler.29373293_0 ollama_install.sh is below ``` #!/bin/sh # This script installs Ollama on Linux. # It detects the current operating system architecture and installs the appropriate version of Ollama. set -eu status() { echo ">>> $*" >&2; } error() { echo "ERROR $*"; exit 1; } warning() { echo "WARNING: $*"; } TEMP_DIR=$(mktemp -d) cleanup() { rm -rf $TEMP_DIR; } trap cleanup EXIT available() { command -v $1 >/dev/null; } require() { local MISSING='' for TOOL in $*; do if ! available $TOOL; then MISSING="$MISSING $TOOL" fi done echo $MISSING } [ "$(uname -s)" = "Linux" ] || error 'This script is intended to run on Linux only.' ARCH=$(uname -m) case "$ARCH" in x86_64) ARCH="amd64" ;; aarch64|arm64) ARCH="arm64" ;; *) error "Unsupported architecture: $ARCH" ;; esac IS_WSL2=false KERN=$(uname -r) case "$KERN" in *icrosoft*WSL2 | *icrosoft*wsl2) IS_WSL2=true;; *icrosoft) error "Microsoft WSL1 is not currently supported. Please upgrade to WSL2 with 'wsl --set-version <distro> 2'" ;; *) ;; esac VER_PARAM="${OLLAMA_VERSION:+?version=$OLLAMA_VERSION}" SUDO= if [ "$(id -u)" -ne 0 ]; then # Running as root, no need for sudo if ! available sudo; then error "This script requires superuser permissions. Please re-run as root." fi SUDO="sudo" fi NEEDS=$(require curl awk grep sed tee xargs) if [ -n "$NEEDS" ]; then status "ERROR: The following tools are required but missing:" for NEED in $NEEDS; do echo " - $NEED" done exit 1 fi for BINDIR in /usr/local/bin /usr/bin /bin; do echo $PATH | grep -q $BINDIR && break || continue done OLLAMA_INSTALL_DIR=$(dirname ${BINDIR}) status "Installing ollama to $OLLAMA_INSTALL_DIR" $SUDO install -o0 -g0 -m755 -d $BINDIR $SUDO install -o0 -g0 -m755 -d "$OLLAMA_INSTALL_DIR" if curl -I --silent --fail --location "https://ollama.com/download/ollama-linux-${ARCH}.tgz${VER_PARAM}" >/dev/null ; then status "Downloading Linux ${ARCH} bundle" curl --fail --show-error --location --progress-bar \ "https://ollama.com/download/ollama-linux-${ARCH}.tgz${VER_PARAM}" | \ $SUDO tar -xzf - -C "$OLLAMA_INSTALL_DIR" BUNDLE=1 if [ "$OLLAMA_INSTALL_DIR/bin/ollama" != "$BINDIR/ollama" ] ; then status "Making ollama accessible in the PATH in $BINDIR" $SUDO ln -sf "$OLLAMA_INSTALL_DIR/ollama" "$BINDIR/ollama" fi else status "Downloading Linux ${ARCH} CLI" curl --fail --show-error --location --progress-bar -o "$TEMP_DIR/ollama"\ "https://ollama.com/download/ollama-linux-${ARCH}${VER_PARAM}" $SUDO install -o0 -g0 -m755 $TEMP_DIR/ollama $OLLAMA_INSTALL_DIR/ollama BUNDLE=0 if [ "$OLLAMA_INSTALL_DIR/ollama" != "$BINDIR/ollama" ] ; then status "Making ollama accessible in the PATH in $BINDIR" $SUDO ln -sf "$OLLAMA_INSTALL_DIR/ollama" "$BINDIR/ollama" fi fi install_success() { status 'The Ollama API is now available at 127.0.0.1:11434.' status 'Install complete. Run "ollama" from the command line.' } trap install_success EXIT # Everything from this point onwards is optional. configure_systemd() { status "Creating ollama systemd service..." cat <<EOF | $SUDO tee /etc/systemd/system/ollama.service >/dev/null [Unit] Description=Ollama Service After=network-online.target [Service] ExecStart=$BINDIR/ollama serve Restart=always RestartSec=3 Environment="PATH=$PATH" [Install] WantedBy=default.target EOF SYSTEMCTL_RUNNING="$(systemctl is-system-running || true)" case $SYSTEMCTL_RUNNING in running|degraded) status "Enabling and starting ollama service..." $SUDO systemctl daemon-reload $SUDO systemctl enable ollama start_service() { $SUDO systemctl restart ollama; } trap start_service EXIT ;; esac } if available systemctl; then configure_systemd fi # WSL2 only supports GPUs via nvidia passthrough # so check for nvidia-smi to determine if GPU is available if [ "$IS_WSL2" = true ]; then if available nvidia-smi && [ -n "$(nvidia-smi | grep -o "CUDA Version: [0-9]*\.[0-9]*")" ]; then status "Nvidia GPU detected." fi install_success exit 0 fi # Install GPU dependencies on Linux if ! available lspci && ! available lshw; then warning "Unable to detect NVIDIA/AMD GPU. Install lspci or lshw to automatically detect and install GPU dependencies." exit 0 fi check_gpu() { # Look for devices based on vendor ID for NVIDIA and AMD case $1 in lspci) case $2 in nvidia) available lspci && lspci -d '10de:' | grep -q 'NVIDIA' || return 1 ;; amdgpu) available lspci && lspci -d '1002:' | grep -q 'AMD' || return 1 ;; esac ;; lshw) case $2 in nvidia) available lshw && $SUDO lshw -c display -numeric -disable network | grep -q 'vendor: .* \[10DE\]' || return 1 ;; amdgpu) available lshw && $SUDO lshw -c display -numeric -disable network | grep -q 'vendor: .* \[1002\]' || return 1 ;; esac ;; nvidia-smi) available nvidia-smi || return 1 ;; esac } if check_gpu nvidia-smi; then status "NVIDIA GPU installed." exit 0 fi if ! check_gpu lspci nvidia && ! check_gpu lshw nvidia && ! check_gpu lspci amdgpu && ! check_gpu lshw amdgpu; then install_success warning "No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode." exit 0 fi if check_gpu lspci amdgpu || check_gpu lshw amdgpu; then if [ $BUNDLE -ne 0 ]; then status "Downloading Linux ROCm ${ARCH} bundle" curl --fail --show-error --location --progress-bar \ "https://ollama.com/download/ollama-linux-${ARCH}-rocm.tgz${VER_PARAM}" | \ $SUDO tar -xzf - -C "$OLLAMA_INSTALL_DIR" install_success status "AMD GPU ready." exit 0 fi # Look for pre-existing ROCm v6 before downloading the dependencies for search in "${HIP_PATH:-''}" "${ROCM_PATH:-''}" "/opt/rocm" "/usr/lib64"; do if [ -n "${search}" ] && [ -e "${search}/libhipblas.so.2" -o -e "${search}/lib/libhipblas.so.2" ]; then status "Compatible AMD GPU ROCm library detected at ${search}" install_success exit 0 fi done status "Downloading AMD GPU dependencies..." $SUDO rm -rf /usr/share/ollama/lib $SUDO chmod o+x /usr/share/ollama $SUDO install -o ollama -g ollama -m 755 -d /usr/share/ollama/lib/rocm curl --fail --show-error --location --progress-bar "https://ollama.com/download/ollama-linux-amd64-rocm.tgz${VER_PARAM}" \ | $SUDO tar zx --owner ollama --group ollama -C /usr/share/ollama/lib/rocm . install_success status "AMD GPU ready." exit 0 fi CUDA_REPO_ERR_MSG="NVIDIA GPU detected, but your OS and Architecture are not supported by NVIDIA. Please install the CUDA driver manually https://docs.nvidia.com/cuda/cuda-installation-guide-linux/" # ref: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#rhel-7-centos-7 # ref: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#rhel-8-rocky-8 # ref: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#rhel-9-rocky-9 # ref: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#fedora install_cuda_driver_yum() { status 'Installing NVIDIA repository...' case $PACKAGE_MANAGER in yum) $SUDO $PACKAGE_MANAGER -y install yum-utils if curl -I --silent --fail --location "https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m | sed -e 's/aarch64/sbsa/')/cuda-$1$2.repo" >/dev/null ; then $SUDO $PACKAGE_MANAGER-config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m | sed -e 's/aarch64/sbsa/')/cuda-$1$2.repo else error $CUDA_REPO_ERR_MSG fi ;; dnf) if curl -I --silent --fail --location "https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m | sed -e 's/aarch64/sbsa/')/cuda-$1$2.repo" >/dev/null ; then $SUDO $PACKAGE_MANAGER config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m | sed -e 's/aarch64/sbsa/')/cuda-$1$2.repo else error $CUDA_REPO_ERR_MSG fi ;; esac case $1 in rhel) status 'Installing EPEL repository...' # EPEL is required for third-party dependencies such as dkms and libvdpau $SUDO $PACKAGE_MANAGER -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-$2.noarch.rpm || true ;; esac status 'Installing CUDA driver...' if [ "$1" = 'centos' ] || [ "$1$2" = 'rhel7' ]; then $SUDO $PACKAGE_MANAGER -y install nvidia-driver-latest-dkms fi $SUDO $PACKAGE_MANAGER -y install cuda-drivers } # ref: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#ubuntu # ref: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#debian install_cuda_driver_apt() { status 'Installing NVIDIA repository...' if curl -I --silent --fail --location "https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m | sed -e 's/aarch64/sbsa/')/cuda-keyring_1.1-1_all.deb" >/dev/null ; then curl -fsSL -o $TEMP_DIR/cuda-keyring.deb https://developer.download.nvidia.com/compute/cuda/repos/$1$2/$(uname -m | sed -e 's/aarch64/sbsa/')/cuda-keyring_1.1-1_all.deb else error $CUDA_REPO_ERR_MSG fi case $1 in debian) status 'Enabling contrib sources...' $SUDO sed 's/main/contrib/' < /etc/apt/sources.list | $SUDO tee /etc/apt/sources.list.d/contrib.list > /dev/null if [ -f "/etc/apt/sources.list.d/debian.sources" ]; then $SUDO sed 's/main/contrib/' < /etc/apt/sources.list.d/debian.sources | $SUDO tee /etc/apt/sources.list.d/contrib.sources > /dev/null fi ;; esac status 'Installing CUDA driver...' $SUDO dpkg -i $TEMP_DIR/cuda-keyring.deb $SUDO apt-get update [ -n "$SUDO" ] && SUDO_E="$SUDO -E" || SUDO_E= DEBIAN_FRONTEND=noninteractive $SUDO_E apt-get -y install cuda-drivers -q } if [ ! -f "/etc/os-release" ]; then error "Unknown distribution. Skipping CUDA installation." fi . /etc/os-release OS_NAME=$ID OS_VERSION=$VERSION_ID PACKAGE_MANAGER= for PACKAGE_MANAGER in dnf yum apt-get; do if available $PACKAGE_MANAGER; then break fi done if [ -z "$PACKAGE_MANAGER" ]; then error "Unknown package manager. Skipping CUDA installation." fi if ! check_gpu nvidia-smi || [ -z "$(nvidia-smi | grep -o "CUDA Version: [0-9]*\.[0-9]*")" ]; then case $OS_NAME in centos|rhel) install_cuda_driver_yum 'rhel' $(echo $OS_VERSION | cut -d '.' -f 1) ;; rocky) install_cuda_driver_yum 'rhel' $(echo $OS_VERSION | cut -c1) ;; fedora) [ $OS_VERSION -lt '39' ] && install_cuda_driver_yum $OS_NAME $OS_VERSION || install_cuda_driver_yum $OS_NAME '39';; amzn) install_cuda_driver_yum 'fedora' '37' ;; debian) install_cuda_driver_apt $OS_NAME $OS_VERSION ;; ubuntu) install_cuda_driver_apt $OS_NAME $(echo $OS_VERSION | sed 's/\.//') ;; *) exit ;; esac fi if ! lsmod | grep -q nvidia || ! lsmod | grep -q nvidia_uvm; then KERNEL_RELEASE="$(uname -r)" case $OS_NAME in rocky) $SUDO $PACKAGE_MANAGER -y install kernel-devel kernel-headers ;; centos|rhel|amzn) $SUDO $PACKAGE_MANAGER -y install kernel-devel-$KERNEL_RELEASE kernel-headers-$KERNEL_RELEASE ;; fedora) $SUDO $PACKAGE_MANAGER -y install kernel-devel-$KERNEL_RELEASE ;; debian|ubuntu) $SUDO apt-get -y install linux-headers-$KERNEL_RELEASE ;; *) exit ;; esac NVIDIA_CUDA_VERSION=$($SUDO dkms status | awk -F: '/added/ { print $1 }') if [ -n "$NVIDIA_CUDA_VERSION" ]; then $SUDO dkms install $NVIDIA_CUDA_VERSION fi if lsmod | grep -q nouveau; then status 'Reboot to complete NVIDIA CUDA driver install.' exit 0 fi $SUDO modprobe nvidia $SUDO modprobe nvidia_uvm fi # make sure the NVIDIA modules are loaded on boot with nvidia-persistenced if command -v nvidia-persistenced > /dev/null 2>&1; then $SUDO touch /etc/modules-load.d/nvidia.conf MODULES="nvidia nvidia-uvm" for MODULE in $MODULES; do if ! grep -qxF "$MODULE" /etc/modules-load.d/nvidia.conf; then echo "$MODULE" | sudo tee -a /etc/modules-load.d/nvidia.conf > /dev/null fi done fi status "NVIDIA GPU ready." install_success ``` run result of the sh file is > $sudo sh ollama_install.sh > >>> Installing ollama to /usr/local > >>> Downloading Linux amd64 bundle > ######################################################################## 100.0% > >>> Creating ollama systemd service... > >>> Enabling and starting ollama service... > Created symlink from /etc/systemd/system/default.target.wants/ollama.service to /etc/systemd/system/ollama.service. > >>> NVIDIA GPU installed. ### OS Linux ### GPU Nvidia ### CPU Intel ### Ollama version 0.3.39
GiteaMirror added the bug label 2026-04-22 09:17:05 -05:00
GiteaMirror commented 2026-04-22 09:17:07 -05:00
Author
Owner
Copy Link

@Wangzg97 commented on GitHub (Sep 4, 2024):

sorry i mistaked it, it used gpu successfully

<!-- gh-comment-id:2328072950 --> @Wangzg97 commented on GitHub (Sep 4, 2024): sorry i mistaked it, it used gpu successfully
GiteaMirror commented 2026-04-22 09:17:07 -05:00
Author
Owner
Copy Link

@XiaoMeimei commented on GitHub (Sep 25, 2024):

i have the same problem as you. How did you discover that there were no issues？

image

<!-- gh-comment-id:2373147997 --> @XiaoMeimei commented on GitHub (Sep 25, 2024): i have the same problem as you. How did you discover that there were no issues？ ![image](https://github.com/user-attachments/assets/687b937a-ce25-48ce-a33d-8ef852daa3e6)
GiteaMirror commented 2026-04-22 09:17:08 -05:00
Author
Owner
Copy Link

@wanyi312 commented on GitHub (Feb 19, 2025):

How was it solved?

<!-- gh-comment-id:2667370797 --> @wanyi312 commented on GitHub (Feb 19, 2025): How was it solved?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#29930
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?