github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-15 12:48:05 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #15964] Loosing GPU accelration with 0.23 #87855

New Issue
Closed
opened 2026-05-10 06:27:12 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-05-10 06:27:12 -05:00
Owner
Copy Link

Originally created by @davidcheeeese on GitHub (May 4, 2026).
Original GitHub issue: https://github.com/ollama/ollama/issues/15964

What is the issue?

With Ollama 0.23 I loose the GPU acceleration. Reverting to version 0.22.1 (ollama and ollama-cuda) make GPU acceleration working as nominal.

Relevant log output

With 0.23.0 version

level=INFO source=routes.go:1782 msg="server config" env="map[CUDA_VISIBLE_DEVICES:0,1 GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 >
level=INFO source=routes.go:1784 msg="Ollama cloud disabled: false"
level=INFO source=images.go:517 msg="total blobs: 91"
level=INFO source=images.go:524 msg="total unused blobs removed: 0"
level=INFO source=routes.go:1847 msg="Listening on [::]:11434 (version 0.23.0)"
level=INFO source=runner.go:67 msg="discovering available GPUs..."
level=WARN source=runner.go:536 msg="user overrode visible devices" CUDA_VISIBLE_DEVICES=0,1
level=WARN source=runner.go:540 msg="if GPUs are not correctly discovered, unset and try again"
level=INFO source=server.go:433 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 44985"
level=INFO source=types.go:60 msg="inference compute" id=cpu library=cpu compute="" name=cpu description=cpu libdirs=ollama driver="" pci_id="" type="" total="31.0 GiB" available="26.8 GiB"
level=INFO source=routes.go:1897 msg="vram-based default context" total_vram="0 B" default_num_ctx=4096
level=INFO source=model_recommendations.go:179 msg="model recommendations cache sleep scheduled" wait=4h47m21.797811923s consecutive_failures=0
level=WARN source=sched.go:423 msg="model architecture does not currently support parallel requests" architecture=qwen35moe
level=INFO source=server.go:259 msg="enabling flash attention"
level=INFO source=server.go:433 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --model /var/lib/ollama/blobs/sha256-7f6d862d0be0d79215b49404c9281587d8c8739dc5fbb8cfa3c5e6e033948636 --port 42577"
level=INFO source=sched.go:484 msg="system memory" total="31.0 GiB" free="26.8 GiB" free_swap="13.2 GiB"
level=INFO source=server.go:792 msg="loading model" "model layers"=41 requested=-1
level=INFO source=runner.go:1450 msg="starting ollama engine"
level=INFO source=runner.go:1485 msg="Server listening on 127.0.0.1:42577"

With 0.22.1

level=INFO source=routes.go:1782 msg="server config" env="map[CUDA_VISIBLE_DEVICES:0,1 GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 >
level=INFO source=routes.go:1784 msg="Ollama cloud disabled: false"
level=INFO source=images.go:517 msg="total blobs: 91"
level=INFO source=images.go:524 msg="total unused blobs removed: 0"
level=INFO source=routes.go:1847 msg="Listening on [::]:11434 (version 0.22.1)"
level=INFO source=runner.go:67 msg="discovering available GPUs..."
level=WARN source=runner.go:485 msg="user overrode visible devices" CUDA_VISIBLE_DEVICES=0,1
level=WARN source=runner.go:489 msg="if GPUs are not correctly discovered, unset and try again"
level=INFO source=server.go:444 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 43269"
level=INFO source=model_recommendations.go:179 msg="model recommendations cache sleep scheduled" wait=4h5m7.917668251s consecutive_failures=0
level=INFO source=server.go:444 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 34855"
level=INFO source=server.go:444 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 41393"
level=INFO source=types.go:42 msg="inference compute" id=GPU-aa085b66-2d3c-7be9-f1c1-8f38e444ce00 filter_id="" library=CUDA compute=12.0 name=CUDA0 description="NVIDIA GeForce RTX 5060 Ti" libdirs=ollama driver=13.2 pci_id=0000:01:00.0 ty>
level=INFO source=types.go:42 msg="inference compute" id=GPU-66028528-2f1f-4634-2951-a07f79fa9124 filter_id="" library=CUDA compute=12.0 name=CUDA1 description="NVIDIA GeForce RTX 5060 Ti" libdirs=ollama driver=13.2 pci_id=0000:05:00.0 ty>
level=INFO source=routes.go:1897 msg="vram-based default context" total_vram="31.9 GiB" default_num_ctx=32768
level=INFO source=server.go:444 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 34275"
level=WARN source=sched.go:423 msg="model architecture does not currently support parallel requests" architecture=qwen35moe
level=INFO source=server.go:259 msg="enabling flash attention"
level=INFO source=server.go:444 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --model /var/lib/ollama/blobs/sha256-7f6d862d0be0d79215b49404c9281587d8c8739dc5fbb8cfa3c5e6e033948636 --port 44629"
level=INFO source=sched.go:484 msg="system memory" total="31.0 GiB" free="26.5 GiB" free_swap="13.2 GiB"
level=INFO source=sched.go:491 msg="gpu memory" id=GPU-aa085b66-2d3c-7be9-f1c1-8f38e444ce00 library=CUDA available="15.0 GiB" free="15.5 GiB" minimum="457.0 MiB" overhead="0 B"
level=INFO source=sched.go:491 msg="gpu memory" id=GPU-66028528-2f1f-4634-2951-a07f79fa9124 library=CUDA available="14.9 GiB" free="15.3 GiB" minimum="457.0 MiB" overhead="0 B"
level=INFO source=server.go:771 msg="loading model" "model layers"=41 requested=-1
level=INFO source=runner.go:1417 msg="starting ollama engine"
level=INFO source=runner.go:1452 msg="Server listening on 127.0.0.1:44629"

OS

Linux

GPU

Nvidia

CPU

AMD

Ollama version

0.23.0

Originally created by @davidcheeeese on GitHub (May 4, 2026). Original GitHub issue: https://github.com/ollama/ollama/issues/15964 ### What is the issue? With Ollama 0.23 I loose the GPU acceleration. Reverting to version 0.22.1 (ollama and ollama-cuda) make GPU acceleration working as nominal. ### Relevant log output ```shell With 0.23.0 version level=INFO source=routes.go:1782 msg="server config" env="map[CUDA_VISIBLE_DEVICES:0,1 GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 > level=INFO source=routes.go:1784 msg="Ollama cloud disabled: false" level=INFO source=images.go:517 msg="total blobs: 91" level=INFO source=images.go:524 msg="total unused blobs removed: 0" level=INFO source=routes.go:1847 msg="Listening on [::]:11434 (version 0.23.0)" level=INFO source=runner.go:67 msg="discovering available GPUs..." level=WARN source=runner.go:536 msg="user overrode visible devices" CUDA_VISIBLE_DEVICES=0,1 level=WARN source=runner.go:540 msg="if GPUs are not correctly discovered, unset and try again" level=INFO source=server.go:433 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 44985" level=INFO source=types.go:60 msg="inference compute" id=cpu library=cpu compute="" name=cpu description=cpu libdirs=ollama driver="" pci_id="" type="" total="31.0 GiB" available="26.8 GiB" level=INFO source=routes.go:1897 msg="vram-based default context" total_vram="0 B" default_num_ctx=4096 level=INFO source=model_recommendations.go:179 msg="model recommendations cache sleep scheduled" wait=4h47m21.797811923s consecutive_failures=0 level=WARN source=sched.go:423 msg="model architecture does not currently support parallel requests" architecture=qwen35moe level=INFO source=server.go:259 msg="enabling flash attention" level=INFO source=server.go:433 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --model /var/lib/ollama/blobs/sha256-7f6d862d0be0d79215b49404c9281587d8c8739dc5fbb8cfa3c5e6e033948636 --port 42577" level=INFO source=sched.go:484 msg="system memory" total="31.0 GiB" free="26.8 GiB" free_swap="13.2 GiB" level=INFO source=server.go:792 msg="loading model" "model layers"=41 requested=-1 level=INFO source=runner.go:1450 msg="starting ollama engine" level=INFO source=runner.go:1485 msg="Server listening on 127.0.0.1:42577" With 0.22.1 level=INFO source=routes.go:1782 msg="server config" env="map[CUDA_VISIBLE_DEVICES:0,1 GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 > level=INFO source=routes.go:1784 msg="Ollama cloud disabled: false" level=INFO source=images.go:517 msg="total blobs: 91" level=INFO source=images.go:524 msg="total unused blobs removed: 0" level=INFO source=routes.go:1847 msg="Listening on [::]:11434 (version 0.22.1)" level=INFO source=runner.go:67 msg="discovering available GPUs..." level=WARN source=runner.go:485 msg="user overrode visible devices" CUDA_VISIBLE_DEVICES=0,1 level=WARN source=runner.go:489 msg="if GPUs are not correctly discovered, unset and try again" level=INFO source=server.go:444 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 43269" level=INFO source=model_recommendations.go:179 msg="model recommendations cache sleep scheduled" wait=4h5m7.917668251s consecutive_failures=0 level=INFO source=server.go:444 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 34855" level=INFO source=server.go:444 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 41393" level=INFO source=types.go:42 msg="inference compute" id=GPU-aa085b66-2d3c-7be9-f1c1-8f38e444ce00 filter_id="" library=CUDA compute=12.0 name=CUDA0 description="NVIDIA GeForce RTX 5060 Ti" libdirs=ollama driver=13.2 pci_id=0000:01:00.0 ty> level=INFO source=types.go:42 msg="inference compute" id=GPU-66028528-2f1f-4634-2951-a07f79fa9124 filter_id="" library=CUDA compute=12.0 name=CUDA1 description="NVIDIA GeForce RTX 5060 Ti" libdirs=ollama driver=13.2 pci_id=0000:05:00.0 ty> level=INFO source=routes.go:1897 msg="vram-based default context" total_vram="31.9 GiB" default_num_ctx=32768 level=INFO source=server.go:444 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --port 34275" level=WARN source=sched.go:423 msg="model architecture does not currently support parallel requests" architecture=qwen35moe level=INFO source=server.go:259 msg="enabling flash attention" level=INFO source=server.go:444 msg="starting runner" cmd="/usr/bin/ollama runner --ollama-engine --model /var/lib/ollama/blobs/sha256-7f6d862d0be0d79215b49404c9281587d8c8739dc5fbb8cfa3c5e6e033948636 --port 44629" level=INFO source=sched.go:484 msg="system memory" total="31.0 GiB" free="26.5 GiB" free_swap="13.2 GiB" level=INFO source=sched.go:491 msg="gpu memory" id=GPU-aa085b66-2d3c-7be9-f1c1-8f38e444ce00 library=CUDA available="15.0 GiB" free="15.5 GiB" minimum="457.0 MiB" overhead="0 B" level=INFO source=sched.go:491 msg="gpu memory" id=GPU-66028528-2f1f-4634-2951-a07f79fa9124 library=CUDA available="14.9 GiB" free="15.3 GiB" minimum="457.0 MiB" overhead="0 B" level=INFO source=server.go:771 msg="loading model" "model layers"=41 requested=-1 level=INFO source=runner.go:1417 msg="starting ollama engine" level=INFO source=runner.go:1452 msg="Server listening on 127.0.0.1:44629" ``` ### OS Linux ### GPU Nvidia ### CPU AMD ### Ollama version 0.23.0
GiteaMirror added the bug label 2026-05-10 06:27:12 -05:00
GiteaMirror commented 2026-05-10 06:27:14 -05:00
Author
Owner
Copy Link

@egitdirector commented on GitHub (May 4, 2026):

confirm same behavior on 2 linux machines

<!-- gh-comment-id:4372151261 --> @egitdirector commented on GitHub (May 4, 2026): confirm same behavior on 2 linux machines
GiteaMirror commented 2026-05-10 06:27:14 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 4, 2026):

Arch? ollama-cuda-0.23.0-1 doesn't contain any files.

<!-- gh-comment-id:4372233074 --> @rick-github commented on GitHub (May 4, 2026): Arch? ollama-cuda-0.23.0-1 doesn't contain any files.
GiteaMirror commented 2026-05-10 06:27:15 -05:00
Author
Owner
Copy Link

@davidcheeeese commented on GitHub (May 4, 2026):

Yes @rick-github, Archlinux, You're right, seems there is a problem with the ollama-cuda package.

<!-- gh-comment-id:4372571128 --> @davidcheeeese commented on GitHub (May 4, 2026): Yes @rick-github, Archlinux, You're right, seems there is a problem with the ollama-cuda package.
GiteaMirror commented 2026-05-10 06:27:16 -05:00
Author
Owner
Copy Link

@FloydianSound commented on GitHub (May 4, 2026):

Potential Root Cause: OLLAMA_LIBRARY_PATH being overridden in v0.23.0

I've been debugging this on CachyOS (Arch-based) and might have found why the GPU detection is failing even when environment variables are set.

It appears that in v0.23.0, the main process might be overriding OLLAMA_LIBRARY_PATH when it spawns the runner subprocess.

What I noticed in the logs (OLLAMA_DEBUG=1):
Even with OLLAMA_LIBRARY_PATH exported to the correct directory in the shell, the logs show the subprocess being started with a different path:

level=INFO source=server.go:433 msg="starting runner" ...
level=DEBUG source=server.go:434 msg=subprocess OLLAMA_LIBRARY_PATH=/usr/lib/ollama ...
level=DEBUG source=runner.go:433 msg="bootstrap discovery took" ... OLLAMA_LIBRARY_PATH=[/usr/lib/ollama]

It seems the runner is looking specifically in /usr/lib/ollama. If your libraries are in a different location (like /usr/local/lib/ollama/cuda_v13 which seems to be the case in some recent Arch/CachyOS updates), detection fails.

Temporary Workaround:
I managed to get my GPU (RTX 3090 Ti) detected again by symlinking the libraries into the path indicated by the logs:

sudo mkdir -p /usr/lib/ollama
sudo ln -sf /usr/local/lib/ollama/cuda_v13/libggml-cuda.so /usr/lib/ollama/libggml-cuda.so
sudo ln -sf /usr/local/lib/ollama/cuda_v13/libcublas.so.13 /usr/lib/ollama/libcublas.so.13
sudo ln -sf /usr/local/lib/ollama/cuda_v13/libcublasLt.so.13 /usr/lib/ollama/libcublasLt.so.13
sudo ln -sf /usr/local/lib/ollama/cuda_v13/libcudart.so.13 /usr/lib/ollama/libcudart.so.13

This allowed v0.23.0 to find the GPU correctly. Hopefully this helps others on Arch/CachyOS while the official package or discovery logic is updated!

<!-- gh-comment-id:4372571184 --> @FloydianSound commented on GitHub (May 4, 2026): ### Potential Root Cause: OLLAMA_LIBRARY_PATH being overridden in v0.23.0 I've been debugging this on CachyOS (Arch-based) and might have found why the GPU detection is failing even when environment variables are set. It appears that in v0.23.0, the main process might be overriding `OLLAMA_LIBRARY_PATH` when it spawns the `runner` subprocess. **What I noticed in the logs (`OLLAMA_DEBUG=1`):** Even with `OLLAMA_LIBRARY_PATH` exported to the correct directory in the shell, the logs show the subprocess being started with a different path: ```text level=INFO source=server.go:433 msg="starting runner" ... level=DEBUG source=server.go:434 msg=subprocess OLLAMA_LIBRARY_PATH=/usr/lib/ollama ... level=DEBUG source=runner.go:433 msg="bootstrap discovery took" ... OLLAMA_LIBRARY_PATH=[/usr/lib/ollama] ``` It seems the runner is looking specifically in `/usr/lib/ollama`. If your libraries are in a different location (like `/usr/local/lib/ollama/cuda_v13` which seems to be the case in some recent Arch/CachyOS updates), detection fails. **Temporary Workaround:** I managed to get my GPU (RTX 3090 Ti) detected again by symlinking the libraries into the path indicated by the logs: ```bash sudo mkdir -p /usr/lib/ollama sudo ln -sf /usr/local/lib/ollama/cuda_v13/libggml-cuda.so /usr/lib/ollama/libggml-cuda.so sudo ln -sf /usr/local/lib/ollama/cuda_v13/libcublas.so.13 /usr/lib/ollama/libcublas.so.13 sudo ln -sf /usr/local/lib/ollama/cuda_v13/libcublasLt.so.13 /usr/lib/ollama/libcublasLt.so.13 sudo ln -sf /usr/local/lib/ollama/cuda_v13/libcudart.so.13 /usr/lib/ollama/libcudart.so.13 ``` This allowed v0.23.0 to find the GPU correctly. Hopefully this helps others on Arch/CachyOS while the official package or discovery logic is updated!
GiteaMirror commented 2026-05-10 06:27:21 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (May 4, 2026):

https://gitlab.archlinux.org/archlinux/packaging/packages/ollama/-/work_items/77

<!-- gh-comment-id:4373211293 --> @rick-github commented on GitHub (May 4, 2026): https://gitlab.archlinux.org/archlinux/packaging/packages/ollama/-/work_items/77
GiteaMirror commented 2026-05-10 06:27:23 -05:00
Author
Owner
Copy Link

@FloydianSound commented on GitHub (May 4, 2026):

Update: Symlink workaround ineffective, Downgrade required

After further verification, the symlink workaround I proposed earlier did not fully restore functionality. While it allowed the server to start, the discovery phase still failed to properly initialize the GPU stack in some scenarios.

Confirmed Fix:
Downgrading both ollama and ollama-cuda to v0.22.1 resolved the issue immediately. On Arch/CachyOS, this restored native GPU detection and full offloading (31/31 layers on an RTX 3090 Ti) without any manual path overrides or environment tweaks.

If you are on Arch/CachyOS and have the old versions in your cache, you can downgrade using:

sudo pacman -U /var/cache/pacman/pkg/ollama-0.22.1-*.pkg.tar.zst /var/cache/pacman/pkg/ollama-cuda-0.22.1-*.pkg.tar.zst

EDIT: oh, i just saw your comment. i will wait for next version then :)

<!-- gh-comment-id:4373229893 --> @FloydianSound commented on GitHub (May 4, 2026): ### Update: Symlink workaround ineffective, Downgrade required After further verification, the symlink workaround I proposed earlier **did not** fully restore functionality. While it allowed the server to start, the discovery phase still failed to properly initialize the GPU stack in some scenarios. **Confirmed Fix:** Downgrading both `ollama` and `ollama-cuda` to **v0.22.1** resolved the issue immediately. On Arch/CachyOS, this restored native GPU detection and full offloading (31/31 layers on an RTX 3090 Ti) without any manual path overrides or environment tweaks. If you are on Arch/CachyOS and have the old versions in your cache, you can downgrade using: ```bash sudo pacman -U /var/cache/pacman/pkg/ollama-0.22.1-*.pkg.tar.zst /var/cache/pacman/pkg/ollama-cuda-0.22.1-*.pkg.tar.zst ``` EDIT: oh, i just saw your comment. i will wait for next version then :)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#87855
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?