github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-07-16 13:19:40 -05:00
Code Issues 16.9k Packages Projects Releases 122 Wiki Activity

[GH-ISSUE #8653] Latest pre-built Ollama binaries (cuda 12.x) do not come with "oob" support for 5.x architecture #83292

New Issue
Open
opened 2026-05-09 17:44:42 -05:00 by GiteaMirror · 1 comment
GiteaMirror commented 2026-05-09 17:44:42 -05:00
Owner
Copy Link

Originally created by @RKouchoo on GitHub (Jan 29, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/8653

Originally assigned to: @mxyng on GitHub.

"oob" support for 5.x architecture is missing on prebuilt binaries

Hello,

I ended up needing some more power so I threw a spare Quadro M5000 into my AI rig only to find it was not being utilsed at all. I did the usual checks and the card has compute capability 5.2 (confirmed compatible in the support matrix here).

As initial troubleshooting steps (bouncing ideas from other issues posted here) I tried:

  • Manually passing through the UUID of all GPU's via CUDA_VISIBLE_DEVICES gvar. ollama would acknowledge this in the logs but would never use the card anway. There was no log message complaining about compute capability or mention of dropping the card.
  • Setting OLLAMA_SCHED_SPREAD gvar to true

I found that the ollama install script also grabbed cuda 11.x by default, but during the installation the GPU's I had installed were a pair of 20GB RTX4000 "Ada" generation + Aspeed AST2500 IPMI/VGA. I also had the 565 driver installed before setting everything else up, it quotes that its built with cuda 12.6, I don't quite understand why the installer would grab the 11.x toolkit.

During a dive through the repo to see what I could find, I noticed the make config file for cuda_v12 (here) does not include 5.0/5.2 support by default but it could. I also found that in the current release notes for cuda 12.8 - Maxwell, Pascal and Volta architectures will be "frozen" (depricated?) in future releases of the cuda toolkit here.

I managed to build everything fine with the cuda 12.8 toolkit and confirmed ollama works as I expected, here are the build flags I used:
make cuda_v12 CUDA_ARCHITECTURES="50;52;60;61;62;70;72;75;80;86;87;89;90;90a" -j 32

I know that there is still heaps of Maxwell (5.2) cards still floating around in systems, people on a budget will definitely try use them with the hype of the recent model releases as they are capable of running them locally to an extent. I believe either the docs need an update or binaries should be compiled with support built in until theres an official notice or documentation change to avoid confusion.

Apologies if I am wrong, I thought I would post this here before opening a pull request incase there was anything already in motion related to this.

Cheers,
RK

OS

Linux - Ubuntu 22.04

GPU

Nvidia - RTX A4000 "Ada" x2, RTX 4070 & Quadro M5000

CPU

2x AMD EPYC 7371

Ollama version

0.5.7 (latest install script)

Originally created by @RKouchoo on GitHub (Jan 29, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/8653 Originally assigned to: @mxyng on GitHub. ### "oob" support for 5.x architecture is missing on prebuilt binaries Hello, I ended up needing some more power so I threw a spare Quadro M5000 into my AI rig only to find it was not being utilsed at all. I did the usual checks and the card has compute capability 5.2 (confirmed compatible in the support matrix [here](https://github.com/ollama/ollama/blob/main/docs/gpu.md)). As initial troubleshooting steps (bouncing ideas from other issues posted here) I tried: - Manually passing through the UUID of all GPU's via `CUDA_VISIBLE_DEVICES` gvar. ollama would acknowledge this in the logs but would never use the card anway. There was no log message complaining about compute capability or mention of dropping the card. - Setting `OLLAMA_SCHED_SPREAD` gvar to `true` I found that the ollama install script also grabbed **cuda 11.x by default**, but during the installation the GPU's I had installed were a pair of 20GB RTX4000 "Ada" generation + Aspeed AST2500 IPMI/VGA. I also had the 565 driver installed before setting everything else up, it quotes that its built with cuda 12.6, I don't quite understand why the installer would grab the 11.x toolkit. During a dive through the repo to see what I could find, I noticed the make config file for cuda_v12 ([here](https://github.com/ollama/ollama/blob/main/make/Makefile.cuda_v12)) does not include 5.0/5.2 support by default but it could. I also found that in the current release notes for cuda 12.8 - Maxwell, Pascal and Volta architectures will be "frozen" (depricated?) in future releases of the cuda toolkit [here](https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#deprecated-architectures). I managed to build everything fine with the cuda 12.8 toolkit and confirmed ollama works as I expected, here are the build flags I used: `make cuda_v12 CUDA_ARCHITECTURES="50;52;60;61;62;70;72;75;80;86;87;89;90;90a" -j 32` I know that there is still heaps of Maxwell (5.2) cards still floating around in systems, people on a budget will definitely try use them with the hype of the recent model releases as they are capable of running them locally to an extent. I believe either the docs need an update or binaries should be compiled with support built in until theres an official notice or documentation change to avoid confusion. Apologies if I am wrong, I thought I would post this here before opening a pull request incase there was anything already in motion related to this. Cheers, RK ### OS Linux - Ubuntu 22.04 ### GPU Nvidia - RTX A4000 "Ada" x2, RTX 4070 & Quadro M5000 ### CPU 2x AMD EPYC 7371 ### Ollama version 0.5.7 (latest install script)
GiteaMirror added the bug label 2026-05-09 17:44:42 -05:00
GiteaMirror commented 2026-05-09 17:44:46 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Jan 29, 2025):

https://github.com/ollama/ollama/pull/8567

<!-- gh-comment-id:2621359145 --> @rick-github commented on GitHub (Jan 29, 2025): https://github.com/ollama/ollama/pull/8567
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#83292
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?