github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-06 16:11:34 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #3550] ollama serve cannot detect GPU #27950

New Issue
Closed
opened 2026-04-22 05:36:58 -05:00 by GiteaMirror · 8 comments
GiteaMirror commented 2026-04-22 05:36:58 -05:00
Owner
Copy Link

Originally created by @g-makerr on GitHub (Apr 9, 2024).
Original GitHub issue: https://github.com/ollama/ollama/issues/3550

Originally assigned to: @dhiltgen on GitHub.

What is the issue?

I run "ollama serve", while it reports that "no GPU detected" and ""[cudart] error looking up CUDART GPU memory: cudart device memory info lookup failure 2"". But no such problems four days ago.
屏幕截图 2024-04-09 144347

What did you expect to see?

GPU detected & in use

Steps to reproduce

No response

Are there any recent changes that introduced the issue?

No response

OS

Linux

Architecture

amd64, x86

Platform

No response

Ollama version

0.1.30

GPU

Nvidia

GPU info

屏幕截图 2024-04-09 164724

CPU

No response

Other software

No response

Originally created by @g-makerr on GitHub (Apr 9, 2024). Original GitHub issue: https://github.com/ollama/ollama/issues/3550 Originally assigned to: @dhiltgen on GitHub. ### What is the issue? I run "ollama serve", while it reports that "no GPU detected" and ""[cudart] error looking up CUDART GPU memory: cudart device memory info lookup failure 2"". But no such problems four days ago. ![屏幕截图 2024-04-09 144347](https://github.com/ollama/ollama/assets/71173795/b9c853a7-e4be-42ec-b4b1-967b7bcdf219) ### What did you expect to see? GPU detected & in use ### Steps to reproduce _No response_ ### Are there any recent changes that introduced the issue? _No response_ ### OS Linux ### Architecture amd64, x86 ### Platform _No response_ ### Ollama version 0.1.30 ### GPU Nvidia ### GPU info ![屏幕截图 2024-04-09 164724](https://github.com/ollama/ollama/assets/71173795/fcb0ae91-bbd2-4de9-86d7-b1a45b103a47) ### CPU _No response_ ### Other software _No response_
GiteaMirror added the gpunvidiabug labels 2026-04-22 05:36:58 -05:00
GiteaMirror commented 2026-04-22 05:36:59 -05:00
Author
Owner
Copy Link

@pdevine commented on GitHub (Apr 12, 2024):

Hey @g-makerr What kind of card is it?

<!-- gh-comment-id:2052524237 --> @pdevine commented on GitHub (Apr 12, 2024): Hey @g-makerr What kind of card is it?
GiteaMirror commented 2026-04-22 05:36:59 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (Apr 12, 2024):

But no such problems four days ago.

Was this running an older version of Ollama? We've been adjusting our GPU discovery logic and recently switched from the nvidia management library to leverage the cuda runtime library. The error code "2" maps to cudaErrorMemoryAllocation which isn't an expected error code for querying available memory according to the API docs, so I'm not sure exactly what's going wrong yet.

Running with OLLAMA_DEBUG=1 might shed a little more light. Also check dmesg for any errors from the driver on the host. Is there anything else interesting/unusual about your configuration?

<!-- gh-comment-id:2052661685 --> @dhiltgen commented on GitHub (Apr 12, 2024): > But no such problems four days ago. Was this running an older version of Ollama? We've been adjusting our GPU discovery logic and recently switched from the nvidia management library to leverage the cuda runtime library. The error code "2" maps to cudaErrorMemoryAllocation which isn't an expected error code for querying available memory according to the API docs, so I'm not sure exactly what's going wrong yet. Running with `OLLAMA_DEBUG=1` might shed a little more light. Also check dmesg for any errors from the driver on the host. Is there anything else interesting/unusual about your configuration?
GiteaMirror commented 2026-04-22 05:37:00 -05:00
Author
Owner
Copy Link

@g-makerr commented on GitHub (Apr 13, 2024):

Hey @g-makerr What kind of card is it?

NVIDIA RTX 3090. I have fixed this issue by updating the GPU Driver version.

<!-- gh-comment-id:2053553388 --> @g-makerr commented on GitHub (Apr 13, 2024): > Hey @g-makerr What kind of card is it? NVIDIA RTX 3090. I have fixed this issue by updating the GPU Driver version.
GiteaMirror commented 2026-04-22 05:37:00 -05:00
Author
Owner
Copy Link

@g-makerr commented on GitHub (Apr 13, 2024):

But no such problems four days ago.

Was this running an older version of Ollama?

It was also running the same version-- ollama 0.1.30. And thanks for your suggestions. I have fixed this issue by updating the GPU Driver version.

<!-- gh-comment-id:2053554918 --> @g-makerr commented on GitHub (Apr 13, 2024): > > But no such problems four days ago. > > Was this running an older version of Ollama? It was also running the same version-- ollama 0.1.30. And thanks for your suggestions. I have fixed this issue by updating the GPU Driver version.
GiteaMirror commented 2026-04-22 05:37:00 -05:00
Author
Owner
Copy Link

@wjaxx commented on GitHub (Apr 21, 2024):

which version you update can solve this probrem?

<!-- gh-comment-id:2067907632 --> @wjaxx commented on GitHub (Apr 21, 2024): which version you update can solve this probrem?
GiteaMirror commented 2026-04-22 05:37:01 -05:00
Author
Owner
Copy Link

@g-makerr commented on GitHub (Apr 21, 2024):

which version you update can solve this probrem?

CUDA Driver Version from 510.8 to 550.67

<!-- gh-comment-id:2068045391 --> @g-makerr commented on GitHub (Apr 21, 2024): > which version you update can solve this probrem? CUDA Driver Version from 510.8 to 550.67
GiteaMirror commented 2026-04-22 05:37:01 -05:00
Author
Owner
Copy Link

@komasoftware commented on GitHub (Apr 30, 2024):

➜ ollama --version
ollama version is 0.1.32

➜ nvidia-smi -l 5
Tue Apr 30 17:19:13 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 545.23.08 Driver Version: 545.23.08 CUDA Version: 12.3 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce GTX 1070 Ti On | 00000000:01:00.0 On | N/A |
| 19% 53C P0 44W / 180W | 1690MiB / 8192MiB | 1% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+

Selection_957

<!-- gh-comment-id:2085636884 --> @komasoftware commented on GitHub (Apr 30, 2024): ➜ ollama --version ollama version is 0.1.32 ➜ nvidia-smi -l 5 Tue Apr 30 17:19:13 2024 +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 545.23.08 Driver Version: 545.23.08 CUDA Version: 12.3 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA GeForce GTX 1070 Ti On | 00000000:01:00.0 On | N/A | | 19% 53C P0 44W / 180W | 1690MiB / 8192MiB | 1% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ ![Selection_957](https://github.com/ollama/ollama/assets/482152/d4565ed1-f6d6-4adf-a7b9-8e82b70320e2)
GiteaMirror commented 2026-04-22 05:37:01 -05:00
Author
Owner
Copy Link

@dhiltgen commented on GitHub (May 1, 2024):

@komasoftware according to the nvidia API docs, that error is:

    /**  
     * This indicates that requested CUDA device is unavailable at the current
     * time. Devices are often unavailable due to use of
     * ::CU_COMPUTEMODE_EXCLUSIVE_PROCESS or ::CU_COMPUTEMODE_PROHIBITED.
     */
    CUDA_ERROR_DEVICE_UNAVAILABLE            = 46,

So my interpretation is either some other app has grabbed the GPU exclusively, or there's a permission problem in your setup preventing Ollama from accessing the GPU.

<!-- gh-comment-id:2088839644 --> @dhiltgen commented on GitHub (May 1, 2024): @komasoftware according to the nvidia API docs, that error is: ``` /** * This indicates that requested CUDA device is unavailable at the current * time. Devices are often unavailable due to use of * ::CU_COMPUTEMODE_EXCLUSIVE_PROCESS or ::CU_COMPUTEMODE_PROHIBITED. */ CUDA_ERROR_DEVICE_UNAVAILABLE = 46, ``` So my interpretation is either some other app has grabbed the GPU exclusively, or there's a permission problem in your setup preventing Ollama from accessing the GPU.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug gpu nvidia
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#27950
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?