github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[GH-ISSUE #8892] Run failure on disabled GPU #5766

New Issue
Closed
opened 2026-04-12 17:05:48 -05:00 by GiteaMirror · 4 comments
GiteaMirror commented 2026-04-12 17:05:48 -05:00
Owner
Copy Link

Originally created by @ihor-sokoliuk on GitHub (Feb 6, 2025).
Original GitHub issue: https://github.com/ollama/ollama/issues/8892

What is the issue?

Here is the BAT file to run Ollama on my Windows machine:

@echo off
set CUDA_VISIBLE_DEVICES=""
set OLLAMA_ORIGINS="*"
set OLLAMA_HOST=0.0.0.0:11434
set OLLAMA_DEBUG=1
ollama serve
timeout /t 100 /nobreak

When I run the Ollama server with the flag CUDA_VISIBLE_DEVICES="" it starts but fails in a minute or two without any error message and with a random return code like 2147842401.

Please take a look at the attached server log.

If I run the same BAT without the CUDA_VISIBLE_DEVICES variable, it works as expected.

How can I run the Ollama server without a GPU (with CPU+RAM only) so that it will continue to work?

Relevant log output

2025/02/06 11:44:15 routes.go:1187: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:C:\\Users\\ihor\\.ollama\\models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[* http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES:]"
time=2025-02-06T11:44:15.892-06:00 level=INFO source=images.go:432 msg="total blobs: 7"
time=2025-02-06T11:44:15.892-06:00 level=INFO source=images.go:439 msg="total unused blobs removed: 0"
time=2025-02-06T11:44:15.893-06:00 level=INFO source=routes.go:1238 msg="Listening on [::]:11434 (version 0.5.7)"
time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=common.go:80 msg="runners located" dir=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners
time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cpu_avx\ollama_llama_server.exe
time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cpu_avx2\ollama_llama_server.exe
time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cuda_v11_avx\ollama_llama_server.exe
time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cuda_v12_avx\ollama_llama_server.exe
time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\rocm_avx\ollama_llama_server.exe
time=2025-02-06T11:44:15.893-06:00 level=INFO source=routes.go:1267 msg="Dynamic LLM libraries" runners="[cuda_v12_avx rocm_avx cpu cpu_avx cpu_avx2 cuda_v11_avx]"
time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=routes.go:1268 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY"
time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=sched.go:105 msg="starting llm scheduler"
time=2025-02-06T11:44:15.893-06:00 level=INFO source=gpu.go:226 msg="looking for compatible GPUs"
time=2025-02-06T11:44:15.893-06:00 level=INFO source=gpu_windows.go:167 msg=packages count=1
time=2025-02-06T11:44:15.893-06:00 level=INFO source=gpu_windows.go:183 msg="efficiency cores detected" maxEfficiencyClass=1
time=2025-02-06T11:44:15.893-06:00 level=INFO source=gpu_windows.go:214 msg="" package=0 cores=24 efficiency=16 threads=32
time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=gpu.go:99 msg="searching for GPU discovery libraries for NVIDIA"
time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=gpu.go:517 msg="Searching for GPU library" name=nvml.dll
time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=gpu.go:543 msg="gpu library search" globs="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvml.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvml.dll C:\\Program Files\\Python313\\Scripts\\nvml.dll C:\\Program Files\\Python313\\nvml.dll C:\\WINDOWS\\system32\\nvml.dll C:\\WINDOWS\\nvml.dll C:\\WINDOWS\\System32\\Wbem\\nvml.dll C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\nvml.dll C:\\WINDOWS\\System32\\OpenSSH\\nvml.dll C:\\Program Files\\Microsoft VS Code\\bin\\nvml.dll C:\\Program Files (x86)\\WinSCP\\nvml.dll C:\\Program Files\\Git\\cmd\\nvml.dll C:\\Program Files\\Docker\\Docker\\resources\\bin\\nvml.dll C:\\Program Files\\Mullvad VPN\\resources\\nvml.dll C:\\Program Files\\PowerShell\\7\\nvml.dll C:\\Users\\ihor\\AppData\\Local\\Microsoft\\WindowsApps\\nvml.dll D:\\Development\\BigBoyEnhancement\\nvml.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\nvml.dll c:\\Windows\\System32\\nvml.dll]"
time=2025-02-06T11:44:15.894-06:00 level=DEBUG source=gpu.go:576 msg="discovered GPU libraries" paths="[C:\\WINDOWS\\system32\\nvml.dll c:\\Windows\\System32\\nvml.dll]"
time=2025-02-06T11:44:15.903-06:00 level=DEBUG source=gpu.go:120 msg="nvidia-ml loaded" library=C:\WINDOWS\system32\nvml.dll
time=2025-02-06T11:44:15.903-06:00 level=DEBUG source=gpu.go:517 msg="Searching for GPU library" name=nvcuda.dll
time=2025-02-06T11:44:15.903-06:00 level=DEBUG source=gpu.go:543 msg="gpu library search" globs="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvcuda.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvcuda.dll C:\\Program Files\\Python313\\Scripts\\nvcuda.dll C:\\Program Files\\Python313\\nvcuda.dll C:\\WINDOWS\\system32\\nvcuda.dll C:\\WINDOWS\\nvcuda.dll C:\\WINDOWS\\System32\\Wbem\\nvcuda.dll C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\nvcuda.dll C:\\WINDOWS\\System32\\OpenSSH\\nvcuda.dll C:\\Program Files\\Microsoft VS Code\\bin\\nvcuda.dll C:\\Program Files (x86)\\WinSCP\\nvcuda.dll C:\\Program Files\\Git\\cmd\\nvcuda.dll C:\\Program Files\\Docker\\Docker\\resources\\bin\\nvcuda.dll C:\\Program Files\\Mullvad VPN\\resources\\nvcuda.dll C:\\Program Files\\PowerShell\\7\\nvcuda.dll C:\\Users\\ihor\\AppData\\Local\\Microsoft\\WindowsApps\\nvcuda.dll D:\\Development\\BigBoyEnhancement\\nvcuda.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\nvcuda.dll c:\\windows\\system*\\nvcuda.dll]"
time=2025-02-06T11:44:15.971-06:00 level=DEBUG source=gpu.go:576 msg="discovered GPU libraries" paths=[C:\WINDOWS\system32\nvcuda.dll]
initializing C:\WINDOWS\system32\nvcuda.dll
dlsym: cuInit - 00007FFF05775F80
dlsym: cuDriverGetVersion - 00007FFF05776020
dlsym: cuDeviceGetCount - 00007FFF05776816
dlsym: cuDeviceGet - 00007FFF05776810
dlsym: cuDeviceGetAttribute - 00007FFF05776170
dlsym: cuDeviceGetUuid - 00007FFF05776822
dlsym: cuDeviceGetName - 00007FFF0577681C
dlsym: cuCtxCreate_v3 - 00007FFF05776894
dlsym: cuMemGetInfo_v2 - 00007FFF05776996
dlsym: cuCtxDestroy - 00007FFF057768A6
calling cuInit
cuInit err: 100
time=2025-02-06T11:44:15.980-06:00 level=INFO source=gpu.go:620 msg="no nvidia devices detected by library C:\\WINDOWS\\system32\\nvcuda.dll"
time=2025-02-06T11:44:15.980-06:00 level=DEBUG source=gpu.go:517 msg="Searching for GPU library" name=cudart64_*.dll
time=2025-02-06T11:44:15.980-06:00 level=DEBUG source=gpu.go:543 msg="gpu library search" globs="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll C:\\Program Files\\Python313\\Scripts\\cudart64_*.dll C:\\Program Files\\Python313\\cudart64_*.dll C:\\WINDOWS\\system32\\cudart64_*.dll C:\\WINDOWS\\cudart64_*.dll C:\\WINDOWS\\System32\\Wbem\\cudart64_*.dll C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\cudart64_*.dll C:\\WINDOWS\\System32\\OpenSSH\\cudart64_*.dll C:\\Program Files\\Microsoft VS Code\\bin\\cudart64_*.dll C:\\Program Files (x86)\\WinSCP\\cudart64_*.dll C:\\Program Files\\Git\\cmd\\cudart64_*.dll C:\\Program Files\\Docker\\Docker\\resources\\bin\\cudart64_*.dll C:\\Program Files\\Mullvad VPN\\resources\\cudart64_*.dll C:\\Program Files\\PowerShell\\7\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Microsoft\\WindowsApps\\cudart64_*.dll D:\\Development\\BigBoyEnhancement\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll c:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v*\\bin\\cudart64_*.dll]"
time=2025-02-06T11:44:15.983-06:00 level=DEBUG source=gpu.go:576 msg="discovered GPU libraries" paths="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_110.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_12.dll]"
cudaSetDevice err: 100
time=2025-02-06T11:44:15.987-06:00 level=DEBUG source=gpu.go:592 msg="Unable to load cudart library C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_110.dll: cudart init failure: 100"
cudaSetDevice err: 100
time=2025-02-06T11:44:15.991-06:00 level=DEBUG source=gpu.go:592 msg="Unable to load cudart library C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_12.dll: cudart init failure: 100"
time=2025-02-06T11:44:15.991-06:00 level=DEBUG source=amd_windows.go:35 msg="unable to load amdhip64_6.dll, please make sure to upgrade to the latest amd driver: The specified module could not be found."
time=2025-02-06T11:44:15.991-06:00 level=INFO source=gpu.go:392 msg="no compatible GPUs were discovered"
releasing nvml library
time=2025-02-06T11:44:15.992-06:00 level=INFO source=types.go:131 msg="inference compute" id=0 library=cpu variant=avx2 compute="" driver=0.0 name="" total="127.7 GiB" available="117.3 GiB"

OS

Windows

GPU

Nvidia

CPU

Intel

Ollama version

0.5.7

Originally created by @ihor-sokoliuk on GitHub (Feb 6, 2025). Original GitHub issue: https://github.com/ollama/ollama/issues/8892 ### What is the issue? Here is the BAT file to run Ollama on my Windows machine: ``` @echo off set CUDA_VISIBLE_DEVICES="" set OLLAMA_ORIGINS="*" set OLLAMA_HOST=0.0.0.0:11434 set OLLAMA_DEBUG=1 ollama serve timeout /t 100 /nobreak ``` When I run the Ollama server with the flag `CUDA_VISIBLE_DEVICES=""` it starts but fails in a minute or two without any error message and with a random return code like `2147842401`. Please take a look at the attached server log. If I run the same BAT without the `CUDA_VISIBLE_DEVICES` variable, it works as expected. How can I run the Ollama server without a GPU (with CPU+RAM only) so that it will continue to work? ### Relevant log output ```shell 2025/02/06 11:44:15 routes.go:1187: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:C:\\Users\\ihor\\.ollama\\models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[* http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES:]" time=2025-02-06T11:44:15.892-06:00 level=INFO source=images.go:432 msg="total blobs: 7" time=2025-02-06T11:44:15.892-06:00 level=INFO source=images.go:439 msg="total unused blobs removed: 0" time=2025-02-06T11:44:15.893-06:00 level=INFO source=routes.go:1238 msg="Listening on [::]:11434 (version 0.5.7)" time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=common.go:80 msg="runners located" dir=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cpu_avx\ollama_llama_server.exe time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cpu_avx2\ollama_llama_server.exe time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cuda_v11_avx\ollama_llama_server.exe time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cuda_v12_avx\ollama_llama_server.exe time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\rocm_avx\ollama_llama_server.exe time=2025-02-06T11:44:15.893-06:00 level=INFO source=routes.go:1267 msg="Dynamic LLM libraries" runners="[cuda_v12_avx rocm_avx cpu cpu_avx cpu_avx2 cuda_v11_avx]" time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=routes.go:1268 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY" time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=sched.go:105 msg="starting llm scheduler" time=2025-02-06T11:44:15.893-06:00 level=INFO source=gpu.go:226 msg="looking for compatible GPUs" time=2025-02-06T11:44:15.893-06:00 level=INFO source=gpu_windows.go:167 msg=packages count=1 time=2025-02-06T11:44:15.893-06:00 level=INFO source=gpu_windows.go:183 msg="efficiency cores detected" maxEfficiencyClass=1 time=2025-02-06T11:44:15.893-06:00 level=INFO source=gpu_windows.go:214 msg="" package=0 cores=24 efficiency=16 threads=32 time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=gpu.go:99 msg="searching for GPU discovery libraries for NVIDIA" time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=gpu.go:517 msg="Searching for GPU library" name=nvml.dll time=2025-02-06T11:44:15.893-06:00 level=DEBUG source=gpu.go:543 msg="gpu library search" globs="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvml.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvml.dll C:\\Program Files\\Python313\\Scripts\\nvml.dll C:\\Program Files\\Python313\\nvml.dll C:\\WINDOWS\\system32\\nvml.dll C:\\WINDOWS\\nvml.dll C:\\WINDOWS\\System32\\Wbem\\nvml.dll C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\nvml.dll C:\\WINDOWS\\System32\\OpenSSH\\nvml.dll C:\\Program Files\\Microsoft VS Code\\bin\\nvml.dll C:\\Program Files (x86)\\WinSCP\\nvml.dll C:\\Program Files\\Git\\cmd\\nvml.dll C:\\Program Files\\Docker\\Docker\\resources\\bin\\nvml.dll C:\\Program Files\\Mullvad VPN\\resources\\nvml.dll C:\\Program Files\\PowerShell\\7\\nvml.dll C:\\Users\\ihor\\AppData\\Local\\Microsoft\\WindowsApps\\nvml.dll D:\\Development\\BigBoyEnhancement\\nvml.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\nvml.dll c:\\Windows\\System32\\nvml.dll]" time=2025-02-06T11:44:15.894-06:00 level=DEBUG source=gpu.go:576 msg="discovered GPU libraries" paths="[C:\\WINDOWS\\system32\\nvml.dll c:\\Windows\\System32\\nvml.dll]" time=2025-02-06T11:44:15.903-06:00 level=DEBUG source=gpu.go:120 msg="nvidia-ml loaded" library=C:\WINDOWS\system32\nvml.dll time=2025-02-06T11:44:15.903-06:00 level=DEBUG source=gpu.go:517 msg="Searching for GPU library" name=nvcuda.dll time=2025-02-06T11:44:15.903-06:00 level=DEBUG source=gpu.go:543 msg="gpu library search" globs="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvcuda.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvcuda.dll C:\\Program Files\\Python313\\Scripts\\nvcuda.dll C:\\Program Files\\Python313\\nvcuda.dll C:\\WINDOWS\\system32\\nvcuda.dll C:\\WINDOWS\\nvcuda.dll C:\\WINDOWS\\System32\\Wbem\\nvcuda.dll C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\nvcuda.dll C:\\WINDOWS\\System32\\OpenSSH\\nvcuda.dll C:\\Program Files\\Microsoft VS Code\\bin\\nvcuda.dll C:\\Program Files (x86)\\WinSCP\\nvcuda.dll C:\\Program Files\\Git\\cmd\\nvcuda.dll C:\\Program Files\\Docker\\Docker\\resources\\bin\\nvcuda.dll C:\\Program Files\\Mullvad VPN\\resources\\nvcuda.dll C:\\Program Files\\PowerShell\\7\\nvcuda.dll C:\\Users\\ihor\\AppData\\Local\\Microsoft\\WindowsApps\\nvcuda.dll D:\\Development\\BigBoyEnhancement\\nvcuda.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\nvcuda.dll c:\\windows\\system*\\nvcuda.dll]" time=2025-02-06T11:44:15.971-06:00 level=DEBUG source=gpu.go:576 msg="discovered GPU libraries" paths=[C:\WINDOWS\system32\nvcuda.dll] initializing C:\WINDOWS\system32\nvcuda.dll dlsym: cuInit - 00007FFF05775F80 dlsym: cuDriverGetVersion - 00007FFF05776020 dlsym: cuDeviceGetCount - 00007FFF05776816 dlsym: cuDeviceGet - 00007FFF05776810 dlsym: cuDeviceGetAttribute - 00007FFF05776170 dlsym: cuDeviceGetUuid - 00007FFF05776822 dlsym: cuDeviceGetName - 00007FFF0577681C dlsym: cuCtxCreate_v3 - 00007FFF05776894 dlsym: cuMemGetInfo_v2 - 00007FFF05776996 dlsym: cuCtxDestroy - 00007FFF057768A6 calling cuInit cuInit err: 100 time=2025-02-06T11:44:15.980-06:00 level=INFO source=gpu.go:620 msg="no nvidia devices detected by library C:\\WINDOWS\\system32\\nvcuda.dll" time=2025-02-06T11:44:15.980-06:00 level=DEBUG source=gpu.go:517 msg="Searching for GPU library" name=cudart64_*.dll time=2025-02-06T11:44:15.980-06:00 level=DEBUG source=gpu.go:543 msg="gpu library search" globs="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll C:\\Program Files\\Python313\\Scripts\\cudart64_*.dll C:\\Program Files\\Python313\\cudart64_*.dll C:\\WINDOWS\\system32\\cudart64_*.dll C:\\WINDOWS\\cudart64_*.dll C:\\WINDOWS\\System32\\Wbem\\cudart64_*.dll C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\cudart64_*.dll C:\\WINDOWS\\System32\\OpenSSH\\cudart64_*.dll C:\\Program Files\\Microsoft VS Code\\bin\\cudart64_*.dll C:\\Program Files (x86)\\WinSCP\\cudart64_*.dll C:\\Program Files\\Git\\cmd\\cudart64_*.dll C:\\Program Files\\Docker\\Docker\\resources\\bin\\cudart64_*.dll C:\\Program Files\\Mullvad VPN\\resources\\cudart64_*.dll C:\\Program Files\\PowerShell\\7\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Microsoft\\WindowsApps\\cudart64_*.dll D:\\Development\\BigBoyEnhancement\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll c:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v*\\bin\\cudart64_*.dll]" time=2025-02-06T11:44:15.983-06:00 level=DEBUG source=gpu.go:576 msg="discovered GPU libraries" paths="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_110.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_12.dll]" cudaSetDevice err: 100 time=2025-02-06T11:44:15.987-06:00 level=DEBUG source=gpu.go:592 msg="Unable to load cudart library C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_110.dll: cudart init failure: 100" cudaSetDevice err: 100 time=2025-02-06T11:44:15.991-06:00 level=DEBUG source=gpu.go:592 msg="Unable to load cudart library C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_12.dll: cudart init failure: 100" time=2025-02-06T11:44:15.991-06:00 level=DEBUG source=amd_windows.go:35 msg="unable to load amdhip64_6.dll, please make sure to upgrade to the latest amd driver: The specified module could not be found." time=2025-02-06T11:44:15.991-06:00 level=INFO source=gpu.go:392 msg="no compatible GPUs were discovered" releasing nvml library time=2025-02-06T11:44:15.992-06:00 level=INFO source=types.go:131 msg="inference compute" id=0 library=cpu variant=avx2 compute="" driver=0.0 name="" total="127.7 GiB" available="117.3 GiB" ``` ### OS Windows ### GPU Nvidia ### CPU Intel ### Ollama version 0.5.7
GiteaMirror added the nvidiabug labels 2026-04-12 17:05:48 -05:00
GiteaMirror commented 2026-04-12 17:05:51 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 6, 2025):

CUDA_VISIBLE_DEVICES="" is not a valid setting. Documentation says comma separated list of identifiers. You can disable all devices with CUDA_VISIBLE_DEVICES="-1".

<!-- gh-comment-id:2640616021 --> @rick-github commented on GitHub (Feb 6, 2025): `CUDA_VISIBLE_DEVICES=""` is not a valid setting. Documentation says [comma separated list of identifiers]( https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars:~:text=Enumeration%20and%20Properties-,CUDA_VISIBLE_DEVICES,-A%20comma%2Dseparated). You can disable all devices with `CUDA_VISIBLE_DEVICES="-1"`.
GiteaMirror commented 2026-04-12 17:05:52 -05:00
Author
Owner
Copy Link

@ihor-sokoliuk commented on GitHub (Feb 6, 2025):

@rick-github Thank you for getting back to me so quickly.

I've tried that, and the same issue remains. It stops working in a minute.

The updated BAT file:

@echo off
set CUDA_VISIBLE_DEVICES="-1"
set OLLAMA_ORIGINS="*"
set OLLAMA_HOST=0.0.0.0:11434
set OLLAMA_DEBUG=1
ollama serve
timeout /t 100 /nobreak

The latest log:

2025/02/06 15:34:24 routes.go:1187: INFO server config env="map[CUDA_VISIBLE_DEVICES:-1 GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:C:\\Users\\ihor\\.ollama\\models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[* http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES:]"
time=2025-02-06T15:34:24.814-06:00 level=INFO source=images.go:432 msg="total blobs: 7"
time=2025-02-06T15:34:24.814-06:00 level=INFO source=images.go:439 msg="total unused blobs removed: 0"
time=2025-02-06T15:34:24.814-06:00 level=INFO source=routes.go:1238 msg="Listening on [::]:11434 (version 0.5.7)"
time=2025-02-06T15:34:24.814-06:00 level=DEBUG source=common.go:80 msg="runners located" dir=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners
time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cpu_avx\ollama_llama_server.exe
time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cpu_avx2\ollama_llama_server.exe
time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cuda_v11_avx\ollama_llama_server.exe
time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cuda_v12_avx\ollama_llama_server.exe
time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\rocm_avx\ollama_llama_server.exe
time=2025-02-06T15:34:24.815-06:00 level=INFO source=routes.go:1267 msg="Dynamic LLM libraries" runners="[cuda_v12_avx rocm_avx cpu cpu_avx cpu_avx2 cuda_v11_avx]"
time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=routes.go:1268 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY"
time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=sched.go:105 msg="starting llm scheduler"
time=2025-02-06T15:34:24.815-06:00 level=INFO source=gpu.go:226 msg="looking for compatible GPUs"
time=2025-02-06T15:34:24.815-06:00 level=INFO source=gpu_windows.go:167 msg=packages count=1
time=2025-02-06T15:34:24.815-06:00 level=INFO source=gpu_windows.go:183 msg="efficiency cores detected" maxEfficiencyClass=1
time=2025-02-06T15:34:24.815-06:00 level=INFO source=gpu_windows.go:214 msg="" package=0 cores=24 efficiency=16 threads=32
time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=gpu.go:99 msg="searching for GPU discovery libraries for NVIDIA"
time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=gpu.go:517 msg="Searching for GPU library" name=nvml.dll
time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=gpu.go:543 msg="gpu library search" globs="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvml.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvml.dll C:\\Program Files\\Python313\\Scripts\\nvml.dll C:\\Program Files\\Python313\\nvml.dll C:\\WINDOWS\\system32\\nvml.dll C:\\WINDOWS\\nvml.dll C:\\WINDOWS\\System32\\Wbem\\nvml.dll C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\nvml.dll C:\\WINDOWS\\System32\\OpenSSH\\nvml.dll C:\\Program Files\\Microsoft VS Code\\bin\\nvml.dll C:\\Program Files (x86)\\WinSCP\\nvml.dll C:\\Program Files\\Git\\cmd\\nvml.dll C:\\Program Files\\Docker\\Docker\\resources\\bin\\nvml.dll C:\\Program Files\\Mullvad VPN\\resources\\nvml.dll C:\\Program Files\\PowerShell\\7\\nvml.dll C:\\Users\\ihor\\AppData\\Local\\Microsoft\\WindowsApps\\nvml.dll D:\\Development\\BigBoyEnhancement\\nvml.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\nvml.dll c:\\Windows\\System32\\nvml.dll]"
time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=gpu.go:576 msg="discovered GPU libraries" paths="[C:\\WINDOWS\\system32\\nvml.dll c:\\Windows\\System32\\nvml.dll]"
time=2025-02-06T15:34:24.824-06:00 level=DEBUG source=gpu.go:120 msg="nvidia-ml loaded" library=C:\WINDOWS\system32\nvml.dll
time=2025-02-06T15:34:24.824-06:00 level=DEBUG source=gpu.go:517 msg="Searching for GPU library" name=nvcuda.dll
time=2025-02-06T15:34:24.825-06:00 level=DEBUG source=gpu.go:543 msg="gpu library search" globs="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvcuda.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvcuda.dll C:\\Program Files\\Python313\\Scripts\\nvcuda.dll C:\\Program Files\\Python313\\nvcuda.dll C:\\WINDOWS\\system32\\nvcuda.dll C:\\WINDOWS\\nvcuda.dll C:\\WINDOWS\\System32\\Wbem\\nvcuda.dll C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\nvcuda.dll C:\\WINDOWS\\System32\\OpenSSH\\nvcuda.dll C:\\Program Files\\Microsoft VS Code\\bin\\nvcuda.dll C:\\Program Files (x86)\\WinSCP\\nvcuda.dll C:\\Program Files\\Git\\cmd\\nvcuda.dll C:\\Program Files\\Docker\\Docker\\resources\\bin\\nvcuda.dll C:\\Program Files\\Mullvad VPN\\resources\\nvcuda.dll C:\\Program Files\\PowerShell\\7\\nvcuda.dll C:\\Users\\ihor\\AppData\\Local\\Microsoft\\WindowsApps\\nvcuda.dll D:\\Development\\BigBoyEnhancement\\nvcuda.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\nvcuda.dll c:\\windows\\system*\\nvcuda.dll]"
time=2025-02-06T15:34:24.825-06:00 level=DEBUG source=gpu.go:576 msg="discovered GPU libraries" paths=[C:\WINDOWS\system32\nvcuda.dll]
initializing C:\WINDOWS\system32\nvcuda.dll
dlsym: cuInit - 00007FFC7C3D5F80
dlsym: cuDriverGetVersion - 00007FFC7C3D6020
dlsym: cuDeviceGetCount - 00007FFC7C3D6816
dlsym: cuDeviceGet - 00007FFC7C3D6810
dlsym: cuDeviceGetAttribute - 00007FFC7C3D6170
dlsym: cuDeviceGetUuid - 00007FFC7C3D6822
dlsym: cuDeviceGetName - 00007FFC7C3D681C
dlsym: cuCtxCreate_v3 - 00007FFC7C3D6894
dlsym: cuMemGetInfo_v2 - 00007FFC7C3D6996
dlsym: cuCtxDestroy - 00007FFC7C3D68A6
calling cuInit
cuInit err: 100
time=2025-02-06T15:34:24.833-06:00 level=INFO source=gpu.go:620 msg="no nvidia devices detected by library C:\\WINDOWS\\system32\\nvcuda.dll"
time=2025-02-06T15:34:24.834-06:00 level=DEBUG source=gpu.go:517 msg="Searching for GPU library" name=cudart64_*.dll
time=2025-02-06T15:34:24.834-06:00 level=DEBUG source=gpu.go:543 msg="gpu library search" globs="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll C:\\Program Files\\Python313\\Scripts\\cudart64_*.dll C:\\Program Files\\Python313\\cudart64_*.dll C:\\WINDOWS\\system32\\cudart64_*.dll C:\\WINDOWS\\cudart64_*.dll C:\\WINDOWS\\System32\\Wbem\\cudart64_*.dll C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\cudart64_*.dll C:\\WINDOWS\\System32\\OpenSSH\\cudart64_*.dll C:\\Program Files\\Microsoft VS Code\\bin\\cudart64_*.dll C:\\Program Files (x86)\\WinSCP\\cudart64_*.dll C:\\Program Files\\Git\\cmd\\cudart64_*.dll C:\\Program Files\\Docker\\Docker\\resources\\bin\\cudart64_*.dll C:\\Program Files\\Mullvad VPN\\resources\\cudart64_*.dll C:\\Program Files\\PowerShell\\7\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Microsoft\\WindowsApps\\cudart64_*.dll D:\\Development\\BigBoyEnhancement\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll c:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v*\\bin\\cudart64_*.dll]"
time=2025-02-06T15:34:24.837-06:00 level=DEBUG source=gpu.go:576 msg="discovered GPU libraries" paths="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_110.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_12.dll]"
cudaSetDevice err: 100
time=2025-02-06T15:34:24.841-06:00 level=DEBUG source=gpu.go:592 msg="Unable to load cudart library C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_110.dll: cudart init failure: 100"
cudaSetDevice err: 100
time=2025-02-06T15:34:24.845-06:00 level=DEBUG source=gpu.go:592 msg="Unable to load cudart library C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_12.dll: cudart init failure: 100"
time=2025-02-06T15:34:24.846-06:00 level=DEBUG source=amd_windows.go:35 msg="unable to load amdhip64_6.dll, please make sure to upgrade to the latest amd driver: The specified module could not be found."
time=2025-02-06T15:34:24.846-06:00 level=INFO source=gpu.go:392 msg="no compatible GPUs were discovered"
releasing nvml library
time=2025-02-06T15:34:24.846-06:00 level=INFO source=types.go:131 msg="inference compute" id=0 library=cpu variant=avx2 compute="" driver=0.0 name="" total="127.7 GiB" available="119.5 GiB"

A screenshot with the failure and without any message:
Image

I have the latest Windows 11 24H2 with all possible OS and Drivers upgrades.

Any other ideas?

<!-- gh-comment-id:2641102220 --> @ihor-sokoliuk commented on GitHub (Feb 6, 2025): @rick-github Thank you for getting back to me so quickly. I've tried that, and the same issue remains. It stops working in a minute. The updated BAT file: ``` @echo off set CUDA_VISIBLE_DEVICES="-1" set OLLAMA_ORIGINS="*" set OLLAMA_HOST=0.0.0.0:11434 set OLLAMA_DEBUG=1 ollama serve timeout /t 100 /nobreak ``` The latest log: ``` 2025/02/06 15:34:24 routes.go:1187: INFO server config env="map[CUDA_VISIBLE_DEVICES:-1 GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:true OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:C:\\Users\\ihor\\.ollama\\models OLLAMA_MULTIUSER_CACHE:false OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[* http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://*] OLLAMA_SCHED_SPREAD:false ROCR_VISIBLE_DEVICES:]" time=2025-02-06T15:34:24.814-06:00 level=INFO source=images.go:432 msg="total blobs: 7" time=2025-02-06T15:34:24.814-06:00 level=INFO source=images.go:439 msg="total unused blobs removed: 0" time=2025-02-06T15:34:24.814-06:00 level=INFO source=routes.go:1238 msg="Listening on [::]:11434 (version 0.5.7)" time=2025-02-06T15:34:24.814-06:00 level=DEBUG source=common.go:80 msg="runners located" dir=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cpu_avx\ollama_llama_server.exe time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cpu_avx2\ollama_llama_server.exe time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cuda_v11_avx\ollama_llama_server.exe time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\cuda_v12_avx\ollama_llama_server.exe time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=common.go:124 msg="availableServers : found" file=C:\Users\ihor\AppData\Local\Programs\Ollama\lib\ollama\runners\rocm_avx\ollama_llama_server.exe time=2025-02-06T15:34:24.815-06:00 level=INFO source=routes.go:1267 msg="Dynamic LLM libraries" runners="[cuda_v12_avx rocm_avx cpu cpu_avx cpu_avx2 cuda_v11_avx]" time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=routes.go:1268 msg="Override detection logic by setting OLLAMA_LLM_LIBRARY" time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=sched.go:105 msg="starting llm scheduler" time=2025-02-06T15:34:24.815-06:00 level=INFO source=gpu.go:226 msg="looking for compatible GPUs" time=2025-02-06T15:34:24.815-06:00 level=INFO source=gpu_windows.go:167 msg=packages count=1 time=2025-02-06T15:34:24.815-06:00 level=INFO source=gpu_windows.go:183 msg="efficiency cores detected" maxEfficiencyClass=1 time=2025-02-06T15:34:24.815-06:00 level=INFO source=gpu_windows.go:214 msg="" package=0 cores=24 efficiency=16 threads=32 time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=gpu.go:99 msg="searching for GPU discovery libraries for NVIDIA" time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=gpu.go:517 msg="Searching for GPU library" name=nvml.dll time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=gpu.go:543 msg="gpu library search" globs="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvml.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvml.dll C:\\Program Files\\Python313\\Scripts\\nvml.dll C:\\Program Files\\Python313\\nvml.dll C:\\WINDOWS\\system32\\nvml.dll C:\\WINDOWS\\nvml.dll C:\\WINDOWS\\System32\\Wbem\\nvml.dll C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\nvml.dll C:\\WINDOWS\\System32\\OpenSSH\\nvml.dll C:\\Program Files\\Microsoft VS Code\\bin\\nvml.dll C:\\Program Files (x86)\\WinSCP\\nvml.dll C:\\Program Files\\Git\\cmd\\nvml.dll C:\\Program Files\\Docker\\Docker\\resources\\bin\\nvml.dll C:\\Program Files\\Mullvad VPN\\resources\\nvml.dll C:\\Program Files\\PowerShell\\7\\nvml.dll C:\\Users\\ihor\\AppData\\Local\\Microsoft\\WindowsApps\\nvml.dll D:\\Development\\BigBoyEnhancement\\nvml.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\nvml.dll c:\\Windows\\System32\\nvml.dll]" time=2025-02-06T15:34:24.815-06:00 level=DEBUG source=gpu.go:576 msg="discovered GPU libraries" paths="[C:\\WINDOWS\\system32\\nvml.dll c:\\Windows\\System32\\nvml.dll]" time=2025-02-06T15:34:24.824-06:00 level=DEBUG source=gpu.go:120 msg="nvidia-ml loaded" library=C:\WINDOWS\system32\nvml.dll time=2025-02-06T15:34:24.824-06:00 level=DEBUG source=gpu.go:517 msg="Searching for GPU library" name=nvcuda.dll time=2025-02-06T15:34:24.825-06:00 level=DEBUG source=gpu.go:543 msg="gpu library search" globs="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvcuda.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\nvcuda.dll C:\\Program Files\\Python313\\Scripts\\nvcuda.dll C:\\Program Files\\Python313\\nvcuda.dll C:\\WINDOWS\\system32\\nvcuda.dll C:\\WINDOWS\\nvcuda.dll C:\\WINDOWS\\System32\\Wbem\\nvcuda.dll C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\nvcuda.dll C:\\WINDOWS\\System32\\OpenSSH\\nvcuda.dll C:\\Program Files\\Microsoft VS Code\\bin\\nvcuda.dll C:\\Program Files (x86)\\WinSCP\\nvcuda.dll C:\\Program Files\\Git\\cmd\\nvcuda.dll C:\\Program Files\\Docker\\Docker\\resources\\bin\\nvcuda.dll C:\\Program Files\\Mullvad VPN\\resources\\nvcuda.dll C:\\Program Files\\PowerShell\\7\\nvcuda.dll C:\\Users\\ihor\\AppData\\Local\\Microsoft\\WindowsApps\\nvcuda.dll D:\\Development\\BigBoyEnhancement\\nvcuda.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\nvcuda.dll c:\\windows\\system*\\nvcuda.dll]" time=2025-02-06T15:34:24.825-06:00 level=DEBUG source=gpu.go:576 msg="discovered GPU libraries" paths=[C:\WINDOWS\system32\nvcuda.dll] initializing C:\WINDOWS\system32\nvcuda.dll dlsym: cuInit - 00007FFC7C3D5F80 dlsym: cuDriverGetVersion - 00007FFC7C3D6020 dlsym: cuDeviceGetCount - 00007FFC7C3D6816 dlsym: cuDeviceGet - 00007FFC7C3D6810 dlsym: cuDeviceGetAttribute - 00007FFC7C3D6170 dlsym: cuDeviceGetUuid - 00007FFC7C3D6822 dlsym: cuDeviceGetName - 00007FFC7C3D681C dlsym: cuCtxCreate_v3 - 00007FFC7C3D6894 dlsym: cuMemGetInfo_v2 - 00007FFC7C3D6996 dlsym: cuCtxDestroy - 00007FFC7C3D68A6 calling cuInit cuInit err: 100 time=2025-02-06T15:34:24.833-06:00 level=INFO source=gpu.go:620 msg="no nvidia devices detected by library C:\\WINDOWS\\system32\\nvcuda.dll" time=2025-02-06T15:34:24.834-06:00 level=DEBUG source=gpu.go:517 msg="Searching for GPU library" name=cudart64_*.dll time=2025-02-06T15:34:24.834-06:00 level=DEBUG source=gpu.go:543 msg="gpu library search" globs="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll C:\\Program Files\\Python313\\Scripts\\cudart64_*.dll C:\\Program Files\\Python313\\cudart64_*.dll C:\\WINDOWS\\system32\\cudart64_*.dll C:\\WINDOWS\\cudart64_*.dll C:\\WINDOWS\\System32\\Wbem\\cudart64_*.dll C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\cudart64_*.dll C:\\WINDOWS\\System32\\OpenSSH\\cudart64_*.dll C:\\Program Files\\Microsoft VS Code\\bin\\cudart64_*.dll C:\\Program Files (x86)\\WinSCP\\cudart64_*.dll C:\\Program Files\\Git\\cmd\\cudart64_*.dll C:\\Program Files\\Docker\\Docker\\resources\\bin\\cudart64_*.dll C:\\Program Files\\Mullvad VPN\\resources\\cudart64_*.dll C:\\Program Files\\PowerShell\\7\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Microsoft\\WindowsApps\\cudart64_*.dll D:\\Development\\BigBoyEnhancement\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_*.dll c:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v*\\bin\\cudart64_*.dll]" time=2025-02-06T15:34:24.837-06:00 level=DEBUG source=gpu.go:576 msg="discovered GPU libraries" paths="[C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_110.dll C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_12.dll]" cudaSetDevice err: 100 time=2025-02-06T15:34:24.841-06:00 level=DEBUG source=gpu.go:592 msg="Unable to load cudart library C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_110.dll: cudart init failure: 100" cudaSetDevice err: 100 time=2025-02-06T15:34:24.845-06:00 level=DEBUG source=gpu.go:592 msg="Unable to load cudart library C:\\Users\\ihor\\AppData\\Local\\Programs\\Ollama\\lib\\ollama\\cudart64_12.dll: cudart init failure: 100" time=2025-02-06T15:34:24.846-06:00 level=DEBUG source=amd_windows.go:35 msg="unable to load amdhip64_6.dll, please make sure to upgrade to the latest amd driver: The specified module could not be found." time=2025-02-06T15:34:24.846-06:00 level=INFO source=gpu.go:392 msg="no compatible GPUs were discovered" releasing nvml library time=2025-02-06T15:34:24.846-06:00 level=INFO source=types.go:131 msg="inference compute" id=0 library=cpu variant=avx2 compute="" driver=0.0 name="" total="127.7 GiB" available="119.5 GiB" ``` A screenshot with the failure and without any message: ![Image](https://github.com/user-attachments/assets/4ba5c7c5-4dd3-4658-9876-51ad35a845f4) I have the latest Windows 11 24H2 with all possible OS and Drivers upgrades. Any other ideas?
GiteaMirror commented 2026-04-12 17:05:53 -05:00
Author
Owner
Copy Link

@rick-github commented on GitHub (Feb 6, 2025):

Hmm, that works to disable CUDA devices on my linux systems, and it doesn't crash. There is another way to disable GPU, set num_gpu to zero, see here.

<!-- gh-comment-id:2641185458 --> @rick-github commented on GitHub (Feb 6, 2025): Hmm, that works to disable CUDA devices on my linux systems, and it doesn't crash. There is another way to disable GPU, set `num_gpu` to zero, see [here](https://github.com/ollama/ollama/issues/6950#issuecomment-2373663650).
GiteaMirror commented 2026-04-12 17:05:54 -05:00
Author
Owner
Copy Link

@ihor-sokoliuk commented on GitHub (Feb 6, 2025):

The num_gpu in Modelfile approach works best for me. Thank you @rick-github!

<!-- gh-comment-id:2641343816 --> @ihor-sokoliuk commented on GitHub (Feb 6, 2025): The `num_gpu in Modelfile` approach works best for me. Thank you @rick-github!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label bug nvidia
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#5766
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?