github-starred/ollama
2
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2026-05-07 00:22:43 -05:00
Code Issues 14.5k Packages Projects Releases 120 Wiki Activity

[PR #4106] [CLOSED] Fix for Nvidia installed deps detection algorithm in gpu.go #16652

New Issue
Closed
opened 2026-04-16 05:38:30 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-04-16 05:38:30 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/ollama/ollama/pull/4106
Author: @alecvern
Created: 5/2/2024
Status: ❌ Closed

Base: main ← Head: main

📝 Commits (2)

  • 415eb2c Fix for Nvidia installed deps detection algorithm in gpu.go
  • 76c4af4 Update gpu.go

📊 Changes

1 file changed (+44 additions, -0 deletions)

View changed files

📝 gpu/gpu.go (+44 -0)

📄 Description

Related to #3593, #4008

In brief: I faced a similar issue and after a long analysis I found the problem in the current version of gpu.go code.

Context:

  • Windows 11
  • Nvidia GPU
  • usage of portable ollama version "ollama-windows-amd64.exe"

Currently, the code is looking for the cudart64_*.dll library in the following folders:

  1. the folder where ollama is installed through the standard installer (C:\Users\%USERNAME%\AppData\Local\Programs\Ollama).
  2. the folder where Cuda Toolkit is installed, being set manually in code via
    var CudartWindowsGlobs = []string{ "c:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v*\\bin\\cudart64_*.dll" }
  3. All folders in the Path array from Windows Environment Variables.

The problem is that among the paths in (3) there is a path to the PhysX installation folder (installed by default with Nvidia drivers).
Screenshot 2024-05-02 212136
The PhysX installation contains cudart64_*.dll, but there are two related (and necessary for ollama) libraries missing: cublas and cublasLt.
This causes the scanner to constantly detect this folder when starting chat:

(when there are no any other cudart at all in the system)

image

(and when everything is in place, among legitimate paths for cudart)

Screenshot 2024-05-02 233040

The problem doesn't manifest itself so obviously on most devices now because:

  1. most install ollama through the installer, and the cublas path from the installed ollama folder is prioritized over the one detected in the PhysX folder
  2. many install the Cuda Toolkit and this path is also interpreted by ollama as higher priority at startup
  3. some people (like me) use portable ollama and run it from the same folder where they store individual instances of the nvidia libraries (when running portable ollama from a folder, that folder is assigned to cmd.exe as a working directory, which is automatically added to the environment variables during startup). This way of starting is also a higher priority for ollama, I guess because the cudart library in the PhysX folder is usually more recent than the ones used next to the portable ollama (and ollama always prefers to start trying to run with the oldest version found -- and that will fire later).

Everything would be fine for now if there was no such thing as PhysX Legacy, which is installed together with some old games and applications.
physx legacy
In this case, it turns out that the portable ollama on run prioritizes the cudart from the PhysX Legacy folder even over the cudart in the folder where the portable ollama .exe running from. This leads to an unhandled error and a crash of the ollama server (exact case of #3593) because once it finds the cudart in the PhysX folder it cannot then discover expected cublas and cublasLt libs needed for the LLM to work.

To summarize: the current implementation of gpu.go may shoot up in the future. Even now, it creates malfunctions, provided 3 conditions are met:

  1. using portable ollama (ollama-windows-amd64.exe) with nvidia libs alongside, in the same folder
  2. you have not installed cuda toolkit
  3. by pure coincidence you have PhysX Legacy installed (or) the current PhysX is installed right now along with the driver and after some future update ollama suddenly prioritizes an attempt to run through cudart from the PhysX folder.

In any case, trying to index the PhysX folder as legitimately containing the necessary Nvidia libraries is an undesigned behavior leading to crashes on startup of chatting session. There are 2 required libraries missing in any version of PhysX.
Also, the proposed code checks for the presence of these very libraries in the scanned folders to avoid similar cases, even not related to PhysX.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/ollama/ollama/pull/4106 **Author:** [@alecvern](https://github.com/alecvern) **Created:** 5/2/2024 **Status:** ❌ Closed **Base:** `main` ← **Head:** `main` --- ### 📝 Commits (2) - [`415eb2c`](https://github.com/ollama/ollama/commit/415eb2c65bdac794aae0228a269848717d955785) Fix for Nvidia installed deps detection algorithm in gpu.go - [`76c4af4`](https://github.com/ollama/ollama/commit/76c4af4d0e0fc0af97e9fe74e6137c5badf5004a) Update gpu.go ### 📊 Changes **1 file changed** (+44 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `gpu/gpu.go` (+44 -0) </details> ### 📄 Description Related to #3593, #4008 In brief: I faced a similar issue and after a long analysis I found the problem in the current version of gpu.go code. Context: - Windows 11 - Nvidia GPU - usage of portable ollama version "ollama-windows-amd64.exe" Currently, the code is looking for the cudart64_*.dll library in the following folders: 1) the folder where ollama is installed through the standard installer (C:\Users\\%USERNAME%\AppData\Local\Programs\Ollama). 2) the folder where Cuda Toolkit is installed, being set manually in code via `var CudartWindowsGlobs = []string{ "c:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v*\\bin\\cudart64_*.dll" }` 3) All folders in the Path array from Windows Environment Variables. The problem is that among the paths in (3) there is a path to the PhysX installation folder (installed by default with Nvidia drivers). ![Screenshot 2024-05-02 212136](https://github.com/ollama/ollama/assets/17222170/d860ca9a-38a0-46c7-97d5-f82facca04b6) The PhysX installation contains cudart64_*.dll, but there are two related (and necessary for ollama) libraries missing: cublas and cublasLt. This causes the scanner to constantly detect this folder when starting chat: (when there are no any other cudart at all in the system) ![image](https://github.com/ollama/ollama/assets/17222170/955cd8c9-1c90-4f40-95eb-3b13a8d3398d) (and when everything is in place, among legitimate paths for cudart) ![Screenshot 2024-05-02 233040](https://github.com/ollama/ollama/assets/17222170/55c0f0a2-8d56-472f-a51e-d9b9a85ffca5) The problem doesn't manifest itself so obviously on most devices **now** because: 1) most install ollama through the installer, and the cublas path from the installed ollama folder is prioritized over the one detected in the PhysX folder 2) many install the Cuda Toolkit and this path is also interpreted by ollama as higher priority at startup 3) some people (like me) use portable ollama and run it from the same folder where they store individual instances of the nvidia libraries (when running portable ollama from a folder, that folder is assigned to cmd.exe as a working directory, which is automatically added to the environment variables during startup). This way of starting is also a higher priority for ollama, I guess because the cudart library in the PhysX folder is usually more recent than the ones used next to the portable ollama (and ollama always prefers to start trying to run with the oldest version found -- and that will fire later). Everything would be fine for now if there was no such thing as PhysX Legacy, which is installed together with some old games and applications. ![physx legacy](https://github.com/ollama/ollama/assets/17222170/cabbee71-6465-4946-8981-6f2656c4094a) In this case, it turns out that the portable ollama on run prioritizes the cudart from the PhysX Legacy folder even over the cudart in the folder where the portable ollama .exe running from. This leads to an unhandled error and a crash of the ollama server (exact case of #3593) because once it finds the cudart in the PhysX folder it cannot then discover expected cublas and cublasLt libs needed for the LLM to work. To summarize: the current implementation of gpu.go may shoot up in the future. Even now, it creates malfunctions, provided 3 conditions are met: 1) using portable ollama (ollama-windows-amd64.exe) with nvidia libs alongside, in the same folder 2) you have not installed cuda toolkit 3) by pure coincidence you have PhysX Legacy installed (or) the current PhysX is installed right now along with the driver and after some future update ollama suddenly prioritizes an attempt to run through cudart from the PhysX folder. In any case, trying to index the PhysX folder as legitimately containing the necessary Nvidia libraries is an undesigned behavior leading to crashes on startup of chatting session. There are 2 required libraries missing in any version of PhysX. Also, the proposed code checks for the presence of these very libraries in the scanned folders to avoid similar cases, even not related to PhysX. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-16 05:38:30 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
amd
api
app
bug
build
cli
cloud
compatibility
context-length
create
docker
documentation
embeddings
feature request
feedback wanted
good first issue
gpt-oss
gpu
harmony
help wanted
image
install
intel
js
launch
linux
macos
memory
mlx
model
needs more info
networking
nvidia
ollama.com
performance
pull-request
Mirrored from GitHub Pull Request
 python
question
registry
rendering
thinking
tools
top
vulkan
windows
wsl
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/ollama#16652
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?