github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-08-30 17:25:30 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[GH-ISSUE #3216] Failed to run CUDA within Windows Desktop #67779

New Issue
Closed
opened 2026-05-12 23:55:52 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-05-12 23:55:52 -05:00
Owner
Copy Link
Copy Source

Originally created by @albertmkc on GitHub (Jun 16, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/3216

Discussed in https://github.com/open-webui/open-webui/discussions/3213

Originally posted by albertmkc June 16, 2024
I used the command as suggested on Ollama.com to run locally in Windows Docker with CUDA, Ollama separately installed. Successful a month ago with my PC with RTX3090 and inference with GPU with no problem. 

Today, I accidentally deleted the container and using the same image pulled before, then it prompts me NVIDIA driver is not available.

I have reinstalled everything, e.g. Nvidia drivers from website, docker 4.31 (latest released), and even the entire system, it is still not working. I believe it had updated something from the PYTorch library but I have no Python/ Docker/ Linux knowledge to fix it.

Can someone help me out on this? Thanks!

2024-06-16 06:24:26 Traceback (most recent call last):
2024-06-16 06:24:26   File "/usr/local/bin/uvicorn", line 8, in <module>
2024-06-16 06:24:26     sys.exit(main())
2024-06-16 06:24:26              ^^^^^^
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
2024-06-16 06:24:26     return self.main(*args, **kwargs)
2024-06-16 06:24:26            ^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
2024-06-16 06:24:26     rv = self.invoke(ctx)
2024-06-16 06:24:26          ^^^^^^^^^^^^^^^^
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
2024-06-16 06:24:26     return ctx.invoke(self.callback, **ctx.params)
2024-06-16 06:24:26            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
2024-06-16 06:24:26     return __callback(*args, **kwargs)
2024-06-16 06:24:26            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 410, in main
2024-06-16 06:24:26     run(
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 578, in run
2024-06-16 06:24:26     server.run()
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 61, in run
2024-06-16 06:24:26     return asyncio.run(self.serve(sockets=sockets))
2024-06-16 06:24:26            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
2024-06-16 06:24:26     return runner.run(main)
2024-06-16 06:24:26            ^^^^^^^^^^^^^^^^
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
2024-06-16 06:24:26     return self._loop.run_until_complete(task)
2024-06-16 06:24:26            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-16 06:24:26   File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 68, in serve
2024-06-16 06:24:26     config.load()
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 473, in load
2024-06-16 06:24:26     self.loaded_app = import_from_string(self.app)
2024-06-16 06:24:26                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 21, in import_from_string
2024-06-16 06:24:26     module = importlib.import_module(module_str)
2024-06-16 06:24:26              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
2024-06-16 06:24:26     return _bootstrap._gcd_import(name[level:], package, level)
2024-06-16 06:24:26            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-16 06:24:26   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2024-06-16 06:24:26   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2024-06-16 06:24:26   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
2024-06-16 06:24:26   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2024-06-16 06:24:26   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2024-06-16 06:24:26   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2024-06-16 06:24:26   File "/app/backend/main.py", line 43, in <module>
2024-06-16 06:24:26     from apps.rag.main import app as rag_app
2024-06-16 06:24:26   File "/app/backend/apps/rag/main.py", line 207, in <module>
2024-06-16 06:24:26     update_embedding_model(
2024-06-16 06:24:26   File "/app/backend/apps/rag/main.py", line 184, in update_embedding_model
2024-06-16 06:24:26     app.state.sentence_transformer_ef = sentence_transformers.SentenceTransformer(
2024-06-16 06:24:26                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 221, in __init__
2024-06-16 06:24:26     self.to(device)
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1173, in to
2024-06-16 06:24:26     return self._apply(convert)
2024-06-16 06:24:26            ^^^^^^^^^^^^^^^^^^^^
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply
2024-06-16 06:24:26     module._apply(fn)
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply
2024-06-16 06:24:26     module._apply(fn)
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply
2024-06-16 06:24:26     module._apply(fn)
2024-06-16 06:24:26   [Previous line repeated 1 more time]
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 804, in _apply
2024-06-16 06:24:26     param_applied = fn(param)
2024-06-16 06:24:26                     ^^^^^^^^^
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1159, in convert
2024-06-16 06:24:26     return t.to(
2024-06-16 06:24:26            ^^^^^
2024-06-16 06:24:26   File "/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init
2024-06-16 06:24:26     torch._C._cuda_init()
2024-06-16 06:24:26 RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx
```</div>
Originally created by @albertmkc on GitHub (Jun 16, 2024). Original GitHub issue: https://github.com/open-webui/open-webui/issues/3216 ### Discussed in https://github.com/open-webui/open-webui/discussions/3213 <div type='discussions-op-text'> <sup>Originally posted by **albertmkc** June 16, 2024</sup> I used the command as suggested on Ollama.com to run locally in Windows Docker with CUDA, Ollama separately installed. Successful a month ago with my PC with RTX3090 and inference with GPU with no problem.  Today, I accidentally deleted the container and using the same image pulled before, then it prompts me NVIDIA driver is not available. I have reinstalled everything, e.g. Nvidia drivers from website, docker 4.31 (latest released), and even the entire system, it is still not working. I believe it had updated something from the PYTorch library but I have no Python/ Docker/ Linux knowledge to fix it. Can someone help me out on this? Thanks! ``` 2024-06-16 06:24:26 Traceback (most recent call last): 2024-06-16 06:24:26 File "/usr/local/bin/uvicorn", line 8, in <module> 2024-06-16 06:24:26 sys.exit(main()) 2024-06-16 06:24:26 ^^^^^^ 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ 2024-06-16 06:24:26 return self.main(*args, **kwargs) 2024-06-16 06:24:26 ^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main 2024-06-16 06:24:26 rv = self.invoke(ctx) 2024-06-16 06:24:26 ^^^^^^^^^^^^^^^^ 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke 2024-06-16 06:24:26 return ctx.invoke(self.callback, **ctx.params) 2024-06-16 06:24:26 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke 2024-06-16 06:24:26 return __callback(*args, **kwargs) 2024-06-16 06:24:26 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 410, in main 2024-06-16 06:24:26 run( 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 578, in run 2024-06-16 06:24:26 server.run() 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 61, in run 2024-06-16 06:24:26 return asyncio.run(self.serve(sockets=sockets)) 2024-06-16 06:24:26 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run 2024-06-16 06:24:26 return runner.run(main) 2024-06-16 06:24:26 ^^^^^^^^^^^^^^^^ 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run 2024-06-16 06:24:26 return self._loop.run_until_complete(task) 2024-06-16 06:24:26 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-16 06:24:26 File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 68, in serve 2024-06-16 06:24:26 config.load() 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 473, in load 2024-06-16 06:24:26 self.loaded_app = import_from_string(self.app) 2024-06-16 06:24:26 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 21, in import_from_string 2024-06-16 06:24:26 module = importlib.import_module(module_str) 2024-06-16 06:24:26 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module 2024-06-16 06:24:26 return _bootstrap._gcd_import(name[level:], package, level) 2024-06-16 06:24:26 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-16 06:24:26 File "<frozen importlib._bootstrap>", line 1204, in _gcd_import 2024-06-16 06:24:26 File "<frozen importlib._bootstrap>", line 1176, in _find_and_load 2024-06-16 06:24:26 File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked 2024-06-16 06:24:26 File "<frozen importlib._bootstrap>", line 690, in _load_unlocked 2024-06-16 06:24:26 File "<frozen importlib._bootstrap_external>", line 940, in exec_module 2024-06-16 06:24:26 File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed 2024-06-16 06:24:26 File "/app/backend/main.py", line 43, in <module> 2024-06-16 06:24:26 from apps.rag.main import app as rag_app 2024-06-16 06:24:26 File "/app/backend/apps/rag/main.py", line 207, in <module> 2024-06-16 06:24:26 update_embedding_model( 2024-06-16 06:24:26 File "/app/backend/apps/rag/main.py", line 184, in update_embedding_model 2024-06-16 06:24:26 app.state.sentence_transformer_ef = sentence_transformers.SentenceTransformer( 2024-06-16 06:24:26 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 221, in __init__ 2024-06-16 06:24:26 self.to(device) 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1173, in to 2024-06-16 06:24:26 return self._apply(convert) 2024-06-16 06:24:26 ^^^^^^^^^^^^^^^^^^^^ 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply 2024-06-16 06:24:26 module._apply(fn) 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply 2024-06-16 06:24:26 module._apply(fn) 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply 2024-06-16 06:24:26 module._apply(fn) 2024-06-16 06:24:26 [Previous line repeated 1 more time] 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 804, in _apply 2024-06-16 06:24:26 param_applied = fn(param) 2024-06-16 06:24:26 ^^^^^^^^^ 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1159, in convert 2024-06-16 06:24:26 return t.to( 2024-06-16 06:24:26 ^^^^^ 2024-06-16 06:24:26 File "/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init 2024-06-16 06:24:26 torch._C._cuda_init() 2024-06-16 06:24:26 RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx ```</div>
GiteaMirror commented 2026-05-12 23:55:55 -05:00
Author
Owner
Copy Link
Copy Source

@pkeffect commented on GitHub (Jun 16, 2024):

Did you also install

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

This is needed for Docker to talk to your gpu.

Edited for correct link.

<!-- gh-comment-id:2171746669 --> @pkeffect commented on GitHub (Jun 16, 2024): Did you also install https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html This is needed for Docker to talk to your gpu. Edited for correct link.
GiteaMirror commented 2026-05-12 23:55:55 -05:00
Author
Owner
Copy Link
Copy Source

@tjbck commented on GitHub (Jun 16, 2024):

#2527

<!-- gh-comment-id:2171754354 --> @tjbck commented on GitHub (Jun 16, 2024): #2527
GiteaMirror commented 2026-05-12 23:55:56 -05:00
Author
Owner
Copy Link
Copy Source

@albertmkc commented on GitHub (Jul 7, 2024):

Thanks! Eventually it was the docker version which is not compatible. After upgrade (requires reinstall of SD) it is working now! Thanks for support.

<!-- gh-comment-id:2212316006 --> @albertmkc commented on GitHub (Jul 7, 2024): Thanks! Eventually it was the docker version which is not compatible. After upgrade (requires reinstall of SD) it is working now! Thanks for support.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#67779