github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-05-06 10:58:17 -05:00
Code Issues 1.3k Packages Projects Releases 126 Wiki Activity

[GH-ISSUE #2527] issue: Unable to run CUDA variant #12917

New Issue
Closed
opened 2026-04-19 19:44:55 -05:00 by GiteaMirror · 14 comments
GiteaMirror commented 2026-04-19 19:44:55 -05:00
Owner
Copy Link

Originally created by @netphantom on GitHub (May 23, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/2527

Bug Report

Description

The command shown in the README does not allow to run the open-webui version with CUDA support
Bug Summary:
[Provide a brief but clear summary of the bug]
I run the command:
docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:cuda

But the container returns an error:

Loading WEBUI_SECRET_KEY from file, not provided as an environment variable.

Loading WEBUI_SECRET_KEY from .webui_secret_key

CUDA is enabled, appending LD_LIBRARY_PATH to include torch/cudnn & cublas libraries.

Traceback (most recent call last):

  File "/usr/local/bin/uvicorn", line 8, in <module>

    sys.exit(main())

             ^^^^^^

  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__

    return self.main(*args, **kwargs)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main

    rv = self.invoke(ctx)

         ^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke

    return ctx.invoke(self.callback, **ctx.params)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke

    return __callback(*args, **kwargs)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 410, in main

    run(

  File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 578, in run

    server.run()

  File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 61, in run

    return asyncio.run(self.serve(sockets=sockets))

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run

    return runner.run(main)

           ^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run

    return self._loop.run_until_complete(task)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete

  File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 68, in serve

    config.load()

  File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 473, in load

    self.loaded_app = import_from_string(self.app)

                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 21, in import_from_string

    module = importlib.import_module(module_str)

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module

    return _bootstrap._gcd_import(name[level:], package, level)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import

  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load

  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked

  File "<frozen importlib._bootstrap_external>", line 940, in exec_module

  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed

  File "/app/backend/main.py", line 33, in <module>

    from apps.rag.main import app as rag_app

  File "/app/backend/apps/rag/main.py", line 165, in <module>

    update_embedding_model(

  File "/app/backend/apps/rag/main.py", line 142, in update_embedding_model

    app.state.sentence_transformer_ef = sentence_transformers.SentenceTransformer(

                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 221, in __init__

    self.to(device)

  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1173, in to

    return self._apply(convert)

           ^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply

    module._apply(fn)

  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply

    module._apply(fn)

  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply

    module._apply(fn)

  [Previous line repeated 1 more time]

  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 804, in _apply

    param_applied = fn(param)

                    ^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1159, in convert

    return t.to(

           ^^^^^

  File "/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init

    torch._C._cuda_init()

Loading WEBUI_SECRET_KEY from file, not provided as an environment variable.

Loading WEBUI_SECRET_KEY from .webui_secret_key

CUDA is enabled, appending LD_LIBRARY_PATH to include torch/cudnn & cublas libraries.

Traceback (most recent call last):

  File "/usr/local/bin/uvicorn", line 8, in <module>

    sys.exit(main())

             ^^^^^^

  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__

    return self.main(*args, **kwargs)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main

    rv = self.invoke(ctx)

         ^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke

    return ctx.invoke(self.callback, **ctx.params)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke

    return __callback(*args, **kwargs)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 410, in main

    run(

  File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 578, in run

    server.run()

  File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 61, in run

    return asyncio.run(self.serve(sockets=sockets))

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run

    return runner.run(main)

           ^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run

    return self._loop.run_until_complete(task)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete

  File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 68, in serve

    config.load()

  File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 473, in load

    self.loaded_app = import_from_string(self.app)

                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 21, in import_from_string

    module = importlib.import_module(module_str)

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module

    return _bootstrap._gcd_import(name[level:], package, level)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import

  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load

  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked

  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked

  File "<frozen importlib._bootstrap_external>", line 940, in exec_module

  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed

  File "/app/backend/main.py", line 33, in <module>

    from apps.rag.main import app as rag_app

  File "/app/backend/apps/rag/main.py", line 165, in <module>

    update_embedding_model(

  File "/app/backend/apps/rag/main.py", line 142, in update_embedding_model

    app.state.sentence_transformer_ef = sentence_transformers.SentenceTransformer(

                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 221, in __init__

    self.to(device)

  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1173, in to

    return self._apply(convert)

           ^^^^^^^^^^^^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply

    module._apply(fn)

  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply

    module._apply(fn)

  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply

    module._apply(fn)

  [Previous line repeated 1 more time]

  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 804, in _apply

    param_applied = fn(param)

                    ^^^^^^^^^

  File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1159, in convert

    return t.to(

           ^^^^^

  File "/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init

    torch._C._cuda_init()

RuntimeError: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero.

Steps to Reproduce:
Just run the command in the README.md.

Expected Behavior:
The delpoyed container with CUDA support

Actual Behavior:
The container crashes and does not startup

Environment

  • Open WebUI Version: latest

  • Ollama (if applicable): 0.1.31

  • Operating System: Manjaro

  • Browser (if applicable): [e.g., Chrome 100.0, Firefox 98.0]

Reproduction Details

Confirmation:

  • [ x] I have read and followed all the instructions provided in the README.md.
  • [ x] I am on the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • [ x] I have included the Docker container logs.

Logs and Screenshots

Browser Console Logs:
[Include relevant browser console logs, if applicable]

Docker Container Logs:
See Bug error

Screenshots (if applicable):
[Attach any relevant screenshots to help illustrate the issue]

Installation Method

CLI

Additional Information

[Include any additional details that may help in understanding and reproducing the issue. This could include specific configurations, error messages, or anything else relevant to the bug.]

Note

If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!

Originally created by @netphantom on GitHub (May 23, 2024). Original GitHub issue: https://github.com/open-webui/open-webui/issues/2527 # Bug Report ## Description The command shown in the README does not allow to run the open-webui version with CUDA support **Bug Summary:** [Provide a brief but clear summary of the bug] I run the command: `docker run -d -p 3000:8080 --gpus all --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:cuda` But the container returns an error: ```shell Loading WEBUI_SECRET_KEY from file, not provided as an environment variable. Loading WEBUI_SECRET_KEY from .webui_secret_key CUDA is enabled, appending LD_LIBRARY_PATH to include torch/cudnn & cublas libraries. Traceback (most recent call last): File "/usr/local/bin/uvicorn", line 8, in <module> sys.exit(main()) ^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 410, in main run( File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 578, in run server.run() File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 61, in run return asyncio.run(self.serve(sockets=sockets)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 68, in serve config.load() File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 473, in load self.loaded_app = import_from_string(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 21, in import_from_string module = importlib.import_module(module_str) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1204, in _gcd_import File "<frozen importlib._bootstrap>", line 1176, in _find_and_load File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 690, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 940, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/app/backend/main.py", line 33, in <module> from apps.rag.main import app as rag_app File "/app/backend/apps/rag/main.py", line 165, in <module> update_embedding_model( File "/app/backend/apps/rag/main.py", line 142, in update_embedding_model app.state.sentence_transformer_ef = sentence_transformers.SentenceTransformer( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 221, in __init__ self.to(device) File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1173, in to return self._apply(convert) ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply module._apply(fn) File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply module._apply(fn) File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply module._apply(fn) [Previous line repeated 1 more time] File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 804, in _apply param_applied = fn(param) ^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1159, in convert return t.to( ^^^^^ File "/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init torch._C._cuda_init() Loading WEBUI_SECRET_KEY from file, not provided as an environment variable. Loading WEBUI_SECRET_KEY from .webui_secret_key CUDA is enabled, appending LD_LIBRARY_PATH to include torch/cudnn & cublas libraries. Traceback (most recent call last): File "/usr/local/bin/uvicorn", line 8, in <module> sys.exit(main()) ^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 410, in main run( File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 578, in run server.run() File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 61, in run return asyncio.run(self.serve(sockets=sockets)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 68, in serve config.load() File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 473, in load self.loaded_app = import_from_string(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 21, in import_from_string module = importlib.import_module(module_str) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1204, in _gcd_import File "<frozen importlib._bootstrap>", line 1176, in _find_and_load File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 690, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 940, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/app/backend/main.py", line 33, in <module> from apps.rag.main import app as rag_app File "/app/backend/apps/rag/main.py", line 165, in <module> update_embedding_model( File "/app/backend/apps/rag/main.py", line 142, in update_embedding_model app.state.sentence_transformer_ef = sentence_transformers.SentenceTransformer( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 221, in __init__ self.to(device) File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1173, in to return self._apply(convert) ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply module._apply(fn) File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply module._apply(fn) File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply module._apply(fn) [Previous line repeated 1 more time] File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 804, in _apply param_applied = fn(param) ^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1159, in convert return t.to( ^^^^^ File "/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init torch._C._cuda_init() RuntimeError: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. ``` **Steps to Reproduce:** Just run the command in the README.md. **Expected Behavior:** The delpoyed container with CUDA support **Actual Behavior:** The container crashes and does not startup ## Environment - **Open WebUI Version:** latest - **Ollama (if applicable):** 0.1.31 - **Operating System:** Manjaro - **Browser (if applicable):** [e.g., Chrome 100.0, Firefox 98.0] ## Reproduction Details **Confirmation:** - [ x] I have read and followed all the instructions provided in the README.md. - [ x] I am on the latest version of both Open WebUI and Ollama. - [ ] I have included the browser console logs. - [ x] I have included the Docker container logs. ## Logs and Screenshots **Browser Console Logs:** [Include relevant browser console logs, if applicable] **Docker Container Logs:** See Bug error **Screenshots (if applicable):** [Attach any relevant screenshots to help illustrate the issue] ## Installation Method CLI ## Additional Information [Include any additional details that may help in understanding and reproducing the issue. This could include specific configurations, error messages, or anything else relevant to the bug.] ## Note If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!
GiteaMirror commented 2026-04-19 19:44:56 -05:00
Author
Owner
Copy Link

@hhuseyincihangir commented on GitHub (May 23, 2024):

I'm having the same problem.

<!-- gh-comment-id:2127979529 --> @hhuseyincihangir commented on GitHub (May 23, 2024): I'm having the same problem.
GiteaMirror commented 2026-04-19 19:44:56 -05:00
Author
Owner
Copy Link

@justinh-rahb commented on GitHub (May 23, 2024):

Recent version of either Docker Desktop + Nvidia driver for Windows, or Docker runtime + Nvidia driver + Nvidia container toolkit for Linux are required for CUDA support to work in containers.

<!-- gh-comment-id:2128247530 --> @justinh-rahb commented on GitHub (May 23, 2024): Recent version of either Docker Desktop + Nvidia driver for Windows, or Docker runtime + Nvidia driver + [Nvidia container toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) for Linux are required for CUDA support to work in containers.
GiteaMirror commented 2026-04-19 19:44:56 -05:00
Author
Owner
Copy Link

@bigsk1 commented on GitHub (May 24, 2024):

Having same issue, was working a few days ago, new nvidia game ready drivers just were installed, my nvidia tool kit for linux hasn't changed - which was cuda-repo-wsl-ubuntu-12-2-local_12.2.1-1_amd64

 NVIDIA-SMI 555.42.03              Driver Version: 555.85         CUDA Version: 12.5

Docker logs last lines

File "/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init
2024-05-24 01:54:18     torch._C._cuda_init()
2024-05-24 01:54:18 RuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 500: named symbol not found
<!-- gh-comment-id:2129022023 --> @bigsk1 commented on GitHub (May 24, 2024): Having same issue, was working a few days ago, new nvidia game ready drivers just were installed, my nvidia tool kit for linux hasn't changed - which was cuda-repo-wsl-ubuntu-12-2-local_12.2.1-1_amd64 ``` NVIDIA-SMI 555.42.03 Driver Version: 555.85 CUDA Version: 12.5 ``` Docker logs last lines ``` File "/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init 2024-05-24 01:54:18 torch._C._cuda_init() 2024-05-24 01:54:18 RuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 500: named symbol not found ```
GiteaMirror commented 2026-04-19 19:44:57 -05:00
Author
Owner
Copy Link

@jannikstdl commented on GitHub (May 24, 2024):

Can you provide us the content of /etc/docker/daemon.json in your Linux or WSL Docker Environment?

<!-- gh-comment-id:2129146115 --> @jannikstdl commented on GitHub (May 24, 2024): Can you provide us the content of `/etc/docker/daemon.json` in your Linux or WSL Docker Environment?
GiteaMirror commented 2026-04-19 19:44:57 -05:00
Author
Owner
Copy Link

@netphantom commented on GitHub (May 24, 2024):

This is mine

{ "features": { "buildkit": true } }
<!-- gh-comment-id:2129410864 --> @netphantom commented on GitHub (May 24, 2024): This is mine ``` { "features": { "buildkit": true } } ```
GiteaMirror commented 2026-04-19 19:44:57 -05:00
Author
Owner
Copy Link

@kelheor commented on GitHub (May 27, 2024):

Having same issue, was working a few days ago, new nvidia game ready drivers just were installed, my nvidia tool kit for linux hasn't changed - which was cuda-repo-wsl-ubuntu-12-2-local_12.2.1-1_amd64

 NVIDIA-SMI 555.42.03              Driver Version: 555.85         CUDA Version: 12.5

Docker logs last lines

File "/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init
2024-05-24 01:54:18     torch._C._cuda_init()
2024-05-24 01:54:18 RuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 500: named symbol not found

I have similar issue:

2024-05-27 19:26:34 Traceback (most recent call last):
2024-05-27 19:26:34   File "/usr/local/bin/uvicorn", line 8, in <module>
2024-05-27 19:26:34     sys.exit(main())
2024-05-27 19:26:34              ^^^^^^
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
2024-05-27 19:26:34     return self.main(*args, **kwargs)
2024-05-27 19:26:34            ^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
2024-05-27 19:26:34     rv = self.invoke(ctx)
2024-05-27 19:26:34          ^^^^^^^^^^^^^^^^
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
2024-05-27 19:26:34     return ctx.invoke(self.callback, **ctx.params)
2024-05-27 19:26:34            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
2024-05-27 19:26:34     return __callback(*args, **kwargs)
2024-05-27 19:26:34            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 410, in main
2024-05-27 19:26:34     run(
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 578, in run
2024-05-27 19:26:34     server.run()
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 61, in run
2024-05-27 19:26:34     return asyncio.run(self.serve(sockets=sockets))
2024-05-27 19:26:34            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
2024-05-27 19:26:34     return runner.run(main)
2024-05-27 19:26:34            ^^^^^^^^^^^^^^^^
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
2024-05-27 19:26:34     return self._loop.run_until_complete(task)
2024-05-27 19:26:34            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-27 19:26:34   File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 68, in serve
2024-05-27 19:26:34     config.load()
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 473, in load
2024-05-27 19:26:34     self.loaded_app = import_from_string(self.app)
2024-05-27 19:26:34                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 21, in import_from_string
2024-05-27 19:26:34     module = importlib.import_module(module_str)
2024-05-27 19:26:34              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
2024-05-27 19:26:34     return _bootstrap._gcd_import(name[level:], package, level)
2024-05-27 19:26:34            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-27 19:26:34   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2024-05-27 19:26:34   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2024-05-27 19:26:34   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
2024-05-27 19:26:34   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2024-05-27 19:26:34   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2024-05-27 19:26:34   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2024-05-27 19:26:34   File "/app/backend/main.py", line 33, in <module>
2024-05-27 19:26:34     from apps.rag.main import app as rag_app
2024-05-27 19:26:34   File "/app/backend/apps/rag/main.py", line 165, in <module>
2024-05-27 19:26:34     update_embedding_model(
2024-05-27 19:26:34   File "/app/backend/apps/rag/main.py", line 142, in update_embedding_model
2024-05-27 19:26:34     app.state.sentence_transformer_ef = sentence_transformers.SentenceTransformer(
2024-05-27 19:26:34                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 221, in __init__
2024-05-27 19:26:34     self.to(device)
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1173, in to
2024-05-27 19:26:34     return self._apply(convert)
2024-05-27 19:26:34            ^^^^^^^^^^^^^^^^^^^^
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply
2024-05-27 19:26:34     module._apply(fn)
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply
2024-05-27 19:26:34     module._apply(fn)
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply
2024-05-27 19:26:34     module._apply(fn)
2024-05-27 19:26:34   [Previous line repeated 1 more time]
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 804, in _apply
2024-05-27 19:26:34     param_applied = fn(param)
2024-05-27 19:26:34                     ^^^^^^^^^
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1159, in convert
2024-05-27 19:26:34     return t.to(
2024-05-27 19:26:34            ^^^^^
2024-05-27 19:26:34   File "/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init
2024-05-27 19:26:34     torch._C._cuda_init()
2024-05-27 19:26:34 RuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 500: named symbol not found

I'm using WSL 2 / Ubuntu 22.04.3 LTS

<!-- gh-comment-id:2133599104 --> @kelheor commented on GitHub (May 27, 2024): > Having same issue, was working a few days ago, new nvidia game ready drivers just were installed, my nvidia tool kit for linux hasn't changed - which was cuda-repo-wsl-ubuntu-12-2-local_12.2.1-1_amd64 > > ``` > NVIDIA-SMI 555.42.03 Driver Version: 555.85 CUDA Version: 12.5 > ``` > > Docker logs last lines > > ``` > File "/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init > 2024-05-24 01:54:18 torch._C._cuda_init() > 2024-05-24 01:54:18 RuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 500: named symbol not found > ``` I have similar issue: ``` 2024-05-27 19:26:34 Traceback (most recent call last): 2024-05-27 19:26:34 File "/usr/local/bin/uvicorn", line 8, in <module> 2024-05-27 19:26:34 sys.exit(main()) 2024-05-27 19:26:34 ^^^^^^ 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ 2024-05-27 19:26:34 return self.main(*args, **kwargs) 2024-05-27 19:26:34 ^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main 2024-05-27 19:26:34 rv = self.invoke(ctx) 2024-05-27 19:26:34 ^^^^^^^^^^^^^^^^ 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke 2024-05-27 19:26:34 return ctx.invoke(self.callback, **ctx.params) 2024-05-27 19:26:34 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke 2024-05-27 19:26:34 return __callback(*args, **kwargs) 2024-05-27 19:26:34 ^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 410, in main 2024-05-27 19:26:34 run( 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 578, in run 2024-05-27 19:26:34 server.run() 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 61, in run 2024-05-27 19:26:34 return asyncio.run(self.serve(sockets=sockets)) 2024-05-27 19:26:34 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run 2024-05-27 19:26:34 return runner.run(main) 2024-05-27 19:26:34 ^^^^^^^^^^^^^^^^ 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run 2024-05-27 19:26:34 return self._loop.run_until_complete(task) 2024-05-27 19:26:34 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-05-27 19:26:34 File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 68, in serve 2024-05-27 19:26:34 config.load() 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 473, in load 2024-05-27 19:26:34 self.loaded_app = import_from_string(self.app) 2024-05-27 19:26:34 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 21, in import_from_string 2024-05-27 19:26:34 module = importlib.import_module(module_str) 2024-05-27 19:26:34 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module 2024-05-27 19:26:34 return _bootstrap._gcd_import(name[level:], package, level) 2024-05-27 19:26:34 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-05-27 19:26:34 File "<frozen importlib._bootstrap>", line 1204, in _gcd_import 2024-05-27 19:26:34 File "<frozen importlib._bootstrap>", line 1176, in _find_and_load 2024-05-27 19:26:34 File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked 2024-05-27 19:26:34 File "<frozen importlib._bootstrap>", line 690, in _load_unlocked 2024-05-27 19:26:34 File "<frozen importlib._bootstrap_external>", line 940, in exec_module 2024-05-27 19:26:34 File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed 2024-05-27 19:26:34 File "/app/backend/main.py", line 33, in <module> 2024-05-27 19:26:34 from apps.rag.main import app as rag_app 2024-05-27 19:26:34 File "/app/backend/apps/rag/main.py", line 165, in <module> 2024-05-27 19:26:34 update_embedding_model( 2024-05-27 19:26:34 File "/app/backend/apps/rag/main.py", line 142, in update_embedding_model 2024-05-27 19:26:34 app.state.sentence_transformer_ef = sentence_transformers.SentenceTransformer( 2024-05-27 19:26:34 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 221, in __init__ 2024-05-27 19:26:34 self.to(device) 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1173, in to 2024-05-27 19:26:34 return self._apply(convert) 2024-05-27 19:26:34 ^^^^^^^^^^^^^^^^^^^^ 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply 2024-05-27 19:26:34 module._apply(fn) 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply 2024-05-27 19:26:34 module._apply(fn) 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 779, in _apply 2024-05-27 19:26:34 module._apply(fn) 2024-05-27 19:26:34 [Previous line repeated 1 more time] 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 804, in _apply 2024-05-27 19:26:34 param_applied = fn(param) 2024-05-27 19:26:34 ^^^^^^^^^ 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1159, in convert 2024-05-27 19:26:34 return t.to( 2024-05-27 19:26:34 ^^^^^ 2024-05-27 19:26:34 File "/usr/local/lib/python3.11/site-packages/torch/cuda/__init__.py", line 293, in _lazy_init 2024-05-27 19:26:34 torch._C._cuda_init() 2024-05-27 19:26:34 RuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 500: named symbol not found ``` I'm using WSL 2 / Ubuntu 22.04.3 LTS
GiteaMirror commented 2026-04-19 19:44:58 -05:00
Author
Owner
Copy Link

@justinh-rahb commented on GitHub (May 27, 2024):

A number of people have reported issues with the recently released Nvidia driver and CUDA 12.5. We've seen reports of the issues being resolved if you downgrade to previously available versions.

<!-- gh-comment-id:2133633714 --> @justinh-rahb commented on GitHub (May 27, 2024): A number of people have reported issues with the recently released Nvidia driver and CUDA 12.5. We've seen reports of the issues being resolved if you downgrade to previously available versions.
GiteaMirror commented 2026-04-19 19:44:59 -05:00
Author
Owner
Copy Link

@kelheor commented on GitHub (May 27, 2024):

A number of people have reported issues with the recently released Nvidia driver and CUDA 12.5. We've seen reports of the issues being resolved if you downgrade to previously available versions.

I'm using CUDA 12.1
Any chance that it will be fixed on your side? Or better wait for the next driver from NVidia?

<!-- gh-comment-id:2133645976 --> @kelheor commented on GitHub (May 27, 2024): > A number of people have reported issues with the recently released Nvidia driver and CUDA 12.5. We've seen reports of the issues being resolved if you downgrade to previously available versions. I'm using CUDA 12.1 Any chance that it will be fixed on your side? Or better wait for the next driver from NVidia?
GiteaMirror commented 2026-04-19 19:44:59 -05:00
Author
Owner
Copy Link

@Erik262 commented on GitHub (May 29, 2024):

Same here, but it worked flawlessly before when using the ollama server separately instead of having everything in one container, which I would prefer.

In my case its even not looking for GPU at all. but nvidia-smi command shows GPU correctly.
image

<!-- gh-comment-id:2137329218 --> @Erik262 commented on GitHub (May 29, 2024): Same here, but it worked flawlessly before when using the ollama server separately instead of having everything in one container, which I would prefer. In my case its even not looking for GPU at all. but nvidia-smi command shows GPU correctly. ![image](https://github.com/open-webui/open-webui/assets/11977647/ad024c52-49a9-4d24-ac3f-2418755cae35)
GiteaMirror commented 2026-04-19 19:45:00 -05:00
Author
Owner
Copy Link

@zimdin12 commented on GitHub (May 29, 2024):

For me problems started after updating to nvidia driver version 555.85. confirmed in 2 PC's
I am now curious if ollama v.39 solves that problem (I shall give it a try)

Edit: i noticed that my error might be different. :)

<!-- gh-comment-id:2137847660 --> @zimdin12 commented on GitHub (May 29, 2024): For me problems started after updating to nvidia driver version 555.85. confirmed in 2 PC's I am now curious if ollama v.39 solves that problem (I shall give it a try) Edit: i noticed that my error might be different. :)
GiteaMirror commented 2026-04-19 19:45:01 -05:00
Author
Owner
Copy Link

@cliffwoolley commented on GitHub (May 29, 2024):

There might be multiple issues mixed here:

  • For those using WSL2 and NVIDIA Driver 555.xx reporting "named symbol not found" (@bigsk1 @kelheor @zimdin12 ), please see https://github.com/NVIDIA/nvidia-container-toolkit/issues/520 .

  • The OP from @netphantom however mentioned "CUDA unknown error" and did not specifically mention WSL2, which might therefore be different.

<!-- gh-comment-id:2138357971 --> @cliffwoolley commented on GitHub (May 29, 2024): There might be multiple issues mixed here: * For those using WSL2 and NVIDIA Driver 555.xx reporting "named symbol not found" (@bigsk1 @kelheor @zimdin12 ), please see https://github.com/NVIDIA/nvidia-container-toolkit/issues/520 . * The OP from @netphantom however mentioned "CUDA unknown error" and did not specifically mention WSL2, which might therefore be different.
GiteaMirror commented 2026-04-19 19:45:01 -05:00
Author
Owner
Copy Link

@bigsk1 commented on GitHub (May 30, 2024):

must be docker-desktop as referanced in the Nvidia Github, i just updated to 12.5 cuda toolkit on both wsl and windows. The fix is use older Nvidia driver or wait for bundled toolkit fix. Guess I'm running the :main :)

<!-- gh-comment-id:2138771308 --> @bigsk1 commented on GitHub (May 30, 2024): must be docker-desktop as referanced in the Nvidia Github, i just updated to 12.5 cuda toolkit on both wsl and windows. The fix is use older Nvidia driver or wait for bundled toolkit fix. Guess I'm running the :main :)
GiteaMirror commented 2026-04-19 19:45:02 -05:00
Author
Owner
Copy Link

@netphantom commented on GitHub (May 30, 2024):

There might be multiple issues mixed here:

* For those using WSL2 and NVIDIA Driver 555.xx reporting "named symbol not found" (@bigsk1 @kelheor @zimdin12 ), please see [NVIDIA Driver 555.85 with WSL2 returns error 500 "named symbol not found" when running CUDA apps inside container NVIDIA/nvidia-container-toolkit#520](https://github.com/NVIDIA/nvidia-container-toolkit/issues/520) .

* The OP from @netphantom however mentioned "CUDA unknown error" and did not specifically mention WSL2, which might therefore be different.

As reported in the first post, I am using Arch/Manjaro, no WSL for me :)

<!-- gh-comment-id:2138800990 --> @netphantom commented on GitHub (May 30, 2024): > There might be multiple issues mixed here: > > * For those using WSL2 and NVIDIA Driver 555.xx reporting "named symbol not found" (@bigsk1 @kelheor @zimdin12 ), please see [NVIDIA Driver 555.85 with WSL2 returns error 500 "named symbol not found" when running CUDA apps inside container NVIDIA/nvidia-container-toolkit#520](https://github.com/NVIDIA/nvidia-container-toolkit/issues/520) . > > * The OP from @netphantom however mentioned "CUDA unknown error" and did not specifically mention WSL2, which might therefore be different. As reported in the first post, I am using Arch/Manjaro, no WSL for me :)
GiteaMirror commented 2026-04-19 19:45:02 -05:00
Author
Owner
Copy Link

@HarukaRepo commented on GitHub (Sep 16, 2024):

Traceback (most recent call last):
  File "/usr/local/bin/uvicorn", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 410, in main
    run(
  File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 577, in run
    server.run()
  File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 65, in run
    return asyncio.run(self.serve(sockets=sockets))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete
  File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 69, in serve
    await self._serve(sockets)
  File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 76, in _serve
    config.load()
  File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 434, in load
    self.loaded_app = import_from_string(self.app)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 19, in import_from_string
    module = importlib.import_module(module_str)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/app/backend/open_webui/main.py", line 30, in <module>
    from open_webui.apps.rag.main import app as rag_app
  File "/app/backend/open_webui/apps/rag/main.py", line 208, in <module>
    update_embedding_model(
  File "/app/backend/open_webui/apps/rag/main.py", line 183, in update_embedding_model
    app.state.sentence_transformer_ef = sentence_transformers.SentenceTransformer(
                                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 287, in __init__
    modules = self._load_sbert_model(
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 1487, in _load_sbert_model
    module = Transformer(model_name_or_path, cache_dir=cache_folder, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 53, in __init__
    config = AutoConfig.from_pretrained(model_name_or_path, **config_args, cache_dir=cache_dir)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 979, in from_pretrained
    trust_remote_code = resolve_trust_remote_code(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/transformers/dynamic_module_utils.py", line 640, in resolve_trust_remote_code
    raise ValueError(
ValueError: Loading /app/backend/data/cache/embedding/models/models--nomic-ai--nomic-embed-text-v1/snapshots/cc62377b015c53a3bf52bb2f4eb8c55326d3f162 requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option `trust_remote_code=True` to remove this error.

Run on Geforce GTX 1660TI Mobile

image

<!-- gh-comment-id:2353037125 --> @HarukaRepo commented on GitHub (Sep 16, 2024): ``` Traceback (most recent call last): File "/usr/local/bin/uvicorn", line 8, in <module> sys.exit(main()) ^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 410, in main run( File "/usr/local/lib/python3.11/site-packages/uvicorn/main.py", line 577, in run server.run() File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 65, in run return asyncio.run(self.serve(sockets=sockets)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "uvloop/loop.pyx", line 1517, in uvloop.loop.Loop.run_until_complete File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 69, in serve await self._serve(sockets) File "/usr/local/lib/python3.11/site-packages/uvicorn/server.py", line 76, in _serve config.load() File "/usr/local/lib/python3.11/site-packages/uvicorn/config.py", line 434, in load self.loaded_app = import_from_string(self.app) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/uvicorn/importer.py", line 19, in import_from_string module = importlib.import_module(module_str) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "<frozen importlib._bootstrap>", line 1204, in _gcd_import File "<frozen importlib._bootstrap>", line 1176, in _find_and_load File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 690, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 940, in exec_module File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed File "/app/backend/open_webui/main.py", line 30, in <module> from open_webui.apps.rag.main import app as rag_app File "/app/backend/open_webui/apps/rag/main.py", line 208, in <module> update_embedding_model( File "/app/backend/open_webui/apps/rag/main.py", line 183, in update_embedding_model app.state.sentence_transformer_ef = sentence_transformers.SentenceTransformer( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 287, in __init__ modules = self._load_sbert_model( ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 1487, in _load_sbert_model module = Transformer(model_name_or_path, cache_dir=cache_folder, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 53, in __init__ config = AutoConfig.from_pretrained(model_name_or_path, **config_args, cache_dir=cache_dir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 979, in from_pretrained trust_remote_code = resolve_trust_remote_code( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/transformers/dynamic_module_utils.py", line 640, in resolve_trust_remote_code raise ValueError( ValueError: Loading /app/backend/data/cache/embedding/models/models--nomic-ai--nomic-embed-text-v1/snapshots/cc62377b015c53a3bf52bb2f4eb8c55326d3f162 requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option `trust_remote_code=True` to remove this error. ``` ## Run on Geforce GTX 1660TI Mobile ![image](https://github.com/user-attachments/assets/d4107c35-6872-4a2c-85c2-5a0fafabe27e)
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#12917
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?