[GH-ISSUE #11188] Docker Script to Use GPUs, Ollama, and Enable Cuda #54802

New Issue

GiteaMirror · 2026-05-05T16:44:27-05:00

GiteaMirror commented

2026-05-05 16:44:27 -05:00

Originally created by @mattvisa on GitHub (Mar 5, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/11188

Bug Report

Utilize cuda along with install script that bundles Ollama.

Important Notes

Installation Method

I utilized this method to install:
docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart unless-stopped ghcr.io/open-webui/open-webui:ollama

I am using an RTX 4080 graphics card on Linux Mint and have installed the Nvidia toolkit, etc.

I looked in portainer and see that enable Docker Cuda is not set to TRUE. I set it to true, but am still using this image: ghcr.io/open-webui/open-webui:ollama. I don't see that when I run a model that my GPU usage jumps, but I do see my CPU usage go up.

I then switched the above image to ghcr.io/open-webui/open-webui:cuda and am not able to download any ollama models. I understand what I am doing, but I want to know if there is a way to preset using cuda AND also the packaged ollama install via docker.

I tried to install Ollama via a separate docker instance and reinstalling ghcr.io/open-webui/open-webui:cuda and pointing it to the port that ollama takes but had no luck that way. This might not be a bug and instead a feature request.

Environment

Open WebUI Version: ghcr.io/open-webui/open-webui:ollama as of 8:58pm this evening.
Operating System: Linux Mint 22.1

Confirmation:

[ *] I have read and followed all the instructions provided in the README.md.
[* ] I am on the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
[ *] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.

Expected Behavior:

I would be able to run docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart unless-stopped ghcr.io/open-webui/open-webui:CUDA_AND_OLLAMA

Actual Behavior:

When I installed it via

docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart unless-stopped ghcr.io/open-webui/open-webui:ollama

I could not seemingly utilize cuda as well.

Description

Bug Summary:

docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart unless-stopped ghcr.io/open-webui/open-webui:ollama

I just want to also be able to enable cuda with this script above as well.

Originally created by @mattvisa on GitHub (Mar 5, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/11188 # Bug Report Utilize cuda along with install script that bundles Ollama. ## Important Notes --- ## Installation Method I utilized this method to install: docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart unless-stopped ghcr.io/open-webui/open-webui:ollama I am using an RTX 4080 graphics card on Linux Mint and have installed the Nvidia toolkit, etc. I looked in portainer and see that enable Docker Cuda is not set to TRUE. I set it to true, but am still using this image: ghcr.io/open-webui/open-webui:ollama. I don't see that when I run a model that my GPU usage jumps, but I do see my CPU usage go up. I then switched the above image to ghcr.io/open-webui/open-webui:cuda and am not able to download any ollama models. I understand what I am doing, but I want to know if there is a way to preset using cuda AND also the packaged ollama install via docker. I tried to install Ollama via a separate docker instance and reinstalling ghcr.io/open-webui/open-webui:cuda and pointing it to the port that ollama takes but had no luck that way. This might not be a bug and instead a feature request. ## Environment - **Open WebUI Version:** ghcr.io/open-webui/open-webui:ollama as of 8:58pm this evening. - **Operating System:** Linux Mint 22.1 **Confirmation:** - [ *] I have read and followed all the instructions provided in the README.md. - [* ] I am on the latest version of both Open WebUI and Ollama. - [ ] I have included the browser console logs. - [ ] I have included the Docker container logs. - [ *] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below. ## Expected Behavior: I would be able to run docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart unless-stopped ghcr.io/open-webui/open-webui:CUDA_AND_OLLAMA ## Actual Behavior: When I installed it via docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart unless-stopped ghcr.io/open-webui/open-webui:ollama I could not seemingly utilize cuda as well. ## Description **Bug Summary:** docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart unless-stopped ghcr.io/open-webui/open-webui:ollama I just want to also be able to enable cuda with this script above as well.

GiteaMirror closed this issue

2026-05-05 16:44:28 -05:00

GiteaMirror commented

2026-05-05 16:44:30 -05:00

@mattvisa commented on GitHub (Mar 5, 2025):

Screenshot from portainer attached to make this more simple what I am seeing/requesting:

@mattvisa commented on GitHub (Mar 5, 2025): Screenshot from portainer attached to make this more simple what I am seeing/requesting: ![Image](https://github.com/user-attachments/assets/96a7deee-cee1-46f5-baae-d77691c78ce6)

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#54802