I have read and followed all the instructions provided in the README.md.
I am on the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
Expected Behavior:
It should uses the GPU to to STT.
Actual Behavior:
It uses the CPU, the GPU remains unused.
Description
Bug Summary:
When using the local Whisper model for Speech-To-Text, the GPU is not being used.
Interestingly, it is detected properly as the output from nvidia-smi from inside the container shows:
❯ sudo docker exec openwebui nvidia-smi
Mon Feb 24 09:49:19 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.144.03 Driver Version: 550.144.03 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 2070 ... Off | 00000000:27:00.0 On | N/A |
| 28% 49C P3 30W / 215W | 503MiB / 8192MiB | 4% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
+-----------------------------------------------------------------------------------------+
Reproduction Details
Steps to Reproduce:
Install the cuda image using Docker compose
Set the STT to Whisper(Local)
Try to speak.
Logs and Screenshots
Docker Container Logs:
INFO [open_webui.routers.audio] file.content_type: audio/wav
INFO [open_webui.routers.audio] Detected language 'en' with probability 0.946448
transcribe /app/backend/data/cache/audio/transcriptions/170d5ccd-c91e-422b-ba14-17d6c25845aa.wav
INFO: 84.20.235.129:0 - "POST /api/v1/audio/transcriptions HTTP/1.1" 200 OK
INFO: 84.20.235.129:0 - "POST /api/v1/chats/new HTTP/1.1" 200 OK
INFO: 84.20.235.129:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK
INFO: 84.20.235.129:0 - "POST /api/v1/chats/4cec67d1-4871-4d04-9edf-b1683f2f3a9a HTTP/1.1" 200 OK
INFO: 84.20.235.129:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK
[pipe] Using grounding search.
INFO: 84.20.235.129:0 - "POST /api/chat/completions HTTP/1.1" 200 OK
INFO: 84.20.235.129:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK
Device set to use cpu
Originally created by @filiptrplan on GitHub (Feb 24, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/10673
# Bug Report
## Installation Method
Docker Compose:
```yml
version: '3.0'
services:
openwebui:
image: ghcr.io/open-webui/open-webui:cuda
container_name: openwebui
ports:
- 8080:8080
volumes:
- /home/filip/.open-webui:/app/backend/data
environment:
- ENV=prod
env_file:
- .env
restart: unless-stopped
extra_hosts:
- "host.docker.internal:host-gateway"
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
tika:
image: apache/tika:latest
container_name: tika
ports:
- "9998:9998"
restart: unless-stopped
```
## Environment
- **Open WebUI Version:** v0.5.16
- **Operating System:** Linux Mint 22
**Confirmation:**
- [x] I have read and followed all the instructions provided in the README.md.
- [x] I am on the latest version of both Open WebUI and Ollama.
- [x] I have included the browser console logs.
- [x] I have included the Docker container logs.
- [x] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
## Expected Behavior:
It should uses the GPU to to STT.
## Actual Behavior:
It uses the CPU, the GPU remains unused.
## Description
**Bug Summary:**
When using the local Whisper model for Speech-To-Text, the GPU is not being used.
Interestingly, it is detected properly as the output from `nvidia-smi` from inside the container shows:
```
❯ sudo docker exec openwebui nvidia-smi
Mon Feb 24 09:49:19 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.144.03 Driver Version: 550.144.03 CUDA Version: 12.4 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 2070 ... Off | 00000000:27:00.0 On | N/A |
| 28% 49C P3 30W / 215W | 503MiB / 8192MiB | 4% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
+-----------------------------------------------------------------------------------------+
```
## Reproduction Details
**Steps to Reproduce:**
- Install the cuda image using Docker compose
- Set the STT to Whisper(Local)
- Try to speak.
## Logs and Screenshots
**Docker Container Logs:**
```
INFO [open_webui.routers.audio] file.content_type: audio/wav
INFO [open_webui.routers.audio] Detected language 'en' with probability 0.946448
transcribe /app/backend/data/cache/audio/transcriptions/170d5ccd-c91e-422b-ba14-17d6c25845aa.wav
INFO: 84.20.235.129:0 - "POST /api/v1/audio/transcriptions HTTP/1.1" 200 OK
INFO: 84.20.235.129:0 - "POST /api/v1/chats/new HTTP/1.1" 200 OK
INFO: 84.20.235.129:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK
INFO: 84.20.235.129:0 - "POST /api/v1/chats/4cec67d1-4871-4d04-9edf-b1683f2f3a9a HTTP/1.1" 200 OK
INFO: 84.20.235.129:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK
[pipe] Using grounding search.
INFO: 84.20.235.129:0 - "POST /api/chat/completions HTTP/1.1" 200 OK
INFO: 84.20.235.129:0 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 OK
Device set to use cpu
```
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @filiptrplan on GitHub (Feb 24, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/10673
Bug Report
Installation Method
Docker Compose:
Environment
Confirmation:
Expected Behavior:
It should uses the GPU to to STT.
Actual Behavior:
It uses the CPU, the GPU remains unused.
Description
Bug Summary:
When using the local Whisper model for Speech-To-Text, the GPU is not being used.
Interestingly, it is detected properly as the output from
nvidia-smifrom inside the container shows:Reproduction Details
Steps to Reproduce:
Logs and Screenshots
Docker Container Logs:
@rgaricano commented on GitHub (Feb 24, 2025):
set env variable:
DEVICE_TYPE="cuda"
(or for cpu: DEVICE_TYPE="cpu" )