I have searched the existing issues and discussions.
Problem Description
Currently, Whisper STT in OpenWebUI runs on PyTorch by default, which may not be fully optimized for Intel architectures.
Desired Solution you'd like
This RFC proposes integrating OpenVINO as an optimized backend for Whisper (Speech-to-Text module) in OpenWebUI when an Intel CPU/GPU/NPU is detected. This enhancement will significantly improve inference speed and efficiency, particularly for users running OpenWebUI on Intel hardware.
In backend/open_webui/routers/audio.py, I can add a implementation for OpenVINO.
Originally created by @xiangyang-95 on GitHub (Mar 25, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/12024
### Check Existing Issues
- [x] I have searched the existing issues and discussions.
### Problem Description
Currently, Whisper STT in OpenWebUI runs on PyTorch by default, which may not be fully optimized for Intel architectures.
### Desired Solution you'd like
This RFC proposes integrating OpenVINO as an optimized backend for Whisper (Speech-to-Text module) in OpenWebUI when an Intel CPU/GPU/NPU is detected. This enhancement will significantly improve inference speed and efficiency, particularly for users running OpenWebUI on Intel hardware.
In `backend/open_webui/routers/audio.py`, I can add a implementation for OpenVINO.
```
if request.app.state.config.STT_ENGINE == "":
request.app.state.faster_whisper_model = set_faster_whisper_model(
form_data.stt.WHISPER_MODEL, WHISPER_MODEL_AUTO_UPDATE
)
elif request.app.state.config.STT_ENGINE == "openvino":
request.app.state.faster_whisper_model = set_openvino_whisper_model(
form_data.stt.WHISPER_MODEL, WHISPER_MODEL_AUTO_UPDATE
)
```
### Alternatives Considered
_No response_
### Additional Context
_No response_
Hi @tjbck, I am willing to contribute to this feature if you think this is a good one to add.
<!-- gh-comment-id:2774658354 -->
@xiangyang-95 commented on GitHub (Apr 3, 2025):
Hi @tjbck, I am willing to contribute to this feature if you think this is a good one to add.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @xiangyang-95 on GitHub (Mar 25, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/12024
Check Existing Issues
Problem Description
Currently, Whisper STT in OpenWebUI runs on PyTorch by default, which may not be fully optimized for Intel architectures.
Desired Solution you'd like
This RFC proposes integrating OpenVINO as an optimized backend for Whisper (Speech-to-Text module) in OpenWebUI when an Intel CPU/GPU/NPU is detected. This enhancement will significantly improve inference speed and efficiency, particularly for users running OpenWebUI on Intel hardware.
In
backend/open_webui/routers/audio.py, I can add a implementation for OpenVINO.Alternatives Considered
No response
Additional Context
No response
@xiangyang-95 commented on GitHub (Apr 3, 2025):
Hi @tjbck, I am willing to contribute to this feature if you think this is a good one to add.