Is your feature request related to a problem? Please describe.
We have seen a few instances where a user's speech input is garbled and interpreted by OpenAI speech-to-text as another language. This causes incorrect input, made even worse because then OpenAI responds in that other language for all subsequent messages.
Describe the solution you'd like
Could we modify the input to the OpenAI API to provide the language from the user's settings?
Additional context
According to the OpenAI docs, providing the language may also improve performance:
Supplying the input language in ISO-639-1 format will improve accuracy and latency.
Originally created by @mike-stewart on GitHub (Nov 13, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/6926
# Feature Request
**Is your feature request related to a problem? Please describe.**
We have seen a few instances where a user's speech input is garbled and interpreted by OpenAI speech-to-text as another language. This causes incorrect input, made even worse because then OpenAI responds in that other language for all subsequent messages.
**Describe the solution you'd like**
Could we modify the input to the OpenAI API to provide the language from the user's settings?
Based on the [OpenAI docs](https://platform.openai.com/docs/api-reference/audio/createTranscription), it looks like we'd just need to provide the language parameter here:
https://github.com/open-webui/open-webui/blob/96c865404d36637eafadb6d2dd2365c85d452648/backend/open_webui/apps/audio/main.py#L434
**Describe alternatives you've considered**
N/A
**Additional context**
According to the OpenAI docs, providing the language may also improve performance:
> Supplying the input language in [ISO-639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes) format will improve accuracy and latency.
Could I also suggest it for local, along with the VAD (voice activity detection) filter? It really improves quality, and the VAD almost completely suppresses the annoying hallucinations that whisper usually has.
I'm currently using it via pip just for that reason: segments, info = model.transcribe(file_path, language="en", beam_size=5, vad_filter=True)
<!-- gh-comment-id:2558172795 -->
@nengoxx commented on GitHub (Dec 21, 2024):
Could I also suggest it for local, along with the VAD (voice activity detection) filter? It really improves quality, and the VAD almost completely suppresses the annoying hallucinations that whisper usually has.
It's in line 470:
https://github.com/open-webui/open-webui/blob/29a271959556743e6deb4d55a5a982983335d7ab/backend/open_webui/apps/audio/main.py#L470
I'm currently using it via pip just for that reason: `segments, info = model.transcribe(file_path, language="en", beam_size=5, vad_filter=True)`
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @mike-stewart on GitHub (Nov 13, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/6926
Feature Request
Is your feature request related to a problem? Please describe.
We have seen a few instances where a user's speech input is garbled and interpreted by OpenAI speech-to-text as another language. This causes incorrect input, made even worse because then OpenAI responds in that other language for all subsequent messages.
Describe the solution you'd like
Could we modify the input to the OpenAI API to provide the language from the user's settings?
Based on the OpenAI docs, it looks like we'd just need to provide the language parameter here:
https://github.com/open-webui/open-webui/blob/96c865404d36637eafadb6d2dd2365c85d452648/backend/open_webui/apps/audio/main.py#L434
Describe alternatives you've considered
N/A
Additional context
According to the OpenAI docs, providing the language may also improve performance:
@nengoxx commented on GitHub (Dec 21, 2024):
Could I also suggest it for local, along with the VAD (voice activity detection) filter? It really improves quality, and the VAD almost completely suppresses the annoying hallucinations that whisper usually has.
It's in line 470:
https://github.com/open-webui/open-webui/blob/29a271959556743e6deb4d55a5a982983335d7ab/backend/open_webui/apps/audio/main.py#L470
I'm currently using it via pip just for that reason:
segments, info = model.transcribe(file_path, language="en", beam_size=5, vad_filter=True)