github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-07-15 21:19:39 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[GH-ISSUE #2256] enhancement: better tts #83553

New Issue
Closed
opened 2026-05-15 08:33:43 -05:00 by GiteaMirror · 11 comments
GiteaMirror commented 2026-05-15 08:33:43 -05:00
Owner
Copy Link

Originally created by @LebToki on GitHub (May 14, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/2256

Problem:
The current implementation of Whisper in Open-WebUI uses a limited, robotic voice for all interactions.
While this is functional, it can be jarring and unnatural, making it difficult for users to engage with the interface.

Solution:
I would like to see the addition of more voices to Open-WebUI, specifically ones that are less robotic and more natural-sounding. This would improve the overall user experience and make the interface more enjoyable to interact with.

Alternatives considered:
I've considered using third-party voice libraries or integrating existing voice assistants, but these would require significant modifications to the Open-WebUI codebase. I've also considered using text-to-speech software, but these often lack the emotional expression and nuance of human speech.

Additional context:
The Whisper implementation in Open-WebUI is a great step forward in providing a more natural-sounding interface, but adding more voices would take it to the next level. This would be especially beneficial for users who rely heavily on voice interfaces for daily tasks, such as seniors or individuals with disabilities.

Some potential voices to consider adding include:

  • A softer, more gentle voice for more intimate interactions
  • A more assertive, commanding voice for tasks that require attention
  • A soothing, calming voice for relaxation and stress relief
  • A playful, whimsical voice for entertainment and leisure activities

By adding more voices to Open-WebUI, we can create a more engaging and immersive experience for users, making it easier for them to interact with the interface and achieve their goals.

Originally created by @LebToki on GitHub (May 14, 2024). Original GitHub issue: https://github.com/open-webui/open-webui/issues/2256 Problem: The current implementation of Whisper in Open-WebUI uses a limited, robotic voice for all interactions. While this is functional, it can be jarring and unnatural, making it difficult for users to engage with the interface. Solution: I would like to see the addition of more voices to Open-WebUI, specifically ones that are less robotic and more natural-sounding. This would improve the overall user experience and make the interface more enjoyable to interact with. Alternatives considered: I've considered using third-party voice libraries or integrating existing voice assistants, but these would require significant modifications to the Open-WebUI codebase. I've also considered using text-to-speech software, but these often lack the emotional expression and nuance of human speech. Additional context: The Whisper implementation in Open-WebUI is a great step forward in providing a more natural-sounding interface, but adding more voices would take it to the next level. This would be especially beneficial for users who rely heavily on voice interfaces for daily tasks, such as seniors or individuals with disabilities. Some potential voices to consider adding include: - A softer, more gentle voice for more intimate interactions - A more assertive, commanding voice for tasks that require attention - A soothing, calming voice for relaxation and stress relief - A playful, whimsical voice for entertainment and leisure activities By adding more voices to Open-WebUI, we can create a more engaging and immersive experience for users, making it easier for them to interact with the interface and achieve their goals.
GiteaMirror commented 2026-05-15 08:33:44 -05:00
Author
Owner
Copy Link

@zhewang1-intc commented on GitHub (May 14, 2024):

echo, i think this is a valuable feature.

<!-- gh-comment-id:2110429148 --> @zhewang1-intc commented on GitHub (May 14, 2024): echo, i think this is a valuable feature.
GiteaMirror commented 2026-05-15 08:33:46 -05:00
Author
Owner
Copy Link

@cheahjs commented on GitHub (May 14, 2024):

Whisper isn't a text to speech model, it handles speech to text.

Currently there's two options for text to speech on Open WebUI, one is generated locally on your browser using the Web Speech API, and another by calling out to an OpenAI-compatible text to speech API. The local one uses whatever the browser provides, which is typically the OS's text to speech models. On the OpenAI front, some in the community have deployed LocalAI or OpenedAI Speech to provide self-hosted TTS models.

<!-- gh-comment-id:2110461558 --> @cheahjs commented on GitHub (May 14, 2024): Whisper isn't a text to speech model, it handles speech to text. Currently there's two options for text to speech on Open WebUI, one is generated locally on your browser using the [Web Speech API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API), and another by calling out to an OpenAI-compatible text to speech API. The local one uses whatever the browser provides, which is typically the OS's text to speech models. On the OpenAI front, some in the community have deployed [LocalAI](https://localai.io/features/text-to-audio/) or [OpenedAI Speech](https://github.com/matatonic/openedai-speech) to provide self-hosted TTS models.
GiteaMirror commented 2026-05-15 08:33:46 -05:00
Author
Owner
Copy Link

@MichaelFomenko commented on GitHub (May 14, 2024):

Local Speech Models:

<!-- gh-comment-id:2110498219 --> @MichaelFomenko commented on GitHub (May 14, 2024): Local Speech Models: - SpeechT5: https://huggingface.co/microsoft/speecht5_tts - Tools: https://huggingface.co/collections/microsoft/speecht5-650995fc647a3ea442cc6c7b
GiteaMirror commented 2026-05-15 08:33:47 -05:00
Author
Owner
Copy Link

@boshk0 commented on GitHub (May 20, 2024):

OpenVoice would be great addition to OpenWebUI!
https://github.com/myshell-ai/OpenVoice

<!-- gh-comment-id:2121143589 --> @boshk0 commented on GitHub (May 20, 2024): OpenVoice would be great addition to OpenWebUI! https://github.com/myshell-ai/OpenVoice
GiteaMirror commented 2026-05-15 08:33:48 -05:00
Author
Owner
Copy Link

@kevin070982 commented on GitHub (Jun 3, 2024):

I can only recommend to use the solution provided by UXVirtual.
When you run Open WebUI in Docker then take a look here: https://github.com/open-webui/open-webui/issues/126#issuecomment-2078407480.

Easy Steps:
Clone https://github.com/matatonic/openedai-speech
Run docker compose -f docker-compose.min.yml up for a minimal docker image with only piper TTs support <1GB (CPU only)
or
Run docker compose up for the HD version but requires around 4GB GPU VRAM
Then add the below lines to the TTS Settings under Audio in the Open WebUI Settings.

API Base URL: http://host.docker.internal:8000/v1
API Key: sk-111111111

following voices can be changed under Set Voice:
alloy
echo
echo-alt
fable
onyx
nova
shimmer

Enjoy more naturally sounding voice.

<!-- gh-comment-id:2145367286 --> @kevin070982 commented on GitHub (Jun 3, 2024): I can only recommend to use the solution provided by [UXVirtual](https://github.com/UXVirtual). When you run Open WebUI in Docker then take a look here: https://github.com/open-webui/open-webui/issues/126#issuecomment-2078407480. Easy Steps: Clone https://github.com/matatonic/openedai-speech Run `docker compose -f docker-compose.min.yml up` for a minimal docker image with only piper TTs support <1GB (CPU only) or Run `docker compose up` for the HD version but requires around 4GB GPU VRAM Then add the below lines to the TTS Settings under Audio in the Open WebUI Settings. ``` API Base URL: http://host.docker.internal:8000/v1 API Key: sk-111111111 ``` following voices can be changed under Set Voice: alloy echo echo-alt fable onyx nova shimmer Enjoy more naturally sounding voice.
GiteaMirror commented 2026-05-15 08:33:49 -05:00
Author
Owner
Copy Link

@ErroneousBosch commented on GitHub (Jun 13, 2024):

There is a very nice solution in piper that has good language support and really high quality voices:
https://github.com/rhasspy/piper

<!-- gh-comment-id:2165595475 --> @ErroneousBosch commented on GitHub (Jun 13, 2024): There is a very nice solution in piper that has good language support and really high quality voices: https://github.com/rhasspy/piper
GiteaMirror commented 2026-05-15 08:33:51 -05:00
Author
Owner
Copy Link

@reneil1337 commented on GitHub (Jun 19, 2024):

I'd love to see one of the suggestions made here integrated. favorably the ability to integrated an entirely onprem solution with easy, more-straight forward cloud services as options/alternatives

<!-- gh-comment-id:2179282078 --> @reneil1337 commented on GitHub (Jun 19, 2024): I'd love to see one of the suggestions made here integrated. favorably the ability to integrated an entirely onprem solution with easy, more-straight forward cloud services as options/alternatives
GiteaMirror commented 2026-05-15 08:33:51 -05:00
Author
Owner
Copy Link

@justinh-rahb commented on GitHub (Jun 19, 2024):

I'd love to see one of the suggestions made here integrated. favorably the ability to integrated an entirely onprem solution with easy, more-straight forward cloud services as options/alternatives

We've for a while supported adding external OpenAI-compatible TTS and STT APIs now.

<!-- gh-comment-id:2179283594 --> @justinh-rahb commented on GitHub (Jun 19, 2024): > I'd love to see one of the suggestions made here integrated. favorably the ability to integrated an entirely onprem solution with easy, more-straight forward cloud services as options/alternatives We've for a while supported adding external OpenAI-compatible TTS and STT APIs now.
GiteaMirror commented 2026-05-15 08:33:52 -05:00
Author
Owner
Copy Link

@chrishart0 commented on GitHub (Jun 21, 2024):

Here's my setup for using openedai-speech which mocks the openai TTS api to run local models to generate text-to-speech.

My Repo: https://github.com/chrishart0/open-webui-nvidia-cuda-setup

The important part is I added a openedai-speech container in my docker-compose.yaml

  openedai-speech:
    image: ghcr.io/matatonic/openedai-speech
    container_name: openedai-speech
    # env_file: .env
    environment:
      - 'TTS_HOME=voices'
      - 'HF_HOME=voices'
      - 'PRELOAD_MODEL=xtts'
    ports:  # No need to expose ports, this is a backend service
      - "8087:8000"
    volumes:
      - ./data/openedai-speech/voices:/app/voices
      - ./data/openedai-speech/config:/app/config
    restart: unless-stopped

I populate the needed configs for openedai-speech, which is these two files pre_process_map.default.yaml, voice_to_speaker.default.yaml

Then I configure the local API in the TTS settings. In my repo I do this via an .env file, but you could just do it through the open web UI GUI too:
image

If you've got a GPU that supports CUDA you should be able to use my repo. If you have any questions just let me know.

<!-- gh-comment-id:2181935166 --> @chrishart0 commented on GitHub (Jun 21, 2024): Here's my setup for using [openedai-speech](https://github.com/matatonic/openedai-speech) which mocks the openai TTS api to run local models to generate text-to-speech. My Repo: https://github.com/chrishart0/open-webui-nvidia-cuda-setup The important part is I added a openedai-speech container in my [docker-compose.yaml]( https://github.com/chrishart0/open-webui-nvidia-cuda-setup/blob/master/docker-compose.yaml) ``` openedai-speech: image: ghcr.io/matatonic/openedai-speech container_name: openedai-speech # env_file: .env environment: - 'TTS_HOME=voices' - 'HF_HOME=voices' - 'PRELOAD_MODEL=xtts' ports: # No need to expose ports, this is a backend service - "8087:8000" volumes: - ./data/openedai-speech/voices:/app/voices - ./data/openedai-speech/config:/app/config restart: unless-stopped ``` I populate the needed configs for openedai-speech, which is these two files [pre_process_map.default.yaml](https://github.com/chrishart0/open-webui-nvidia-cuda-setup/blob/master/data/openedai-speech/config/pre_process_map.default.yaml), [voice_to_speaker.default.yaml](https://github.com/chrishart0/open-webui-nvidia-cuda-setup/blob/master/data/openedai-speech/config/voice_to_speaker.default.yaml) Then I configure the local API in the TTS settings. In my repo I do this via an .env file, but you could just do it through the open web UI GUI too: ![image](https://github.com/open-webui/open-webui/assets/64378248/26e8ad85-4329-40fe-b972-7c2d0fb07fe3) If you've got a GPU that supports CUDA you should be able to use my repo. If you have any questions just let me know.
GiteaMirror commented 2026-05-15 08:33:52 -05:00
Author
Owner
Copy Link

@PeterHagen commented on GitHub (Aug 4, 2024):

For additional information, Piper is also available through the Wyoming protocol, which can provide fast responses. This is implemented for use in Home Assistant. A Docker image is also available. For me Piper runs fine without a GPU. Might be an interesting option, in my opinion.

<!-- gh-comment-id:2267464585 --> @PeterHagen commented on GitHub (Aug 4, 2024): For additional information, Piper is also available through the [Wyoming protocol](https://github.com/rhasspy/wyoming-piper), which can provide fast responses. This is implemented for use in Home Assistant. A Docker image is also available. For me Piper runs fine without a GPU. Might be an interesting option, in my opinion.
GiteaMirror commented 2026-05-15 08:33:54 -05:00
Author
Owner
Copy Link

@thiswillbeyourgithub commented on GitHub (Aug 23, 2024):

Just fyi, the very fine folks over at openedai-speech are currently implementing automatic language detection for piper. Meaning it will transparently switch voices that are trained on the detected language in the sentence etc. It's in the dev branch

<!-- gh-comment-id:2306580414 --> @thiswillbeyourgithub commented on GitHub (Aug 23, 2024): Just fyi, the very fine folks over at [openedai-speech](https://github.com/matatonic/openedai-speech/pull/29) are currently implementing automatic language detection for piper. Meaning it will transparently switch voices that are trained on the detected language in the sentence etc. It's in the dev branch
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#83553
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?