I have read and followed all the instructions provided in the README.md.
I am on the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
Expected Behavior:
OpenWebUI changes the voice model when changed in settings.
Actual Behavior:
OpenWebUI doesn't change the voice model when changed in settings.
Description
Bug Summary:
When changing the CMU Artic speaker model the voice stays the same.
Reproduction Details
Steps to Reproduce:
In Settings > Audio set TTS Settings to Transformers (Local)
Set TTS Model to slt (US female)
Open new chat and test the voice
Change the TTS Model to bdl (US male)
Logs and Screenshots
Docker Container Logs:
___ __ __ _ _ _ ___
/ _ \ _ __ ___ _ __ \ \ / /__| |__ | | | |_ _|
| | | | '_ \ / _ \ '_ \ \ \ /\ / / _ \ '_ \| | | || |
| |_| | |_) | __/ | | | \ V V / __/ |_) | |_| || |
\___/| .__/ \___|_| |_| \_/\_/ \___|_.__/ \___/|___|
|_|
v0.4.8 - building the best open-source AI user interface.
https://github.com/open-webui/open-webui
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprMH- HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=websocket&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 400 Bad Request
INFO: 192.168.1.13:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=PEprMIA&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprMIA.0&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprMIR&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=PEprSOo&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /static/favicon.png HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/users/ HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/users/ HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprSOo.0&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=PEprYVZ&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprYVZ.0&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=PEprecL&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprecM&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=PEprkj8&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprkj8.0&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=PEprqpv&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /static/favicon.png HTTP/1.0" 304 Not Modified
INFO: 192.168.1.13:0 - "GET /api/v1/auths/admin/config HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/webhook HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/auths/admin/config/ldap/server HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/auths/admin/config/ldap HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /audio/api/v1/config HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /audio/api/v1/voices HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /audio/api/v1/models HTTP/1.0" 200 OK
INFO [open_webui.env] Saving 'AUDIO_TTS_OPENAI_API_BASE_URL' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_OPENAI_API_KEY' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_API_KEY' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_ENGINE' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_MODEL' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_VOICE' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_SPLIT_ON' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_AZURE_SPEECH_REGION' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_AZURE_SPEECH_OUTPUT_FORMAT' to the database
INFO [open_webui.env] Saving 'AUDIO_STT_OPENAI_API_BASE_URL' to the database
INFO [open_webui.env] Saving 'AUDIO_STT_OPENAI_API_KEY' to the database
INFO [open_webui.env] Saving 'AUDIO_STT_ENGINE' to the database
INFO [open_webui.env] Saving 'AUDIO_STT_MODEL' to the database
INFO [open_webui.env] Saving 'WHISPER_MODEL' to the database
INFO: 192.168.1.13:0 - "POST /audio/api/v1/config/update HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/config HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /static/favicon.png HTTP/1.0" 304 Not Modified
INFO: 192.168.1.13:0 - "GET /api/v1/chats/00381707-0a57-4ce0-b336-16ff7c6557a8 HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/chats/00381707-0a57-4ce0-b336-16ff7c6557a8/tags HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/users/user/settings HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ollama/api/version HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/chats/002cca7f-dfbb-40c2-8238-328e4e02befe HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/chats/002cca7f-dfbb-40c2-8238-328e4e02befe/tags HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/users/user/settings HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprqpv.0&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=PEprwwh&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "POST /audio/api/v1/speech HTTP/1.0" 200 OK
Originally created by @nphalem on GitHub (Dec 11, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/7779
# Bug Report
## Installation Method
Docker compose with searxng
## Environment
- **Open WebUI Version:** v0.4.8
- **Ollama (if applicable):** v.0.5.1
- **Operating System:** Arch Linux 6.12.4
**Confirmation:**
- [x] I have read and followed all the instructions provided in the README.md.
- [x] I am on the latest version of both Open WebUI and Ollama.
- [ ] I have included the browser console logs.
- [x] I have included the Docker container logs.
- [x] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
## Expected Behavior:
OpenWebUI changes the voice model when changed in settings.
## Actual Behavior:
OpenWebUI doesn't change the voice model when changed in settings.
## Description
**Bug Summary:**
When changing the CMU Artic speaker model the voice stays the same.
## Reproduction Details
**Steps to Reproduce:**
1. In **Settings** > **Audio** set **TTS Settings** to **Transformers (Local)**
2. Set **TTS Model** to `slt` (US female)
3. Open new chat and test the voice
4. Change the **TTS Model** to `bdl` (US male)
## Logs and Screenshots
**Docker Container Logs:**
```
___ __ __ _ _ _ ___
/ _ \ _ __ ___ _ __ \ \ / /__| |__ | | | |_ _|
| | | | '_ \ / _ \ '_ \ \ \ /\ / / _ \ '_ \| | | || |
| |_| | |_) | __/ | | | \ V V / __/ |_) | |_| || |
\___/| .__/ \___|_| |_| \_/\_/ \___|_.__/ \___/|___|
|_|
v0.4.8 - building the best open-source AI user interface.
https://github.com/open-webui/open-webui
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprMH- HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=websocket&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 400 Bad Request
INFO: 192.168.1.13:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=PEprMIA&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprMIA.0&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprMIR&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=PEprSOo&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /static/favicon.png HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/users/ HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/users/ HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprSOo.0&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=PEprYVZ&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprYVZ.0&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=PEprecL&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprecM&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=PEprkj8&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprkj8.0&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=PEprqpv&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /static/favicon.png HTTP/1.0" 304 Not Modified
INFO: 192.168.1.13:0 - "GET /api/v1/auths/admin/config HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/webhook HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/auths/admin/config/ldap/server HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/auths/admin/config/ldap HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /audio/api/v1/config HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /audio/api/v1/voices HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /audio/api/v1/models HTTP/1.0" 200 OK
INFO [open_webui.env] Saving 'AUDIO_TTS_OPENAI_API_BASE_URL' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_OPENAI_API_KEY' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_API_KEY' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_ENGINE' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_MODEL' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_VOICE' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_SPLIT_ON' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_AZURE_SPEECH_REGION' to the database
INFO [open_webui.env] Saving 'AUDIO_TTS_AZURE_SPEECH_OUTPUT_FORMAT' to the database
INFO [open_webui.env] Saving 'AUDIO_STT_OPENAI_API_BASE_URL' to the database
INFO [open_webui.env] Saving 'AUDIO_STT_OPENAI_API_KEY' to the database
INFO [open_webui.env] Saving 'AUDIO_STT_ENGINE' to the database
INFO [open_webui.env] Saving 'AUDIO_STT_MODEL' to the database
INFO [open_webui.env] Saving 'WHISPER_MODEL' to the database
INFO: 192.168.1.13:0 - "POST /audio/api/v1/config/update HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/config HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /static/favicon.png HTTP/1.0" 304 Not Modified
INFO: 192.168.1.13:0 - "GET /api/v1/chats/00381707-0a57-4ce0-b336-16ff7c6557a8 HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/chats/00381707-0a57-4ce0-b336-16ff7c6557a8/tags HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/users/user/settings HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ollama/api/version HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/chats/002cca7f-dfbb-40c2-8238-328e4e02befe HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/chats/002cca7f-dfbb-40c2-8238-328e4e02befe/tags HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /api/v1/users/user/settings HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=PEprqpv.0&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=PEprwwh&sid=ZgnRn8YsAqIms3UoAAAA HTTP/1.0" 200 OK
INFO: 192.168.1.13:0 - "POST /audio/api/v1/speech HTTP/1.0" 200 OK
```
Apologies @tjbck but could you clarify what I did wrong so I can correct it? Or are you saying the TTS model has incorrect formatting?
<!-- gh-comment-id:2538789098 -->
@nphalem commented on GitHub (Dec 12, 2024):
Apologies @tjbck but could you clarify what I did wrong so I can correct it? Or are you saying the TTS model has incorrect formatting?
@nphalem I had the same issue and I dove into the code to figure it out.
tldr; use cmu_us_bdl_arctic-wav-arctic_a0001 as the TTS_MODEL and it should work.
I wrote a little python script that pulls down the embeddings and lists out the files in the dataset. The open-webui backend code basically does a lookup for that file name and uses the index to "unsqueeze". Here is the specific code that should help you understand what is going on.
elifapp.state.config.TTS_ENGINE=="transformers":payload=Nonetry:payload=json.loads(body.decode("utf-8"))exceptExceptionase:log.exception(e)raiseHTTPException(status_code=400,detail="Invalid JSON payload")importtorchimportsoundfileassfload_speech_pipeline()embeddings_dataset=app.state.speech_speaker_embeddings_datasetspeaker_index=6799try:speaker_index=embeddings_dataset["filename"].index(app.state.config.TTS_MODEL)exceptException:passspeaker_embedding=torch.tensor(embeddings_dataset[speaker_index]["xvector"]).unsqueeze(0)speech=app.state.speech_synthesiser(payload["input"],forward_params={"speaker_embeddings":speaker_embedding},)sf.write(file_path,speech["audio"],samplerate=speech["sampling_rate"])withopen(file_body_path,"w")asf:json.dump(json.loads(body.decode("utf-8")),f)returnFileResponse(file_path)```So,ifyouputitsomethingthatisn't found like "bdl" then it defaults to index 6799 which is why you couldn'tgetthevoicetochange.
<!-- gh-comment-id:2557254631 -->
@dmabry commented on GitHub (Dec 20, 2024):
@nphalem I had the same issue and I dove into the code to figure it out.
tldr; use cmu_us_bdl_arctic-wav-arctic_a0001 as the TTS_MODEL and it should work.
I wrote a little python script that pulls down the embeddings and lists out the files in the dataset. The open-webui backend code basically does a lookup for that file name and uses the index to "unsqueeze". Here is the specific code that should help you understand what is going on.
```python
elif app.state.config.TTS_ENGINE == "transformers":
payload = None
try:
payload = json.loads(body.decode("utf-8"))
except Exception as e:
log.exception(e)
raise HTTPException(status_code=400, detail="Invalid JSON payload")
import torch
import soundfile as sf
load_speech_pipeline()
embeddings_dataset = app.state.speech_speaker_embeddings_dataset
speaker_index = 6799
try:
speaker_index = embeddings_dataset["filename"].index(
app.state.config.TTS_MODEL
)
except Exception:
pass
speaker_embedding = torch.tensor(
embeddings_dataset[speaker_index]["xvector"]
).unsqueeze(0)
speech = app.state.speech_synthesiser(
payload["input"],
forward_params={"speaker_embeddings": speaker_embedding},
)
sf.write(file_path, speech["audio"], samplerate=speech["sampling_rate"])
with open(file_body_path, "w") as f:
json.dump(json.loads(body.decode("utf-8")), f)
return FileResponse(file_path)
```
So, if you put it something that isn't found like "bdl" then it defaults to index 6799 which is why you couldn't get the voice to change.
The below code lists all files in the dataset, but you only need to reference the first one for each speaker in order for it to work.
list_speakers.py
#!/usr/bin/env python3fromdatasetsimportload_datasetdeflist_available_speakers():# Load the speaker embeddings datasetembeddings_dataset=load_dataset("Matthijs/cmu-arctic-xvectors",split="validation")# Get a list of all available filenames (speaker model identifiers)speaker_names=embeddings_dataset["filename"]returnspeaker_namesif__name__=="__main__":# Fetch and print all valid speaker namesspeakers=list_available_speakers()print("Available Speaker Names:")fori,speakerinenumerate(speakers):print(f"{i}: {speaker}")
<!-- gh-comment-id:2557263850 -->
@dmabry commented on GitHub (Dec 20, 2024):
And to just put it out there. Here is the list of relevant speakers with their indexes and names:
```
0: cmu_us_awb_arctic-wav-arctic_a0001
1138: cmu_us_bdl_arctic-wav-arctic_a0001
2271: cmu_us_clb_arctic-wav-arctic_a0001
3403: cmu_us_jmk_arctic-wav-arctic_a0001
4535: cmu_us_ksp_arctic-wav-arctic_a0001
5667: cmu_us_rms_arctic-wav-arctic_a0001
6799: cmu_us_slt_arctic-wav-arctic_a0001
```
The below code lists all files in the dataset, but you only need to reference the first one for each speaker in order for it to work.
list_speakers.py
```python
#!/usr/bin/env python3
from datasets import load_dataset
def list_available_speakers():
# Load the speaker embeddings dataset
embeddings_dataset = load_dataset("Matthijs/cmu-arctic-xvectors", split="validation")
# Get a list of all available filenames (speaker model identifiers)
speaker_names = embeddings_dataset["filename"]
return speaker_names
if __name__ == "__main__":
# Fetch and print all valid speaker names
speakers = list_available_speakers()
print("Available Speaker Names:")
for i, speaker in enumerate(speakers):
print(f"{i}: {speaker}")
```
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @nphalem on GitHub (Dec 11, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/7779
Bug Report
Installation Method
Docker compose with searxng
Environment
Open WebUI Version: v0.4.8
Ollama (if applicable): v.0.5.1
Operating System: Arch Linux 6.12.4
Confirmation:
Expected Behavior:
OpenWebUI changes the voice model when changed in settings.
Actual Behavior:
OpenWebUI doesn't change the voice model when changed in settings.
Description
Bug Summary:
When changing the CMU Artic speaker model the voice stays the same.
Reproduction Details
Steps to Reproduce:
slt(US female)bdl(US male)Logs and Screenshots
Docker Container Logs:
@tjbck commented on GitHub (Dec 11, 2024):
Incorrect format.
@nphalem commented on GitHub (Dec 12, 2024):
Apologies @tjbck but could you clarify what I did wrong so I can correct it? Or are you saying the TTS model has incorrect formatting?
@dmabry commented on GitHub (Dec 20, 2024):
@nphalem I had the same issue and I dove into the code to figure it out.
tldr; use cmu_us_bdl_arctic-wav-arctic_a0001 as the TTS_MODEL and it should work.
I wrote a little python script that pulls down the embeddings and lists out the files in the dataset. The open-webui backend code basically does a lookup for that file name and uses the index to "unsqueeze". Here is the specific code that should help you understand what is going on.
@dmabry commented on GitHub (Dec 20, 2024):
And to just put it out there. Here is the list of relevant speakers with their indexes and names:
The below code lists all files in the dataset, but you only need to reference the first one for each speaker in order for it to work.
list_speakers.py