mirror of
https://github.com/open-webui/open-webui.git
synced 2026-07-17 08:21:12 -05:00
[GH-ISSUE #3343] Start TTS generation as soon as 1 sentence worth of tokens are generated. #132957
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/open-webui#132957
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @chrishart0 on GitHub (Jun 21, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/3343
Is your feature request related to a problem? Please describe.
The new audio generation features are awesome, but audio could generation much closer to real time. As it stands, text does not get sent to the audio generation API until the full response has been received. Preventing anything like a real time chat experience.
Describe the solution you'd like
Since the TTS functionality already sends 1 sentence at a time to get generated, we should send the first sentence out to the audio API as soon as it has streamed in. That way, we can get audio back withing seconds of the message being sent instead of waiting for the whole response to come back.
Describe alternatives you've considered
None
Additional context
Technical Notes
There are two places audio is generated which will need to be touched. The regular chat interface and the call interface. We will need to figure out how to tap into stream and, I suppose use the same sentence chunker which is already written chunking text to send to the TTS API for chunking the stream. I've only glanced at the code but this doesn't sound too hard.