[GH-ISSUE #7509] issue/enh: STT retrofeeds to mic during call #118081

Closed
opened 2026-05-20 19:33:07 -05:00 by GiteaMirror · 16 comments
Owner

Originally created by @Pekkari on GitHub (Dec 1, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/7509

Installation Method

Docker

Environment

  • Open WebUI Version: v0.4.6

  • Ollama (if applicable): v0.4.6

  • Operating System: Fedora 41

  • Browser (if applicable): Firefox 133.0

Confirmation:

  • I have read and followed all the instructions provided in the README.md.
  • I am on the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.
  • I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.

Expected Behavior:

Have a conversation over call feature with the model of your choice using speakers and mic, no headsets

Actual Behavior:

The model receives first sentence from user and keeps looping its own voice until user stops the call

Description

Bug Summary:
STT retrofeeds model voice while on speakers

Reproduction Details

Steps to Reproduce:

  • Open a new conversation with any model
  • Click the call feature
  • Give initial request
  • See how the conversation continues on its own until you stop the call feature

Logs and Screenshots

Screenshots/Screen Recordings (if applicable):
Screenshot_20241201_133001
Screenshot_20241201_133043

Additional info

This is reproducible using openedai-speech configuration also.

Originally created by @Pekkari on GitHub (Dec 1, 2024). Original GitHub issue: https://github.com/open-webui/open-webui/issues/7509 ## Installation Method Docker ## Environment - **Open WebUI Version:** v0.4.6 - **Ollama (if applicable):** v0.4.6 - **Operating System:** Fedora 41 - **Browser (if applicable):** Firefox 133.0 **Confirmation:** - [x] I have read and followed all the instructions provided in the README.md. - [x] I am on the latest version of both Open WebUI and Ollama. - [ ] I have included the browser console logs. - [ ] I have included the Docker container logs. - [x] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below. ## Expected Behavior: Have a conversation over call feature with the model of your choice using speakers and mic, no headsets ## Actual Behavior: The model receives first sentence from user and keeps looping its own voice until user stops the call ## Description **Bug Summary:** STT retrofeeds model voice while on speakers ## Reproduction Details **Steps to Reproduce:** - Open a new conversation with any model - Click the call feature - Give initial request - See how the conversation continues on its own until you stop the call feature ## Logs and Screenshots **Screenshots/Screen Recordings (if applicable):** ![Screenshot_20241201_133001](https://github.com/user-attachments/assets/d1da0d1a-3957-4fec-b9ae-8585e8e98bd9) ![Screenshot_20241201_133043](https://github.com/user-attachments/assets/2547f869-6db9-4e07-838a-3586393d4b84) ## Additional info This is reproducible using openedai-speech configuration also.
GiteaMirror added the help wantedgood first issue labels 2026-05-20 19:33:11 -05:00
Author
Owner

@SDEscobedo commented on GitHub (Dec 2, 2024):

Same problem for me.

  • Open WebUI Version: v0.4.6

  • Using LM Studio as server: 0.2.31

  • Operating System: Docker (Doker desktop in wsl2)

  • Browser (if applicable): Chrome 131.0.6778.86

<!-- gh-comment-id:2511273420 --> @SDEscobedo commented on GitHub (Dec 2, 2024): Same problem for me. - Open WebUI Version: v0.4.6 - Using LM Studio as server: 0.2.31 - Operating System: Docker (Doker desktop in wsl2) - Browser (if applicable): Chrome 131.0.6778.86
Author
Owner

@ghost commented on GitHub (Dec 3, 2024):

the author of this message laughs. This problem has been around for a very long time, and I wrote about it, but was ignored by the developers. https://github.com/open-webui/open-webui/issues/6710#event-15110815656 I had to write a program to bypass this bug so that the microphone would turn off when the sound was going to the speakers. I also wrote about the fact that silence is recorded, for some reason the program stupidly writes files when the microphone is completely turned off. https://github.com/open-webui/open-webui/issues/6953 This leads to unnecessary wear and tear on the hard drive, and whisper hallucinations. I even offered code to remove silence from audio files. But the developers ignored this. It’s strange that no one has written about such obvious bugs before. I don’t understand at all why you can’t set a silence threshold. Until the silence threshold is exceeded, files should not be recorded from the microphone at all. Apparently in 10 years the developers will finally correct such obvious errors. While they are doing some minor interface edits, apparently this is more important for them.

<!-- gh-comment-id:2513689090 --> @ghost commented on GitHub (Dec 3, 2024): the author of this message laughs. This problem has been around for a very long time, and I wrote about it, but was ignored by the developers. https://github.com/open-webui/open-webui/issues/6710#event-15110815656 I had to write a program to bypass this bug so that the microphone would turn off when the sound was going to the speakers. I also wrote about the fact that silence is recorded, for some reason the program stupidly writes files when the microphone is completely turned off. https://github.com/open-webui/open-webui/issues/6953 This leads to unnecessary wear and tear on the hard drive, and whisper hallucinations. I even offered code to remove silence from audio files. But the developers ignored this. It’s strange that no one has written about such obvious bugs before. I don’t understand at all why you can’t set a silence threshold. Until the silence threshold is exceeded, files should not be recorded from the microphone at all. Apparently in 10 years the developers will finally correct such obvious errors. While they are doing some minor interface edits, apparently this is more important for them.
Author
Owner

@SirBadfish commented on GitHub (Dec 20, 2024):

Yeah, I just can't get the Whisper hallucinations to go away no matter what I do. It ruins OWUI entirely for me and makes it unusable for my purposes. "Thank you." gets added to everything I say. Sometimes, "Thank you. Thank you. Thank you." gets added. I thought it might be because I was using OWUI within Pinkokio, so I switched to a Docker install. I thought maybe it was my mic, my device, a bad wire, the Whisper model I was using, etc. Nothing I do gets it to go away. Happens on my Macbook Pro M1 Max 64gb, and on my PC w a 4090 as well. I've spent a good 8 hours troubleshooting this issue at this point, but it very much seems to be the way it's implemented into OWUI and not something on my end.

<!-- gh-comment-id:2556896586 --> @SirBadfish commented on GitHub (Dec 20, 2024): Yeah, I just can't get the Whisper hallucinations to go away no matter what I do. It ruins OWUI entirely for me and makes it unusable for my purposes. "Thank you." gets added to everything I say. Sometimes, "Thank you. Thank you. Thank you." gets added. I thought it might be because I was using OWUI within Pinkokio, so I switched to a Docker install. I thought maybe it was my mic, my device, a bad wire, the Whisper model I was using, etc. Nothing I do gets it to go away. Happens on my Macbook Pro M1 Max 64gb, and on my PC w a 4090 as well. I've spent a good 8 hours troubleshooting this issue at this point, but it very much seems to be the way it's implemented into OWUI and not something on my end.
Author
Owner

@Pekkari commented on GitHub (Dec 20, 2024):

Hummmm... yes, I also tried to work it around by looking for the pipewire echo cancelling module, using the configuration from the arch community, no luck :\

<!-- gh-comment-id:2556917661 --> @Pekkari commented on GitHub (Dec 20, 2024): Hummmm... yes, I also tried to work it around by looking for the pipewire echo cancelling module, using the configuration from the [arch community](https://wiki.archlinux.org/title/PipeWire/Examples), no luck :\
Author
Owner

@Pekkari commented on GitHub (Dec 23, 2024):

I got this nearly working fine by using the echo cancel module of pipewire, by setting the following local file:

$ cat .config/pipewire/pipewire.conf.d/60-echo-cancel.conf 
context.modules = [
  {   name = libpipewire-module-echo-cancel
      args = {
          sink.props = {
              filter.smart = true
              filter.smart.name = echo-cancel-sink
          }
          source.props = {
              filter.smart = true
              filter.smart.name = echo-cancel-source
          }
      }
  }
]
$

Every now and then I see some extra "thank you" that I never said, and some input get totally distorted, but I think this may be because of using the small model of whisper, so I'm going to attempt to use something more powerful.

<!-- gh-comment-id:2559876696 --> @Pekkari commented on GitHub (Dec 23, 2024): I got this nearly working fine by using the echo cancel module of pipewire, by setting the following local file: ``` $ cat .config/pipewire/pipewire.conf.d/60-echo-cancel.conf context.modules = [ { name = libpipewire-module-echo-cancel args = { sink.props = { filter.smart = true filter.smart.name = echo-cancel-sink } source.props = { filter.smart = true filter.smart.name = echo-cancel-source } } } ] $ ``` Every now and then I see some extra "thank you" that I never said, and some input get totally distorted, but I think this may be because of using the small model of whisper, so I'm going to attempt to use something more powerful.
Author
Owner

@SirBadfish commented on GitHub (Dec 24, 2024):

I got this nearly working fine by using the echo cancel module of pipewire, by setting the following local file:

@Pekkari , thank you so much for taking the time to come back here and giving us this fix... Sincerely appreciated!

<!-- gh-comment-id:2561084502 --> @SirBadfish commented on GitHub (Dec 24, 2024): > I got this nearly working fine by using the echo cancel module of pipewire, by setting the following local file: @Pekkari , thank you so much for taking the time to come back here and giving us this fix... Sincerely appreciated!
Author
Owner

@cryptokeenz commented on GitHub (Dec 26, 2024):

Hi, I also encountered the issue of the llm model's voice mixing with the speaker's voice. I noticed that when the llm model is thinking, my speech cannot interrupt its thought process. Maybe this problem can be solved by simply blocking voice input while it’s thinking. Hope the developers can see this :D. Additionally, like the user who provided feedback above, whether I use large-v3 or medium, there is automatic input of “Thank you.” This is quite weird.
image

<!-- gh-comment-id:2563002300 --> @cryptokeenz commented on GitHub (Dec 26, 2024): Hi, I also encountered the issue of the llm model's voice mixing with the speaker's voice. I noticed that when the llm model is thinking, my speech cannot interrupt its thought process. Maybe this problem can be solved by simply blocking voice input while it’s thinking. Hope the developers can see this :D. Additionally, like the user who provided feedback above, whether I use large-v3 or medium, there is automatic input of “Thank you.” This is quite weird. <img width="1194" alt="image" src="https://github.com/user-attachments/assets/848d64c3-7511-4d76-9f4a-965aa2121452" />
Author
Owner

@Pekkari commented on GitHub (Dec 27, 2024):

Hi, I also encountered the issue of the llm model's voice mixing with the speaker's voice. I noticed that when the llm model is thinking, my speech cannot interrupt its thought process. Maybe this problem can be solved by simply blocking voice input while it’s thinking.

I wouldn't recommend to do that, indeed, I would recommend the opposite, try to avoid losing voice input while thinking, since it is about time that open source LLMs that do full duplex communication turn up around. Some of them are already available, yet it requires a bit of work to integrate them in ollama + open-webui, so I yet didn't get the time to try them myself, but it is something I have on my schedules.

<!-- gh-comment-id:2563551297 --> @Pekkari commented on GitHub (Dec 27, 2024): > Hi, I also encountered the issue of the llm model's voice mixing with the speaker's voice. I noticed that when the llm model is thinking, my speech cannot interrupt its thought process. Maybe this problem can be solved by simply blocking voice input while it’s thinking. I wouldn't recommend to do that, indeed, I would recommend the opposite, try to avoid losing voice input while thinking, since it is about time that open source LLMs that do full duplex communication turn up around. Some of them are already available, yet it requires a bit of work to integrate them in ollama + open-webui, so I yet didn't get the time to try them myself, but it is something I have on my schedules.
Author
Owner

@nengoxx commented on GitHub (Dec 30, 2024):

With the option to interrupt in call turned off, I just think the mic should be muted while thinking and playing the TTS. Because if you talk with that option off, the LLM would keep responding, and since whisper is faster than the TTS, the TTS only ends up responding to the last message if you talk too fast.

<!-- gh-comment-id:2565482126 --> @nengoxx commented on GitHub (Dec 30, 2024): With the option to interrupt in call turned off, I just think the mic should be muted while thinking and playing the TTS. Because if you talk with that option off, the LLM would keep responding, and since whisper is faster than the TTS, the TTS only ends up responding to the last message if you talk too fast.
Author
Owner

@tjbck commented on GitHub (Jan 3, 2025):

@AliveDedSec

Let me start by clarifying something that seems to be misunderstood: There are no "developers" being careless or ignoring you—there’s just me. A single person maintaining this entire project, in my free time, on top of paying bills and managing the life I have outside of this. I’ve been asking for help for months, countless times, but nobody has stepped up to contribute in any meaningful way, and realistically, I don’t have the time to personally address every single issue on my own.

You claim your feedback was ignored, but let's address that. First, you are not following the provided issue formats, which are there for a reason—to save time and help me prioritize. When I spend extra time trying to make sense of incomplete or unstructured reports, that time could have been spent on fixing other urgent, properly reported bugs that affect everyone. So no, it's not that your feedback was ignored—it's that you've made it more difficult for me to address it.

If the issues you’re pointing out truly matter to you, I'd ask you to go beyond just pointing fingers. I've invited contributions repeatedly. Feel free to submit a pull request with a proper implementation and extensive testing—I’d be happy to review it. But until then, I can only prioritize so much while juggling everything myself.

Lastly, your tone here does not help. Calling out “developers” and implying negligence when you know full well this is a one-person, unpaid endeavor is frustrating and honestly demotivating. I'm doing my best to keep this project alive for free. If that’s not something you're willing to support, either through following formats, providing useful contributions, or adjusting your attitude, then I’m not sure what else I can do to meet your expectations.

That said, I do appreciate passionate users who want to see the project improve. I just ask for basic respect for my time and effort in return. The door is always open for constructive contributions. You have the ability to help make this project better, but until then, please understand that I’m only one person, and my time is finite.

Thank you for understanding.

<!-- gh-comment-id:2568565631 --> @tjbck commented on GitHub (Jan 3, 2025): @AliveDedSec Let me start by clarifying something that seems to be misunderstood: There are no "developers" being careless or ignoring you—there’s just me. A single person maintaining this entire project, in my free time, on top of paying bills and managing the life I have outside of this. I’ve been asking for help for months, countless times, but nobody has stepped up to contribute in any meaningful way, and realistically, I don’t have the time to personally address every single issue on my own. You claim your feedback was ignored, but let's address that. First, you are *not* following the provided issue formats, which are there for a reason—to save time and help me prioritize. When I spend extra time trying to make sense of incomplete or unstructured reports, that time could have been spent on fixing other urgent, properly reported bugs that affect everyone. So no, it's not that your feedback was ignored—it's that you've made it more difficult for me to address it. If the issues you’re pointing out truly matter to you, I'd ask you to go beyond just pointing fingers. I've invited contributions repeatedly. Feel free to submit a pull request with a proper implementation and extensive testing—I’d be happy to review it. But until then, I can only prioritize so much while juggling everything myself. Lastly, your tone here does not help. Calling out “developers” and implying negligence when you know full well this is a one-person, unpaid endeavor is frustrating and honestly demotivating. I'm doing my best to keep this project alive for *free*. If that’s not something you're willing to support, either through following formats, providing useful contributions, or adjusting your attitude, then I’m not sure what else I can do to meet your expectations. That said, I *do* appreciate passionate users who want to see the project improve. I just ask for basic respect for my time and effort in return. The door is always open for constructive contributions. You have the ability to help make this project better, but until then, please understand that I’m only one person, and my time is finite. Thank you for understanding.
Author
Owner

@tjbck commented on GitHub (Jan 3, 2025):

To everyone else reading this thread, I want to take a moment to address you directly.

Please understand that things take time, especially when a project is maintained by just one person. I want to emphasize that I will address issues eventually, but I ask for your patience and understanding while I do so. Your patience and support truly mean a lot.

With that being said, I’ve recently made strides towards improving the experience. I’ve modified the frontend code to incorporate audio echo cancellation from the Media Capture and Streams API. This has the potential to help mitigate some of the problems mentioned and make things a bit smoother moving forward.

Your feedback helps make this project better, but remember, this is all a process. Thank you to those who understand and continue to support the project thoughtfully.

<!-- gh-comment-id:2568573219 --> @tjbck commented on GitHub (Jan 3, 2025): To everyone else reading this thread, I want to take a moment to address you directly. Please understand that things take time, especially when a project is maintained by just one person. I want to emphasize that I *will* address issues eventually, but I ask for your patience and understanding while I do so. Your patience and support truly mean a lot. With that being said, I’ve recently made strides towards improving the experience. I’ve modified the frontend code to incorporate audio echo cancellation from the Media Capture and Streams API. This has the potential to help mitigate some of the problems mentioned and make things a bit smoother moving forward. Your feedback helps make this project better, but remember, this is all a process. Thank you to those who understand and continue to support the project thoughtfully.
Author
Owner

@evolu8 commented on GitHub (Jan 3, 2025):

Hi Timothy. Firstly - thank you! You have done a stunning job of making this an incredibly powerful and useful tool.

Secondly, please don't be demoralized! Sadly it is not always clear what a project is 'under the water'. And now that it is clear it is your legs paddling like crazy, I'm in awe. But I'm also concerned for your risk of extreme burnout.

I humbly suggest a new year resolution for you. To focus on delegating. I hear you when you say you've tried and its not worked, but it would be wonderful if we can fix that. If you are open to discussing, I'd be happy to at least research around how to build an effective community to help you, not just throw demands at you.

A HUGE well done from me, and I'm sure many many others. THANK YOU. :)

<!-- gh-comment-id:2569011431 --> @evolu8 commented on GitHub (Jan 3, 2025): Hi Timothy. Firstly - thank you! You have done a stunning job of making this an incredibly powerful and useful tool. Secondly, please don't be demoralized! Sadly it is not always clear what a project is 'under the water'. And now that it is clear it is your legs paddling like crazy, I'm in awe. But I'm also concerned for your risk of extreme burnout. I humbly suggest a new year resolution for you. To focus on delegating. I hear you when you say you've tried and its not worked, but it would be wonderful if we can fix that. If you are open to discussing, I'd be happy to at least research around how to build an effective community to help you, not just throw demands at you. A HUGE well done from me, and I'm sure many many others. THANK YOU. :)
Author
Owner

@Pekkari commented on GitHub (Jan 3, 2025):

Impressive, I learnt of this tool a few months ago and I loved to start using it regularly, nowadays I'm also grooming to use it as part of the code base of a downstream product that will be, as required legally, open source, so you may expect contributions coming from my end in the future. Thanks for your commitment Timothy!

<!-- gh-comment-id:2569032450 --> @Pekkari commented on GitHub (Jan 3, 2025): Impressive, I learnt of this tool a few months ago and I loved to start using it regularly, nowadays I'm also grooming to use it as part of the code base of a downstream product that will be, as required legally, open source, so you may expect contributions coming from my end in the future. Thanks for your commitment Timothy!
Author
Owner

@ghost commented on GitHub (Jan 16, 2025):

@tjbck
Dear tjbck, respected developer! I apologize to you. I ask for your forgiveness. I certainly did not want to offend you or hurt you in any way. What you are doing is wonderful. The project is just amazing. I didn't know that you were working on the project alone. As far as I understood from the description of your project on GitHub, you have a whole team. It seems that I was mistaken. I just sincerely wanted to help you make the project better. It seemed to me that you were ignoring me. Thank you very much for your work, your contribution is invaluable. If you saw, for example, I tried to improve the translation into Russian. And you accepted this contribution, thank you. It's just that I'm not a programmer. I'm just an enthusiast. And all the code I write, I write with the help of artificial intelligence, including ChatGPT. And I also solve all the problems with the help of artificial intelligence, problems related to your project. But I understand very little about programming. I don't understand the complete structure of your project at all. For me, it's something huge and incomprehensible. I'm just trying to make some minor fixes to specific problems. Apparently, I was very disrespectful to you. I apologize once again. If I could write the perfect code for your project, I would. But unfortunately, I don't have enough knowledge. I just thought that you would take my code or problem description as an example for an independent solution. Because you are a professional and a specialist. Continue to develop and improve your project, you are very necessary for all of us. With best wishes for you and kindness. And with great gratitude. Thank you for everything you do for us. This is one of the best projects I know at the moment.

<!-- gh-comment-id:2594456655 --> @ghost commented on GitHub (Jan 16, 2025): @tjbck Dear [tjbck](https://github.com/tjbck), respected developer! I apologize to you. I ask for your forgiveness. I certainly did not want to offend you or hurt you in any way. What you are doing is wonderful. The project is just amazing. I didn't know that you were working on the project alone. As far as I understood from the description of your project on GitHub, you have a whole team. It seems that I was mistaken. I just sincerely wanted to help you make the project better. It seemed to me that you were ignoring me. Thank you very much for your work, your contribution is invaluable. If you saw, for example, I tried to improve the translation into Russian. And you accepted this contribution, thank you. It's just that I'm not a programmer. I'm just an enthusiast. And all the code I write, I write with the help of artificial intelligence, including ChatGPT. And I also solve all the problems with the help of artificial intelligence, problems related to your project. But I understand very little about programming. I don't understand the complete structure of your project at all. For me, it's something huge and incomprehensible. I'm just trying to make some minor fixes to specific problems. Apparently, I was very disrespectful to you. I apologize once again. If I could write the perfect code for your project, I would. But unfortunately, I don't have enough knowledge. I just thought that you would take my code or problem description as an example for an independent solution. Because you are a professional and a specialist. Continue to develop and improve your project, you are very necessary for all of us. With best wishes for you and kindness. And with great gratitude. Thank you for everything you do for us. This is one of the best projects I know at the moment.
Author
Owner

@ghost commented on GitHub (Jan 16, 2025):

I also wanted to mention that I noticed in your latest update, you've significantly improved the noise reduction and audio processing in the speech recognition models. This is an incredible advancement and a testament to your dedication and skill. Your work is truly making a difference, and I am in awe of your expertise. Keep up the fantastic work!

<!-- gh-comment-id:2594458867 --> @ghost commented on GitHub (Jan 16, 2025): I also wanted to mention that I noticed in your latest update, you've significantly improved the noise reduction and audio processing in the speech recognition models. This is an incredible advancement and a testament to your dedication and skill. Your work is truly making a difference, and I am in awe of your expertise. Keep up the fantastic work!
Author
Owner

@michaelmarziani commented on GitHub (Jan 29, 2025):

It's encouraging to me to see people expressing themselves, communicating, and clarifying the situation with kindness and compassion. Well done, and I wish the best for this project.

<!-- gh-comment-id:2621959859 --> @michaelmarziani commented on GitHub (Jan 29, 2025): It's encouraging to me to see people expressing themselves, communicating, and clarifying the situation with kindness and compassion. Well done, and I wish the best for this project.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#118081