github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-07-11 10:34:13 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[GH-ISSUE #20002] issue: Severe TTFT increase when accessing vLLM through Open WebUI #19059

New Issue
Closed
opened 2026-04-20 01:23:03 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-04-20 01:23:03 -05:00
Owner
Copy Link

Originally created by @mickeytheseal on GitHub (Dec 17, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/20002

Check Existing Issues

  • I have searched for any existing and/or related issues.
  • I have searched for any existing and/or related discussions.
  • I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!).
  • I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

0.6.41

Ollama Version (if applicable)

No response

Operating System

k8s

Browser (if applicable)

No response

Confirmation

  • I have read and followed all instructions in README.md.
  • I am using the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.
  • I have provided every relevant configuration, setting, and environment variable used in my setup.
  • I have clearly listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc).
  • I have documented step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation. My steps:
  • Start with the initial platform/version/OS and dependencies used,
  • Specify exact install/launch/configure commands,
  • List URLs visited, user input (incl. example values/emails/passwords if needed),
  • Describe all options and toggles enabled or changed,
  • Include any files or environmental changes,
  • Identify the expected and actual result at each stage,
  • Ensure any reasonably skilled user can follow and hit the same issue.

Expected Behavior

When sending requests to vLLM through OpenWebUI API TTFT should be reasonably close to TTFT via direct vLLM requests with only minor overhead produced by Open WebUI.

Actual Behavior

When sending same requests through Open WebUI API TTFT increases to ~5–6 seconds, even with the same model, prompts, hw and inference backend. This also increases overall E2E latency.
Direct vLLM requests in my case show TTFT of ~600 ms.

Steps to Reproduce

  1. Use an Open WebUI instance with default config
  2. Configure vLLM as the inference backend
  3. Use the same model, prompts, and profile for all tests
  4. Send requests directly to vLLM API and measure TTFT and E2E.
  5. Send the same requests through OpenWebUI API and measure TTFT and E2E

Logs & Screenshots

No logs at the moment. TTFT measurements were collected using benchmarking tools like locust and aiperf.

Additional Information

  • The issue reproduces even at low concurrency
  • Network latency between OpenWebUI and vLLM is ok
Originally created by @mickeytheseal on GitHub (Dec 17, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/20002 ### Check Existing Issues - [x] I have searched for any existing and/or related issues. - [x] I have searched for any existing and/or related discussions. - [x] I have also searched in the CLOSED issues AND CLOSED discussions and found no related items (your issue might already be addressed on the development branch!). - [x] I am using the latest version of Open WebUI. ### Installation Method Docker ### Open WebUI Version 0.6.41 ### Ollama Version (if applicable) _No response_ ### Operating System k8s ### Browser (if applicable) _No response_ ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have **provided every relevant configuration, setting, and environment variable used in my setup.** - [x] I have clearly **listed every relevant configuration, custom setting, environment variable, and command-line option that influences my setup** (such as Docker Compose overrides, .env values, browser settings, authentication configurations, etc). - [x] I have documented **step-by-step reproduction instructions that are precise, sequential, and leave nothing to interpretation**. My steps: - Start with the initial platform/version/OS and dependencies used, - Specify exact install/launch/configure commands, - List URLs visited, user input (incl. example values/emails/passwords if needed), - Describe all options and toggles enabled or changed, - Include any files or environmental changes, - Identify the expected and actual result at each stage, - Ensure any reasonably skilled user can follow and hit the same issue. ### Expected Behavior When sending requests to vLLM through OpenWebUI API TTFT should be reasonably close to TTFT via direct vLLM requests with only minor overhead produced by Open WebUI. ### Actual Behavior When sending same requests through Open WebUI API TTFT increases to ~5–6 **seconds**, even with the same model, prompts, hw and inference backend. This also increases overall E2E latency. Direct vLLM requests in my case show TTFT of ~600 ms. ### Steps to Reproduce 1. Use an Open WebUI instance with default config 2. Configure vLLM as the inference backend 3. Use the same model, prompts, and profile for all tests 4. Send requests directly to vLLM API and measure TTFT and E2E. 5. Send the same requests through OpenWebUI API and measure TTFT and E2E ### Logs & Screenshots No logs at the moment. TTFT measurements were collected using benchmarking tools like locust and aiperf. ### Additional Information - The issue reproduces even at low concurrency - Network latency between OpenWebUI and vLLM is ok
GiteaMirror added the bug label 2026-04-20 01:23:03 -05:00
GiteaMirror commented 2026-04-20 01:23:04 -05:00
Author
Owner
Copy Link

@owui-terminator[bot] commented on GitHub (Dec 17, 2025):

🔍 Similar Issues Found

I found some existing issues that might be related to this one. Please check if any of these are duplicates or contain helpful solutions:

  1. #19777 issue:
    by Yaute7 • Dec 05, 2025 • bug

  2. #19861 issue:
    by QuitHub • Dec 10, 2025 • bug

  3. #19877 issue:
    by dotmobo • Dec 11, 2025 • bug

  4. #19864 issue:
    by Haervwe • Dec 10, 2025 • bug

  5. #19063 TTS not working in Open-WebUi
    by iSamPrime • Nov 09, 2025 • bug

Show 5 more related issues

  1. #11999 issue: Open WebUI not giving all tokens provided by VLLM
    by taltoris • Mar 24, 2025 • bug

  2. #19211 issue:
    by Byrnes9 • Nov 16, 2025 • bug

  3. #19563 issue:
    by naruto7g • Nov 28, 2025 • bug

  4. #17194 issue: The LLM fails to recognize large documents correctly on the first attempt.
    by Cyp9715 • Sep 04, 2025 • bug

  5. #19496 issue: 500 internal server error appears in v0.6.40
    by cloudtuotuo • Nov 26, 2025 • bug

💡 Tips:

  • If this is a duplicate, please consider closing this issue and adding any additional details to the existing one
  • If you found a solution in any of these issues, please share it here to help others

This comment was generated automatically by a bot. Please react with a 👍 if this comment was helpful, or a 👎 if it was not.

<!-- gh-comment-id:3664486975 --> @owui-terminator[bot] commented on GitHub (Dec 17, 2025): 🔍 **Similar Issues Found** I found some existing issues that might be related to this one. Please check if any of these are duplicates or contain helpful solutions: 1. [#19777](https://github.com/open-webui/open-webui/issues/19777) **issue:** *by Yaute7 • Dec 05, 2025 • `bug`* 2. [#19861](https://github.com/open-webui/open-webui/issues/19861) **issue:** *by QuitHub • Dec 10, 2025 • `bug`* 3. [#19877](https://github.com/open-webui/open-webui/issues/19877) **issue:** *by dotmobo • Dec 11, 2025 • `bug`* 4. [#19864](https://github.com/open-webui/open-webui/issues/19864) **issue:** *by Haervwe • Dec 10, 2025 • `bug`* 5. [#19063](https://github.com/open-webui/open-webui/issues/19063) **TTS not working in Open-WebUi** *by iSamPrime • Nov 09, 2025 • `bug`* <details> <summary>Show 5 more related issues</summary> 6. [#11999](https://github.com/open-webui/open-webui/issues/11999) **issue: Open WebUI not giving all tokens provided by VLLM** *by taltoris • Mar 24, 2025 • `bug`* 7. [#19211](https://github.com/open-webui/open-webui/issues/19211) **issue:** *by Byrnes9 • Nov 16, 2025 • `bug`* 8. [#19563](https://github.com/open-webui/open-webui/issues/19563) **issue:** *by naruto7g • Nov 28, 2025 • `bug`* 9. [#17194](https://github.com/open-webui/open-webui/issues/17194) **issue: The LLM fails to recognize large documents correctly on the first attempt.** *by Cyp9715 • Sep 04, 2025 • `bug`* 10. [#19496](https://github.com/open-webui/open-webui/issues/19496) **issue: 500 internal server error appears in v0.6.40** *by cloudtuotuo • Nov 26, 2025 • `bug`* </details> --- 💡 **Tips:** - If this is a duplicate, please consider closing this issue and adding any additional details to the existing one - If you found a solution in any of these issues, please share it here to help others *This comment was generated automatically by a bot.* Please react with a 👍 if this comment was helpful, or a 👎 if it was not.
GiteaMirror commented 2026-04-20 01:23:05 -05:00
Author
Owner
Copy Link

@gaby commented on GitHub (Dec 19, 2025):

👍 I've noticed this too. This is expected since Open-WebUI is checking your token and also checking if you are allowed to use the model before sending the request to the backend. The backend is then also checking if the backend API Key matches and if the model matches the one you requested. That's the latency you are seeing.

You could try enabling the caching of models list, to see if that improves your throughput. (Admin -> Connections -> Enable the cache models list toggle)

<!-- gh-comment-id:3674878847 --> @gaby commented on GitHub (Dec 19, 2025): 👍 I've noticed this too. This is expected since Open-WebUI is checking your token and also checking if you are allowed to use the model before sending the request to the backend. The backend is then also checking if the backend API Key matches and if the model matches the one you requested. That's the latency you are seeing. You could try enabling the caching of models list, to see if that improves your throughput. (Admin -> Connections -> Enable the cache models list toggle)
GiteaMirror commented 2026-04-20 01:23:05 -05:00
Author
Owner
Copy Link

@mickeytheseal commented on GitHub (Dec 19, 2025):

You could try enabling the caching of models list, to see if that improves your throughput. (Admin -> Connections -> Enable the cache models list toggle)

Thanks for your advice! Ran some tests with caching enabled and sadly there was not any visible effect. I will be glad to run more tests on my installation if there is any other workaround available.

<!-- gh-comment-id:3675575113 --> @mickeytheseal commented on GitHub (Dec 19, 2025): > You could try enabling the caching of models list, to see if that improves your throughput. (Admin -> Connections -> Enable the cache models list toggle) Thanks for your advice! Ran some tests with caching enabled and sadly there was not any visible effect. I will be glad to run more tests on my installation if there is any other workaround available.
GiteaMirror commented 2026-04-20 01:23:06 -05:00
Author
Owner
Copy Link

@scottybo commented on GitHub (Dec 20, 2025):

Just a note to say I'm having the same issue - makes the whole experience feel very slow

<!-- gh-comment-id:3677732082 --> @scottybo commented on GitHub (Dec 20, 2025): Just a note to say I'm having the same issue - makes the whole experience feel very slow
GiteaMirror commented 2026-04-20 01:23:06 -05:00
Author
Owner
Copy Link

@druellan commented on GitHub (Dec 20, 2025):

@mickeytheseal if you didn't, perhaps you can check the browser network activity, just to confirm if the latency happens in a particular call.
Also, another test you can try is to disable everything in the "interface" settings, to minimize how many calls OWUI do to the models, and check if the issue is related to usage.

<!-- gh-comment-id:3677756795 --> @druellan commented on GitHub (Dec 20, 2025): @mickeytheseal if you didn't, perhaps you can check the browser network activity, just to confirm if the latency happens in a particular call. Also, another test you can try is to disable everything in the "interface" settings, to minimize how many calls OWUI do to the models, and check if the issue is related to usage.
GiteaMirror commented 2026-04-20 01:23:07 -05:00
Author
Owner
Copy Link

@tjbck commented on GitHub (Dec 21, 2025):

@mickeytheseal @gaby how many models do you guys have available here? Performance hit from the ACL logic should be negligible and we are not able to reproduce this issue.

<!-- gh-comment-id:3678606227 --> @tjbck commented on GitHub (Dec 21, 2025): @mickeytheseal @gaby how many models do you guys have available here? Performance hit from the ACL logic should be negligible and we are not able to reproduce this issue.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#19059
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?