I have searched the existing issues and discussions.
Problem Description
In the current start.sh script uvicorn is started without any flags related to workers, this makes uvicorn default to using just 1 worker. In systems with high-number of cores, they won't get used by Open-WebUI.
Add support for specifying how many workers should uvicorn use via ENV.
Alternatives Considered
N/a
Additional Context
N/a
Originally created by @gaby on GitHub (Apr 1, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/12286
### Check Existing Issues
- [x] I have searched the existing issues and discussions.
### Problem Description
In the current start.sh script uvicorn is started without any flags related to workers, this makes uvicorn default to using just 1 worker. In systems with high-number of cores, they won't get used by `Open-WebUI`.
More information here: https://fastapi.tiangolo.com/deployment/server-workers/
Affecting line: https://github.com/open-webui/open-webui/blob/v0.6.0/backend/start.sh#L68
### Desired Solution you'd like
Add support for specifying how many workers should `uvicorn` use via `ENV`.
### Alternatives Considered
N/a
### Additional Context
N/a
hmm interesting. I didn't think of this but could multiple uvicorn workers lead to higher performance/scalability of the application?
<!-- gh-comment-id:2769314267 -->
@Classic298 commented on GitHub (Apr 1, 2025):
hmm interesting. I didn't think of this but could multiple uvicorn workers lead to higher performance/scalability of the application?
@Classic298 Yes, it basically runs multiple instances of the FastAPI server which is what serves the API.
When doing benchmarks I notice mine was only using 1 core, and I believe its related to this
<!-- gh-comment-id:2769319743 -->
@gaby commented on GitHub (Apr 1, 2025):
@Classic298 Yes, it basically runs multiple instances of the FastAPI server which is what serves the API.
When doing benchmarks I notice mine was only using 1 core, and I believe its related to this
interesting. If you set it to use multiple workers and run another benchmark is there a noticable difference?
<!-- gh-comment-id:2769408271 -->
@Classic298 commented on GitHub (Apr 1, 2025):
interesting. If you set it to use multiple workers and run another benchmark is there a noticable difference?
@Classic298 I run it in Docker so I can't do other benchmarks unless this option is added.
<!-- gh-comment-id:2769435063 -->
@gaby commented on GitHub (Apr 1, 2025):
@Classic298 I run it in Docker so I can't do other benchmarks unless this option is added.
@gaby you could edit the start.sh to manually start multiple service workers, but okay yeah fair point.
Anyways, if it works as we think/hope, it could be a real performance and scalability benefit without having to use kubernetes (which can be a pain in the butt).
And yes, if it does work as we think it does, then i'd absolutely love to see a configurable environment variable for setting a number of service workers for multiple web instances :). If you have a more powerful machine you can set more, default value can remain "1" and so forth 😄
<!-- gh-comment-id:2769484176 -->
@Classic298 commented on GitHub (Apr 1, 2025):
@gaby you could edit the start.sh to manually start multiple service workers, but okay yeah fair point.
Anyways, if it works as we think/hope, it could be a real performance and scalability benefit without having to use kubernetes (which can be a pain in the butt).
And yes, if it does work as we think it does, then i'd absolutely love to see a configurable environment variable for setting a number of service workers for multiple web instances :). If you have a more powerful machine you can set more, default value can remain "1" and so forth 😄
You can use the uvicorn env variable "WEB_CONCURRENCY" to set the number of workers in uvicorn. However, you must use Redis for websocket support, otherwise message streaming is not working any more in the frontend with workers > 1.
<!-- gh-comment-id:2771793617 -->
@almajo commented on GitHub (Apr 2, 2025):
You can use the uvicorn env variable "WEB_CONCURRENCY" to set the number of workers in uvicorn. However, you must use Redis for websocket support, otherwise message streaming is not working any more in the frontend with workers > 1.
See https://github.com/open-webui/open-webui/discussions/9032
@almajo Great find! Issues being moved to discussions by maintainers make finding stuff like this hard.
I wonder if It would work with DragonflyDB which auto scales unlike Redis. Redis is only able to use 1 CPU
<!-- gh-comment-id:2772217937 -->
@gaby commented on GitHub (Apr 2, 2025):
@almajo Great find! Issues being moved to `discussions` by maintainers make finding stuff like this hard.
I wonder if It would work with `DragonflyDB` which auto scales unlike Redis. Redis is only able to use 1 CPU
Please check for existing issues and discussions. It's the bare minimum we ask.
<!-- gh-comment-id:2773346570 -->
@tjbck commented on GitHub (Apr 2, 2025):
Please check for existing issues and discussions. It's the bare minimum we ask.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @gaby on GitHub (Apr 1, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/12286
Check Existing Issues
Problem Description
In the current start.sh script uvicorn is started without any flags related to workers, this makes uvicorn default to using just 1 worker. In systems with high-number of cores, they won't get used by
Open-WebUI.More information here: https://fastapi.tiangolo.com/deployment/server-workers/
Affecting line: https://github.com/open-webui/open-webui/blob/v0.6.0/backend/start.sh#L68
Desired Solution you'd like
Add support for specifying how many workers should
uvicornuse viaENV.Alternatives Considered
N/a
Additional Context
N/a
@Classic298 commented on GitHub (Apr 1, 2025):
hmm interesting. I didn't think of this but could multiple uvicorn workers lead to higher performance/scalability of the application?
@gaby commented on GitHub (Apr 1, 2025):
@Classic298 Yes, it basically runs multiple instances of the FastAPI server which is what serves the API.
When doing benchmarks I notice mine was only using 1 core, and I believe its related to this
@Classic298 commented on GitHub (Apr 1, 2025):
interesting. If you set it to use multiple workers and run another benchmark is there a noticable difference?
@gaby commented on GitHub (Apr 1, 2025):
@Classic298 I run it in Docker so I can't do other benchmarks unless this option is added.
@Classic298 commented on GitHub (Apr 1, 2025):
@gaby you could edit the start.sh to manually start multiple service workers, but okay yeah fair point.
Anyways, if it works as we think/hope, it could be a real performance and scalability benefit without having to use kubernetes (which can be a pain in the butt).
And yes, if it does work as we think it does, then i'd absolutely love to see a configurable environment variable for setting a number of service workers for multiple web instances :). If you have a more powerful machine you can set more, default value can remain "1" and so forth 😄
@almajo commented on GitHub (Apr 2, 2025):
You can use the uvicorn env variable "WEB_CONCURRENCY" to set the number of workers in uvicorn. However, you must use Redis for websocket support, otherwise message streaming is not working any more in the frontend with workers > 1.
See https://github.com/open-webui/open-webui/discussions/9032
@gaby commented on GitHub (Apr 2, 2025):
@almajo Great find! Issues being moved to
discussionsby maintainers make finding stuff like this hard.I wonder if It would work with
DragonflyDBwhich auto scales unlike Redis. Redis is only able to use 1 CPU@tjbck commented on GitHub (Apr 2, 2025):
Please check for existing issues and discussions. It's the bare minimum we ask.