FastAPI uses AnyIO which by default has a limit of 40 threads for the whole application. These threads are used by any non-async or threaded function. This PR adds support for configuring this value via ENV, it also bumps the default value to 64 threads.
This should help the server from getting blocked when handling web-searches or any other non-async calls.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/13031
**Author:** [@gaby](https://github.com/gaby)
**Created:** 4/18/2025
**Status:** ✅ Merged
**Merged:** 4/19/2025
**Merged by:** [@tjbck](https://github.com/tjbck)
**Base:** `dev` ← **Head:** `anyio-workers`
---
### 📝 Commits (4)
- [`19b3d88`](https://github.com/open-webui/open-webui/commit/19b3d88737e49eb24a37f27fde52c6bf4efc9a59) Add support for configuring FastAPI/AnyIO Thread Pool Size
- [`e19554e`](https://github.com/open-webui/open-webui/commit/e19554e9a8c9c2259527e44660eb9826557070be) Simplify import
- [`f77f6cd`](https://github.com/open-webui/open-webui/commit/f77f6cd1e24b3fc1e6ba99c1d16bbdeb330e1cb3) Update imports to match anyio docs
- [`62f87c2`](https://github.com/open-webui/open-webui/commit/62f87c2e38ebd5d1ee785bb10c76aaef6ffd4223) Use default AnyIO pool size
### 📊 Changes
**4 files changed** (+21 additions, -5 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/open_webui/config.py` (+3 -0)
📝 `backend/open_webui/main.py` (+8 -0)
📝 `backend/open_webui/routers/knowledge.py` (+2 -1)
📝 `backend/open_webui/routers/users.py` (+8 -4)
</details>
### 📄 Description
### Description
- FastAPI uses AnyIO which by default has a limit of `40` threads for the whole application. These threads are used by any non-async or threaded function. This PR adds support for configuring this value via `ENV`, ~it also bumps the default value to 64 threads.~
- This should help the server from getting blocked when handling web-searches or any other `non-async` calls.
Implementation Source: https://github.com/Kludex/fastapi-tips#2-be-careful-with-non-async-functions
AnyIO Docs: https://www.starlette.io/threadpool/#concurrency-limitations
Related: https://github.com/open-webui/open-webui/pull/12958
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/13031
Author: @gaby
Created: 4/18/2025
Status: ✅ Merged
Merged: 4/19/2025
Merged by: @tjbck
Base:
dev← Head:anyio-workers📝 Commits (4)
19b3d88Add support for configuring FastAPI/AnyIO Thread Pool Sizee19554eSimplify importf77f6cdUpdate imports to match anyio docs62f87c2Use default AnyIO pool size📊 Changes
4 files changed (+21 additions, -5 deletions)
View changed files
📝
backend/open_webui/config.py(+3 -0)📝
backend/open_webui/main.py(+8 -0)📝
backend/open_webui/routers/knowledge.py(+2 -1)📝
backend/open_webui/routers/users.py(+8 -4)📄 Description
Description
40threads for the whole application. These threads are used by any non-async or threaded function. This PR adds support for configuring this value viaENV,it also bumps the default value to 64 threads.non-asynccalls.Implementation Source: https://github.com/Kludex/fastapi-tips#2-be-careful-with-non-async-functions
AnyIO Docs: https://www.starlette.io/threadpool/#concurrency-limitations
Related: https://github.com/open-webui/open-webui/pull/12958
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.