github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-07-16 06:03:26 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[GH-ISSUE #24631] RFC: Backend-only, config-driven, per-user endpoint rate limiting middleware #91104

New Issue
Closed
opened 2026-05-15 16:22:40 -05:00 by GiteaMirror · 2 comments
GiteaMirror commented 2026-05-15 16:22:40 -05:00
Owner
Copy Link

Originally created by @Srujan4812 on GitHub (May 12, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/24631

RFC: Backend-only, config-driven, per-user endpoint rate limiting middleware

Summary

Add a pure-ASGI middleware that applies per-user, per-endpoint-class rate limits across the API surface (chat, embeddings, uploads, admin), using the existing Redis connection and AppConfig plumbing. Backend-only, off by default, zero UI and zero telemetry changes in this RFC. Admin-UI bindings, observability, and per-user overrides are intentionally deferred to follow-ups once this lands.

Today the project only rate-limits signin (RateLimiter at 15 req / 180 s in routers/auths.py). Every other endpoint — including the ones that directly drive upstream LLM cost — is unthrottled.

Problem

A legitimately-issued or compromised JWT / API key can:

  1. Exhaust upstream LLM budgets via /api/chat/completions (Ollama / OpenAI / Anthropic / Gemini) with no per-user ceiling.
  2. Saturate the embedding path (/api/embeddings, /api/v1/retrieval/*) — SentenceTransformer calls use the shared anyio thread pool, so a burst starves every other handler in the worker.
  3. Fill disk or S3 via uploads (/api/v1/files, /api/v1/retrieval/process/*, /api/v1/knowledge/*) — only a total-size cap exists, no per-user rate.
  4. Mutate admin config at machine speed (/api/v1/configs/*, /api/v1/models/*, /api/v1/groups/*) if an admin token is misused or a SCIM bulk sync is misconfigured.

The algorithmic primitive (utils/rate_limit.RateLimiter, Redis rolling window with in-memory fallback) already exists. It just isn't wired into anything beyond signin.

Scope (intentionally small)

In scope for this RFC / first PR:

  • One new file: backend/open_webui/utils/rate_limit_middleware.py.
  • Middleware wiring in backend/open_webui/main.py.
  • New AppConfig fields in backend/open_webui/config.py and an env bootstrap flag in backend/open_webui/env.py.
  • One error message in backend/open_webui/constants.py.
  • Unit tests in backend/open_webui/test/util/test_rate_limit_middleware.py using fakeredis.aioredis.
  • Feature ships disabled by default. Enabling it is additive; no response semantics change for requests under the limit.

Explicitly out of scope for this RFC (deferred):

  • Admin UI panel (no src/ changes).
  • OTel metrics / tracing additions.
  • A /stats endpoint or runtime introspection API.
  • Per-user or per-role overrides.
  • Runtime-configurable route-to-class mapping (the mapping ships hard-coded; overrides come later).
  • Migrating the existing signin_rate_limiter to the new primitive.

This keeps the first PR narrow enough for a single reviewer pass while establishing the primitive that the deferred work can plug into.

Design

Identity resolution (no DB hits)

The middleware runs immediately after AuthTokenMiddleware, so request.state.token is already parsed. Priority:

  1. JWT → decode with utils/auth.decode_token, take the id claim. No DB lookup; if is_valid_token revocation matters, the route handler's get_current_user will reject shortly after.
  2. API key (sk-…) → SHA-256 first 12 chars as identity (never the raw key in a Redis key).
  3. Anonymous → request.client.host (respecting forwarded_allow_ips='*' already configured in uvicorn.run).

Endpoint classification

A compiled regex table inside the middleware module — ships hard-coded for this PR:

Class Path prefixes
chat /api/chat/completions, /api/v1/chat/completions, /ollama/api/chat, /openai/chat/completions
embeddings /api/embeddings, /api/v1/embeddings
uploads /api/v1/files, /api/v1/retrieval/process, /api/v1/knowledge/*/file/add
admin /api/v1/configs, /api/v1/models, /api/v1/groups, /api/v1/users (write methods only)

Unmatched paths are not rate-limited. Skip list (always bypass): /health, /ready, /health/db, /ws/*, SPA static assets.

Algorithm

Sliding-window log via a single Lua script on a Redis sorted set — one atomic EVALSHA round-trip per limited request.

-- KEYS[1] : sliding-window key
-- ARGV[1] : window seconds
-- ARGV[2] : limit
-- ARGV[3] : now (ms, server time)
-- ARGV[4] : request id (uuid)
local min = tonumber(ARGV[3]) - (tonumber(ARGV[1]) * 1000)
redis.call('ZREMRANGEBYSCORE', KEYS[1], 0, min)
local count = redis.call('ZCARD', KEYS[1])
if count >= tonumber(ARGV[2]) then
  local oldest = redis.call('ZRANGE', KEYS[1], 0, 0, 'WITHSCORES')
  local retry_ms = (tonumber(ARGV[1]) * 1000) - (tonumber(ARGV[3]) - tonumber(oldest[2]))
  return {1, count, retry_ms}
end
redis.call('ZADD', KEYS[1], ARGV[3], ARGV[4])
redis.call('EXPIRE', KEYS[1], tonumber(ARGV[1]) + 1)
return {0, count + 1, 0}

Script is loaded once per worker with SCRIPT LOAD; requests use EVALSHA and fall back to EVAL on NOSCRIPT. Sorted-set size is bounded by limit + 1 entries per active (user, class) tuple — ~3 KB peak at default caps, aggressively cleaned by ZREMRANGEBYSCORE.

Key layout

{REDIS_KEY_PREFIX}:rl:{class}:{id_type}:{id}
  • id_type: u (user_id), k (api_key_hash), i (ip).
  • Scoping by class lets operators tune each surface independently and makes the stats endpoint (future) trivial.

Config shape

Extend AppConfig in config.py:

ENABLE_RATE_LIMITING              # bool, default False (also boot-overridable via env)
RATE_LIMITING_FAIL_OPEN           # bool, default True
RATE_LIMITING_CHAT_LIMIT          # int,  default 30
RATE_LIMITING_CHAT_WINDOW         # int,  default 60  (seconds)
RATE_LIMITING_EMBEDDINGS_LIMIT    # int,  default 20
RATE_LIMITING_EMBEDDINGS_WINDOW   # int,  default 60
RATE_LIMITING_UPLOADS_LIMIT       # int,  default 10
RATE_LIMITING_UPLOADS_WINDOW      # int,  default 60
RATE_LIMITING_ADMIN_LIMIT         # int,  default 60
RATE_LIMITING_ADMIN_WINDOW        # int,  default 60

Defaults are conservative so operators who flip the switch don't get surprise 429s on normal usage.

Failure semantics

  • Redis down / timeout → fail-open by default (configurable). Reasoning: a Redis outage must never take the app offline. Middleware imposes a 50 ms timeout on the Lua call; exceeded or errored → allow the request, log at DEBUG.
  • Limit exceeded → respond 429 Too Many Requests with Retry-After seconds and standard RateLimit-Limit / RateLimit-Remaining / RateLimit-Reset headers (draft-ietf-httpapi-ratelimit-headers-07 form). Body: {"detail": "Rate limit exceeded for <class>"}.

Middleware placement

RedirectMiddleware
  → SecurityHeadersMiddleware
  → CommitSessionMiddleware
  → AuthTokenMiddleware
  → RateLimitMiddleware            ← new
  → WebsocketUpgradeGuardMiddleware
  → CORSMiddleware

Rationale: sits after AuthTokenMiddleware (so it can read request.state.token without re-parsing), before WebsocketUpgradeGuardMiddleware and CORSMiddleware (so it short-circuits before any WS/Engine.IO negotiation and before CORS preflight costs).

Why this is worth accepting

  • Fills a concrete security gap with a solution entirely built from primitives the project already ships (Redis client, AppConfig, pure-ASGI middleware convention, existing RateLimiter algorithm). No new dependencies.
  • Backward compatible by default — off unless ENABLE_RATE_LIMITING=true. Operators upgrade without noticing.
  • Bounded blast radius — additive middleware, short-circuits before routing, fail-open, skips health probes.
  • Matches project taste — identical conventions to utils/asgi_middleware.py; uses app.state.redis exactly like the Socket.IO pool, Yjs manager, and task listener.
  • Small surface — one new file, four modified files, one new test file. Estimated < 500 net LOC including tests.
  • Foundation for follow-ups — admin UI, per-user overrides, OTel counters, and migration of the signin limiter can all ride on this primitive in later PRs without breaking anything.

Proposed PR sequence (for transparency; only #1 is up for debate here)

  1. This PR — middleware + config + env flag + tests, disabled by default.
  2. OTel counters (openwebui.ratelimit.hits{class,outcome}) and a minimal /api/v1/configs/rate-limits read endpoint.
  3. Admin UI panel in src/routes/(app)/admin/settings/.
  4. Per-user and per-role overrides.
  5. Migration of signin_rate_limiter to the new primitive.

Open questions for maintainers

  • Preferred default caps? The numbers above are a starting point; happy to adjust.
  • Fail-open vs fail-closed default? I propose fail-open for availability; fail-closed is available via config.
  • Is a regex-based route-to-class table acceptable, or would you prefer per-route Depends(rate_limit("chat")) decorators? Middleware is faster and centralises policy; decorators are more explicit. Happy to switch before implementation.

Happy to iterate scope, defaults, and algorithm choice here before opening the PR.

Originally created by @Srujan4812 on GitHub (May 12, 2026). Original GitHub issue: https://github.com/open-webui/open-webui/issues/24631 # RFC: Backend-only, config-driven, per-user endpoint rate limiting middleware ## Summary Add a pure-ASGI middleware that applies **per-user, per-endpoint-class** rate limits across the API surface (`chat`, `embeddings`, `uploads`, `admin`), using the existing Redis connection and `AppConfig` plumbing. Backend-only, off by default, zero UI and zero telemetry changes in this RFC. Admin-UI bindings, observability, and per-user overrides are intentionally deferred to follow-ups once this lands. Today the project only rate-limits signin (`RateLimiter` at 15 req / 180 s in `routers/auths.py`). Every other endpoint — including the ones that directly drive upstream LLM cost — is unthrottled. ## Problem A legitimately-issued or compromised JWT / API key can: 1. Exhaust upstream LLM budgets via `/api/chat/completions` (Ollama / OpenAI / Anthropic / Gemini) with no per-user ceiling. 2. Saturate the embedding path (`/api/embeddings`, `/api/v1/retrieval/*`) — `SentenceTransformer` calls use the shared anyio thread pool, so a burst starves every other handler in the worker. 3. Fill disk or S3 via uploads (`/api/v1/files`, `/api/v1/retrieval/process/*`, `/api/v1/knowledge/*`) — only a total-size cap exists, no per-user rate. 4. Mutate admin config at machine speed (`/api/v1/configs/*`, `/api/v1/models/*`, `/api/v1/groups/*`) if an admin token is misused or a SCIM bulk sync is misconfigured. The algorithmic primitive (`utils/rate_limit.RateLimiter`, Redis rolling window with in-memory fallback) already exists. It just isn't wired into anything beyond signin. ## Scope (intentionally small) **In scope for this RFC / first PR:** - One new file: `backend/open_webui/utils/rate_limit_middleware.py`. - Middleware wiring in `backend/open_webui/main.py`. - New `AppConfig` fields in `backend/open_webui/config.py` and an env bootstrap flag in `backend/open_webui/env.py`. - One error message in `backend/open_webui/constants.py`. - Unit tests in `backend/open_webui/test/util/test_rate_limit_middleware.py` using `fakeredis.aioredis`. - Feature ships **disabled by default**. Enabling it is additive; no response semantics change for requests under the limit. **Explicitly out of scope for this RFC (deferred):** - Admin UI panel (no `src/` changes). - OTel metrics / tracing additions. - A `/stats` endpoint or runtime introspection API. - Per-user or per-role overrides. - Runtime-configurable route-to-class mapping (the mapping ships hard-coded; overrides come later). - Migrating the existing `signin_rate_limiter` to the new primitive. This keeps the first PR narrow enough for a single reviewer pass while establishing the primitive that the deferred work can plug into. ## Design ### Identity resolution (no DB hits) The middleware runs immediately after `AuthTokenMiddleware`, so `request.state.token` is already parsed. Priority: 1. **JWT** → decode with `utils/auth.decode_token`, take the `id` claim. No DB lookup; if `is_valid_token` revocation matters, the route handler's `get_current_user` will reject shortly after. 2. **API key** (`sk-…`) → SHA-256 first 12 chars as identity (never the raw key in a Redis key). 3. **Anonymous** → `request.client.host` (respecting `forwarded_allow_ips='*'` already configured in `uvicorn.run`). ### Endpoint classification A compiled regex table inside the middleware module — ships hard-coded for this PR: | Class | Path prefixes | |---|---| | `chat` | `/api/chat/completions`, `/api/v1/chat/completions`, `/ollama/api/chat`, `/openai/chat/completions` | | `embeddings` | `/api/embeddings`, `/api/v1/embeddings` | | `uploads` | `/api/v1/files`, `/api/v1/retrieval/process`, `/api/v1/knowledge/*/file/add` | | `admin` | `/api/v1/configs`, `/api/v1/models`, `/api/v1/groups`, `/api/v1/users` (write methods only) | Unmatched paths are not rate-limited. Skip list (always bypass): `/health`, `/ready`, `/health/db`, `/ws/*`, SPA static assets. ### Algorithm **Sliding-window log via a single Lua script on a Redis sorted set** — one atomic `EVALSHA` round-trip per limited request. ```lua -- KEYS[1] : sliding-window key -- ARGV[1] : window seconds -- ARGV[2] : limit -- ARGV[3] : now (ms, server time) -- ARGV[4] : request id (uuid) local min = tonumber(ARGV[3]) - (tonumber(ARGV[1]) * 1000) redis.call('ZREMRANGEBYSCORE', KEYS[1], 0, min) local count = redis.call('ZCARD', KEYS[1]) if count >= tonumber(ARGV[2]) then local oldest = redis.call('ZRANGE', KEYS[1], 0, 0, 'WITHSCORES') local retry_ms = (tonumber(ARGV[1]) * 1000) - (tonumber(ARGV[3]) - tonumber(oldest[2])) return {1, count, retry_ms} end redis.call('ZADD', KEYS[1], ARGV[3], ARGV[4]) redis.call('EXPIRE', KEYS[1], tonumber(ARGV[1]) + 1) return {0, count + 1, 0} ``` Script is loaded once per worker with `SCRIPT LOAD`; requests use `EVALSHA` and fall back to `EVAL` on `NOSCRIPT`. Sorted-set size is bounded by `limit + 1` entries per active (user, class) tuple — ~3 KB peak at default caps, aggressively cleaned by `ZREMRANGEBYSCORE`. ### Key layout ``` {REDIS_KEY_PREFIX}:rl:{class}:{id_type}:{id} ``` - `id_type`: `u` (user_id), `k` (api_key_hash), `i` (ip). - Scoping by class lets operators tune each surface independently and makes the stats endpoint (future) trivial. ### Config shape Extend `AppConfig` in `config.py`: ```python ENABLE_RATE_LIMITING # bool, default False (also boot-overridable via env) RATE_LIMITING_FAIL_OPEN # bool, default True RATE_LIMITING_CHAT_LIMIT # int, default 30 RATE_LIMITING_CHAT_WINDOW # int, default 60 (seconds) RATE_LIMITING_EMBEDDINGS_LIMIT # int, default 20 RATE_LIMITING_EMBEDDINGS_WINDOW # int, default 60 RATE_LIMITING_UPLOADS_LIMIT # int, default 10 RATE_LIMITING_UPLOADS_WINDOW # int, default 60 RATE_LIMITING_ADMIN_LIMIT # int, default 60 RATE_LIMITING_ADMIN_WINDOW # int, default 60 ``` Defaults are conservative so operators who flip the switch don't get surprise 429s on normal usage. ### Failure semantics - **Redis down / timeout** → fail-open by default (configurable). Reasoning: a Redis outage must never take the app offline. Middleware imposes a 50 ms timeout on the Lua call; exceeded or errored → allow the request, log at DEBUG. - **Limit exceeded** → respond `429 Too Many Requests` with `Retry-After` seconds and standard `RateLimit-Limit` / `RateLimit-Remaining` / `RateLimit-Reset` headers (draft-ietf-httpapi-ratelimit-headers-07 form). Body: `{"detail": "Rate limit exceeded for <class>"}`. ### Middleware placement ``` RedirectMiddleware → SecurityHeadersMiddleware → CommitSessionMiddleware → AuthTokenMiddleware → RateLimitMiddleware ← new → WebsocketUpgradeGuardMiddleware → CORSMiddleware ``` Rationale: sits after `AuthTokenMiddleware` (so it can read `request.state.token` without re-parsing), before `WebsocketUpgradeGuardMiddleware` and `CORSMiddleware` (so it short-circuits before any WS/Engine.IO negotiation and before CORS preflight costs). ## Why this is worth accepting - **Fills a concrete security gap** with a solution entirely built from primitives the project already ships (Redis client, `AppConfig`, pure-ASGI middleware convention, existing `RateLimiter` algorithm). No new dependencies. - **Backward compatible by default** — off unless `ENABLE_RATE_LIMITING=true`. Operators upgrade without noticing. - **Bounded blast radius** — additive middleware, short-circuits before routing, fail-open, skips health probes. - **Matches project taste** — identical conventions to `utils/asgi_middleware.py`; uses `app.state.redis` exactly like the Socket.IO pool, Yjs manager, and task listener. - **Small surface** — one new file, four modified files, one new test file. Estimated < 500 net LOC including tests. - **Foundation for follow-ups** — admin UI, per-user overrides, OTel counters, and migration of the signin limiter can all ride on this primitive in later PRs without breaking anything. ## Proposed PR sequence (for transparency; only #1 is up for debate here) 1. **This PR** — middleware + config + env flag + tests, disabled by default. 2. OTel counters (`openwebui.ratelimit.hits{class,outcome}`) and a minimal `/api/v1/configs/rate-limits` read endpoint. 3. Admin UI panel in `src/routes/(app)/admin/settings/`. 4. Per-user and per-role overrides. 5. Migration of `signin_rate_limiter` to the new primitive. ## Open questions for maintainers - Preferred default caps? The numbers above are a starting point; happy to adjust. - Fail-open vs fail-closed default? I propose fail-open for availability; fail-closed is available via config. - Is a regex-based route-to-class table acceptable, or would you prefer per-route `Depends(rate_limit("chat"))` decorators? Middleware is faster and centralises policy; decorators are more explicit. Happy to switch before implementation. Happy to iterate scope, defaults, and algorithm choice here before opening the PR.
GiteaMirror commented 2026-05-15 16:22:42 -05:00
Author
Owner
Copy Link

@owui-terminator[bot] commented on GitHub (May 12, 2026):

🔍 Related Issues Found

I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions:

  1. 🟣 #21632 feat: Firebase Auth & API Rate limit
    This is the closest prior request for API rate limiting. It explicitly asks for throttling API usage for approved users, which overlaps with the new RFC’s backend rate limiting goals even though the proposed mechanism is broader and per-endpoint.
    by amjiddader

  2. 🟣 #23323 feat: built-in per-user and per-group token/message usage limits
    This issue asks for built-in enforcement of per-user and per-group usage limits. While it focuses on token/message quotas rather than request-rate middleware, it addresses the same unmet need of platform-level limits on user consumption and is a strong conceptual follow-up.
    by smorello87

  3. 🟣 #23400 bug: [Security] Unauthenticated /api/config endpoint and missing rate limiting on /signup
    This security issue mentions missing rate limiting on /signup, showing prior concern about absent rate limiting in the backend. It is related as evidence of the project already having endpoint rate-limit gaps, though it targets a different route and threat model.
    by autoailabadmin

💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead.

This comment was generated automatically. React with 👍 if helpful, 👎 if not.

<!-- gh-comment-id:4435066851 --> @owui-terminator[bot] commented on GitHub (May 12, 2026): <!-- terminator-bot:related-issues-reply --> 🔍 **Related Issues Found** I found some existing issues that might be related. Please check if any of these are duplicates or contain helpful solutions: 1. 🟣 [#21632](https://github.com/open-webui/open-webui/issues/21632) **feat: Firebase Auth & API Rate limit** *This is the closest prior request for API rate limiting. It explicitly asks for throttling API usage for approved users, which overlaps with the new RFC’s backend rate limiting goals even though the proposed mechanism is broader and per-endpoint.* *by amjiddader* 2. 🟣 [#23323](https://github.com/open-webui/open-webui/issues/23323) **feat: built-in per-user and per-group token/message usage limits** *This issue asks for built-in enforcement of per-user and per-group usage limits. While it focuses on token/message quotas rather than request-rate middleware, it addresses the same unmet need of platform-level limits on user consumption and is a strong conceptual follow-up.* *by smorello87* 3. 🟣 [#23400](https://github.com/open-webui/open-webui/issues/23400) **bug: [Security] Unauthenticated /api/config endpoint and missing rate limiting on /signup** *This security issue mentions missing rate limiting on /signup, showing prior concern about absent rate limiting in the backend. It is related as evidence of the project already having endpoint rate-limit gaps, though it targets a different route and threat model.* *by autoailabadmin* --- 💡 If your issue is a duplicate, please close it and add any additional details to the existing issue instead. *This comment was generated automatically.* React with 👍 if helpful, 👎 if not.
GiteaMirror commented 2026-05-15 16:22:42 -05:00
Author
Owner
Copy Link

@Classic298 commented on GitHub (May 12, 2026):

As per issue template - discussions not issues

<!-- gh-comment-id:4435165839 --> @Classic298 commented on GitHub (May 12, 2026): As per issue template - discussions not issues
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#91104
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?