github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-07-16 06:03:26 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[PR #24633] [CLOSED] feat: backend-only, config-driven, per-user endpoint rate limiting middleware #131415

New Issue
Closed
opened 2026-05-21 16:52:12 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-21 16:52:12 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/24633
Author: @Srujan4812
Created: 5/12/2026
Status: ❌ Closed

Base: dev ← Head: feature/rate-limit-middleware

📝 Commits (10+)

📊 Changes

4 files changed (+923 additions, -0 deletions)

View changed files

📝 backend/open_webui/config.py (+86 -0)
📝 backend/open_webui/main.py (+31 -0)
➕ backend/open_webui/test/util/test_rate_limit_middleware.py (+407 -0)
➕ backend/open_webui/utils/rate_limit_middleware.py (+399 -0)

📄 Description

Summary

Implements the backend-only first phase of the rate-limiting primitive proposed in #24631. Adds a pure-ASGI middleware that applies per-user, per-endpoint-class limits across four endpoint classes (chat, embeddings, uploads, admin), backed by the existing Redis connection and AppConfig plumbing.

Feature is disabled by default (ENABLE_RATE_LIMITING=false). Enabling it is additive — no response semantics change for requests under the limit.

Closes #24631 (first phase).

Scope (intentionally narrow)

In scope:

  • One new file: backend/open_webui/utils/rate_limit_middleware.py
  • Middleware wired into main.py between AuthTokenMiddleware and WebsocketUpgradeGuardMiddleware
  • 10 new PersistentConfig fields in config.py under rate_limiting.*
  • Unit tests in backend/open_webui/test/util/test_rate_limit_middleware.py (no new test dependency — follows the AsyncMock pattern from test_redis.py)

Explicitly out of scope (deferred to follow-ups, as outlined in the RFC):

  • Admin UI panel (no src/ changes)
  • OTel metrics / tracing additions
  • /stats endpoint or runtime introspection API
  • Per-user / per-role overrides
  • Migration of the existing signin_rate_limiter

Design

Endpoint classification

A compiled regex table at module import time — ships hard-coded, overridable in a follow-up. First match wins.

Class Paths
chat /api/*/chat/completions, /ollama/api/chat, /ollama/api/generate, /openai/chat/completions
embeddings /api/*/embeddings
uploads /api/v1/files, /api/v1/retrieval/process, /api/v1/knowledge/*/file/add
admin /api/v1/configs|models|groups|users — write methods only (GET/HEAD/OPTIONS are not rate-limited)

Skip list (never rate-limited): /health, /ready, /ws/*, /static/*, /assets/*, /cache/*, /_app/*, and the SPA root.

Identity resolution

Runs after AuthTokenMiddleware so request.state.token is already parsed. Priority:

  1. JWT — id claim via utils.auth.decode_token. No DB lookup. Revocation and user-existence checks happen in get_current_user downstream.
  2. API key (sk-…) — SHA-256 first 16 hex chars. Raw key never appears in Redis.
  3. Anonymous — client IP from the ASGI scope.

Algorithm

Sliding-window log via a single Lua script on a Redis sorted set — one atomic EVALSHA per classified request, with EVAL fallback on NOSCRIPT. Sorted-set size is bounded by limit + 1 entries per active (identity, class) tuple and aggressively cleaned with ZREMRANGEBYSCORE. 50 ms timeout on every Redis call.

ZREMRANGEBYSCORE key 0 (now - window*1000)
count = ZCARD key
if count >= limit then
  oldest_ts = ZRANGE key 0 0 WITHSCORES
  return {1, count, retry_after_ms}
end
ZADD key now request_id
EXPIRE key (window + 1)
return {0, count + 1, 0}

Key layout

{REDIS_KEY_PREFIX}:rl:{class}:{id_type}:{id}

id_type ∈ {u, k, i} (user / key-hash / IP). Scoping by class lets operators tune each surface independently.

Failure semantics

  • Limit exceeded → 429 Too Many Requests, JSON body {"detail": "Rate limit exceeded for <class>", "class": "<class>"}, plus standard RateLimit-Limit / RateLimit-Remaining / RateLimit-Reset / Retry-After headers.
  • Redis timeout or error → fail-open by default (RATE_LIMITING_FAIL_OPEN=true). Configurable; rationale is that a Redis incident must not take the app offline. Logged at DEBUG to avoid log floods under attack.

Middleware placement

RedirectMiddleware
  → SecurityHeadersMiddleware
  → CommitSessionMiddleware
  → AuthTokenMiddleware
  → RateLimitMiddleware            ← new
  → WebsocketUpgradeGuardMiddleware
  → CORSMiddleware

Sits after AuthTokenMiddleware (reads request.state.token without re-parsing), before WebsocketUpgradeGuardMiddleware and CORSMiddleware (short-circuits abusive traffic before WS/Engine.IO negotiation or CORS preflight costs).

Config fields

All hot-reloadable via AppConfig under rate_limiting.*. Defaults are conservative so operators who flip the switch don't get surprise 429s on normal usage.

Field Default
ENABLE_RATE_LIMITING false
RATE_LIMITING_FAIL_OPEN true
RATE_LIMITING_CHAT_LIMIT / _WINDOW 30 / 60 s
RATE_LIMITING_EMBEDDINGS_LIMIT / _WINDOW 20 / 60 s
RATE_LIMITING_UPLOADS_LIMIT / _WINDOW 10 / 60 s
RATE_LIMITING_ADMIN_LIMIT / _WINDOW 60 / 60 s

Setting any per-class limit to 0 disables that class without any Redis traffic.

Testing

20 unit tests in test/util/test_rate_limit_middleware.py covering:

  • 15 classify() cases — all four classes plus the skip list, with admin honouring the write-method filter.
  • 4 resolve_identity() cases — JWT > API key hash > client IP > unknown.
  • Middleware is a no-op when ENABLE_RATE_LIMITING=false.
  • Skipped paths never touch Redis.
  • Allowed requests propagate RateLimit-* headers.
  • Over-limit responds 429 with Retry-After, correct content-type, and structured body.
  • Fail-open on Redis ConnectionError; fail-closed on the same condition when configured.
  • NOSCRIPT fallback path exercises EVAL.
  • Zero per-class limit disables the class without hitting Redis.
  • Redis key format is stable: {prefix}:rl:{class}:{id_type}:{id}.

Tests use unittest.mock.AsyncMock to mirror test_redis.py — no new third-party dependency.

Compatibility

  • Ships disabled. No behavioural change for existing deployments.
  • When enabled, skip list covers /health, /ready, /ws/*, static assets — probes and WebSocket traffic are unaffected.
  • Fail-open default means a Redis outage degrades to "no rate limiting" rather than "no service".
  • Requires Redis only when ENABLE_RATE_LIMITING=true (and even then fails open if Redis is unreachable).

Follow-ups (not in this PR)

Tracked against #24631:

  1. OTel counters (openwebui.ratelimit.hits{class,outcome}) and a minimal /api/v1/configs/rate-limits read endpoint.
  2. Admin UI panel in src/routes/(app)/admin/settings/.
  3. Per-user and per-role overrides.
  4. Migration of signin_rate_limiter onto this primitive.

Each follow-up is a pure addition to the primitive landed in this PR — no refactor required.

Checklist

  • Backend-only — no frontend or i18n changes
  • Disabled by default
  • No new runtime or test dependencies
  • New files pass ruff check and ruff format --check
  • Follows the pure-ASGI convention in utils/asgi_middleware.py
  • Uses the existing app.state.redis async client
  • Uses REDIS_KEY_PREFIX so multi-tenant / multi-env Redis is safe
  • Honours the existing skip-list philosophy (no impact on health, readiness, or WebSocket traffic)

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/24633 **Author:** [@Srujan4812](https://github.com/Srujan4812) **Created:** 5/12/2026 **Status:** ❌ Closed **Base:** `dev` ← **Head:** `feature/rate-limit-middleware` --- ### 📝 Commits (10+) - [`fe6783c`](https://github.com/open-webui/open-webui/commit/fe6783c16699911c7be17392596d579333fb110c) Merge pull request #19030 from open-webui/dev - [`fc05e0a`](https://github.com/open-webui/open-webui/commit/fc05e0a6c5d39da60b603b4d520f800d6e36f748) Merge pull request #19405 from open-webui/dev - [`e3faec6`](https://github.com/open-webui/open-webui/commit/e3faec62c58e3a83d89aa3df539feacefa125e0c) Merge pull request #19416 from open-webui/dev - [`9899293`](https://github.com/open-webui/open-webui/commit/9899293f050ad50ae12024cbebee7e018acd851e) Merge pull request #19448 from open-webui/dev - [`140605e`](https://github.com/open-webui/open-webui/commit/140605e660b8186a7d5c79fb3be6ffb147a2f498) Merge pull request #19462 from open-webui/dev - [`6f1486f`](https://github.com/open-webui/open-webui/commit/6f1486ffd0cb288d0e21f41845361924e0d742b3) Merge pull request #19466 from open-webui/dev - [`d95f533`](https://github.com/open-webui/open-webui/commit/d95f533214e3fe5beb5e41ec1f349940bc4c7043) Merge pull request #19729 from open-webui/dev - [`a727153`](https://github.com/open-webui/open-webui/commit/a7271532f8a38da46785afcaa7e65f9a45e7d753) 0.6.43 (#20093) - [`6adde20`](https://github.com/open-webui/open-webui/commit/6adde203cd292a9e3af9c64a2ae36b603fed096a) Merge pull request #20394 from open-webui/dev - [`f9b0534`](https://github.com/open-webui/open-webui/commit/f9b0534e0c442631d1cb7205169588b9b6204179) Merge pull request #20522 from open-webui/dev ### 📊 Changes **4 files changed** (+923 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/config.py` (+86 -0) 📝 `backend/open_webui/main.py` (+31 -0) ➕ `backend/open_webui/test/util/test_rate_limit_middleware.py` (+407 -0) ➕ `backend/open_webui/utils/rate_limit_middleware.py` (+399 -0) </details> ### 📄 Description ## Summary Implements the backend-only first phase of the rate-limiting primitive proposed in #24631. Adds a pure-ASGI middleware that applies **per-user, per-endpoint-class** limits across four endpoint classes (`chat`, `embeddings`, `uploads`, `admin`), backed by the existing Redis connection and `AppConfig` plumbing. **Feature is disabled by default** (`ENABLE_RATE_LIMITING=false`). Enabling it is additive — no response semantics change for requests under the limit. Closes #24631 (first phase). ## Scope (intentionally narrow) In scope: - One new file: `backend/open_webui/utils/rate_limit_middleware.py` - Middleware wired into `main.py` between `AuthTokenMiddleware` and `WebsocketUpgradeGuardMiddleware` - 10 new `PersistentConfig` fields in `config.py` under `rate_limiting.*` - Unit tests in `backend/open_webui/test/util/test_rate_limit_middleware.py` (no new test dependency — follows the `AsyncMock` pattern from `test_redis.py`) Explicitly out of scope (deferred to follow-ups, as outlined in the RFC): - Admin UI panel (no `src/` changes) - OTel metrics / tracing additions - `/stats` endpoint or runtime introspection API - Per-user / per-role overrides - Migration of the existing `signin_rate_limiter` ## Design ### Endpoint classification A compiled regex table at module import time — ships hard-coded, overridable in a follow-up. First match wins. | Class | Paths | |---|---| | `chat` | `/api/*/chat/completions`, `/ollama/api/chat`, `/ollama/api/generate`, `/openai/chat/completions` | | `embeddings` | `/api/*/embeddings` | | `uploads` | `/api/v1/files`, `/api/v1/retrieval/process`, `/api/v1/knowledge/*/file/add` | | `admin` | `/api/v1/configs\|models\|groups\|users` — **write methods only** (GET/HEAD/OPTIONS are not rate-limited) | Skip list (never rate-limited): `/health`, `/ready`, `/ws/*`, `/static/*`, `/assets/*`, `/cache/*`, `/_app/*`, and the SPA root. ### Identity resolution Runs after `AuthTokenMiddleware` so `request.state.token` is already parsed. Priority: 1. **JWT** — `id` claim via `utils.auth.decode_token`. No DB lookup. Revocation and user-existence checks happen in `get_current_user` downstream. 2. **API key** (`sk-…`) — SHA-256 first 16 hex chars. Raw key never appears in Redis. 3. **Anonymous** — client IP from the ASGI scope. ### Algorithm Sliding-window log via a single Lua script on a Redis sorted set — one atomic `EVALSHA` per classified request, with `EVAL` fallback on `NOSCRIPT`. Sorted-set size is bounded by `limit + 1` entries per active `(identity, class)` tuple and aggressively cleaned with `ZREMRANGEBYSCORE`. 50 ms timeout on every Redis call. ```lua ZREMRANGEBYSCORE key 0 (now - window*1000) count = ZCARD key if count >= limit then oldest_ts = ZRANGE key 0 0 WITHSCORES return {1, count, retry_after_ms} end ZADD key now request_id EXPIRE key (window + 1) return {0, count + 1, 0} ``` ### Key layout ``` {REDIS_KEY_PREFIX}:rl:{class}:{id_type}:{id} ``` `id_type` ∈ `{u, k, i}` (user / key-hash / IP). Scoping by class lets operators tune each surface independently. ### Failure semantics - **Limit exceeded** → `429 Too Many Requests`, JSON body `{"detail": "Rate limit exceeded for <class>", "class": "<class>"}`, plus standard `RateLimit-Limit` / `RateLimit-Remaining` / `RateLimit-Reset` / `Retry-After` headers. - **Redis timeout or error** → fail-open by default (`RATE_LIMITING_FAIL_OPEN=true`). Configurable; rationale is that a Redis incident must not take the app offline. Logged at DEBUG to avoid log floods under attack. ### Middleware placement ``` RedirectMiddleware → SecurityHeadersMiddleware → CommitSessionMiddleware → AuthTokenMiddleware → RateLimitMiddleware ← new → WebsocketUpgradeGuardMiddleware → CORSMiddleware ``` Sits after `AuthTokenMiddleware` (reads `request.state.token` without re-parsing), before `WebsocketUpgradeGuardMiddleware` and `CORSMiddleware` (short-circuits abusive traffic before WS/Engine.IO negotiation or CORS preflight costs). ## Config fields All hot-reloadable via `AppConfig` under `rate_limiting.*`. Defaults are conservative so operators who flip the switch don't get surprise 429s on normal usage. | Field | Default | |---|---| | `ENABLE_RATE_LIMITING` | `false` | | `RATE_LIMITING_FAIL_OPEN` | `true` | | `RATE_LIMITING_CHAT_LIMIT` / `_WINDOW` | `30` / `60` s | | `RATE_LIMITING_EMBEDDINGS_LIMIT` / `_WINDOW` | `20` / `60` s | | `RATE_LIMITING_UPLOADS_LIMIT` / `_WINDOW` | `10` / `60` s | | `RATE_LIMITING_ADMIN_LIMIT` / `_WINDOW` | `60` / `60` s | Setting any per-class limit to `0` disables that class without any Redis traffic. ## Testing 20 unit tests in `test/util/test_rate_limit_middleware.py` covering: - 15 `classify()` cases — all four classes plus the skip list, with admin honouring the write-method filter. - 4 `resolve_identity()` cases — JWT > API key hash > client IP > `unknown`. - Middleware is a no-op when `ENABLE_RATE_LIMITING=false`. - Skipped paths never touch Redis. - Allowed requests propagate `RateLimit-*` headers. - Over-limit responds 429 with `Retry-After`, correct content-type, and structured body. - Fail-open on Redis `ConnectionError`; fail-closed on the same condition when configured. - `NOSCRIPT` fallback path exercises `EVAL`. - Zero per-class limit disables the class without hitting Redis. - Redis key format is stable: `{prefix}:rl:{class}:{id_type}:{id}`. Tests use `unittest.mock.AsyncMock` to mirror `test_redis.py` — **no new third-party dependency**. ## Compatibility - Ships disabled. No behavioural change for existing deployments. - When enabled, skip list covers `/health`, `/ready`, `/ws/*`, static assets — probes and WebSocket traffic are unaffected. - Fail-open default means a Redis outage degrades to "no rate limiting" rather than "no service". - Requires Redis only when `ENABLE_RATE_LIMITING=true` (and even then fails open if Redis is unreachable). ## Follow-ups (not in this PR) Tracked against #24631: 1. OTel counters (`openwebui.ratelimit.hits{class,outcome}`) and a minimal `/api/v1/configs/rate-limits` read endpoint. 2. Admin UI panel in `src/routes/(app)/admin/settings/`. 3. Per-user and per-role overrides. 4. Migration of `signin_rate_limiter` onto this primitive. Each follow-up is a pure addition to the primitive landed in this PR — no refactor required. ## Checklist - [x] Backend-only — no frontend or i18n changes - [x] Disabled by default - [x] No new runtime or test dependencies - [x] New files pass `ruff check` and `ruff format --check` - [x] Follows the pure-ASGI convention in `utils/asgi_middleware.py` - [x] Uses the existing `app.state.redis` async client - [x] Uses `REDIS_KEY_PREFIX` so multi-tenant / multi-env Redis is safe - [x] Honours the existing skip-list philosophy (no impact on health, readiness, or WebSocket traffic) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-05-21 16:52:12 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#131415
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?