23 Commits

Author SHA1 Message Date
G30
6ea591491e perf(images): offload validate_url() DNS resolution with asyncio.to_thread (#25825)
validate_url() calls socket.getaddrinfo() for SSRF protection, which
blocks the event loop for 100-700ms per DNS lookup. This affects:

- Image generation (get_image_data) — every external image URL
- Image editing (load_url_image) — every external image URL
- OAuth profile pictures (_process_picture_url) — every login
- Webhook delivery (post_webhook) — every notification
- Image base64 conversion (get_image_base64_from_url) — chat images

Wrap all 5 async call sites in asyncio.to_thread() so DNS resolution
runs in the thread pool. The event loop remains free to serve other
requests during the lookup.

Benchmark (3 domains, 3 trials averaged):
- BEFORE: max jitter 479ms, 1 blocked ping per trial
- AFTER:  max jitter 1ms, 0 blocked pings (324x improvement)

Co-authored-by: Tim Baek <tim@openwebui.com>
2026-06-29 02:31:24 -05:00
Classic298
de8ea08f5c Route user-supplied image-URL fetches through an SSRF-safe session (DNS rebinding, CWE-918) (#25960)
The connection-layer DNS-rebinding guard (_SSRFSafeResolver / _SSRFSafeAdapter, PR #24759) was
mounted only on SafeWebBaseLoader. Two user-reachable image fetches validate the URL then fetch
it through the shared get_session() pool with the default resolver, so a TTL-0 rebinding answer
that passed validate_url reaches an internal address at connect:

- get_image_base64_from_url (utils/files.py): user image_url on every chat completion.
- load_url_image (routers/images.py, POST /api/v1/images/edit): user-supplied image field.

Add get_ssrf_safe_session() (a one-off aiohttp session mounting _SSRFSafeResolver) and use it
for both fetches, so the connect-time IP is re-validated and a rebound loopback / RFC1918 /
metadata address is rejected. The shared pool is left untouched for the admin-configured
image-generation callers, which legitimately reach internal hosts.

Co-authored-by: dhyabi2 <32069256+dhyabi2@users.noreply.github.com>
Co-authored-by: geo-chen <2404584+geo-chen@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 23:36:53 +02:00
Timothy Jaeryang Baek
6fce92aa12 chore: format 2026-06-01 13:56:55 -07:00
Timothy Jaeryang Baek
c75fe8e74b fix: get_image_base64_from_file_id 2026-05-19 20:33:46 +04:00
Timothy Jaeryang Baek
6d0295588e refac: modernize type annotations (PEP 604 / PEP 585) 2026-05-12 17:10:15 +09:00
Classic298
d11e06f1b7 fix: prevent redirect-based SSRF and enforce collecton write access (#24524)
* fix: prevent redirect-based SSRF in get_image_base64_from_url

Cohort follow-up to PR #24491. That PR patched three call sites
(SafeWebBaseLoader._scrape, get_content_from_url, load_url_image) to
pass allow_redirects=False on the underlying HTTP client; this fourth
call site in utils/files.py was missed.

get_image_base64_from_url() is invoked from convert_url_images_to_base64
in utils/middleware.py on every /api/chat/completions request whose
message content includes an image_url part. validate_url() is called on
the originally-submitted URL only; the aiohttp session.get() call had
no allow_redirects argument and the shared session pool does not
override the aiohttp default (allow_redirects=True). An authenticated
user sending a chat message with image_url pointing at an attacker host
that 302-redirects to 169.254.169.254 / 127.0.0.1 / RFC1918 reached the
internal target. This is the most reachable variant in the redirect
cluster: no special endpoint, no admin permission, no feature flag.

Apply the same one-line fix as the other three call sites: pass
allow_redirects=AIOHTTP_CLIENT_ALLOW_REDIRECTS (defaults to False).

Reported by nayakchinmohan in GHSA-88jq-grjp-jx6f; consolidated under
GHSA-rh5x-h6pp-cjj6.

Co-authored-by: nayakchinmohan <nayakchinmohan@users.noreply.github.com>

* fix: enforce collection write access on process_file endpoint

Cohort follow-up to ba83613ff. That commit added _validate_collection_access
to process_text and process_web (the user-supplied collection_name path)
but missed process_file in the same router.

process_file accepts a user-supplied collection_name and writes the file's
embedded content into that collection via save_docs_to_vector_db. The
file_id is gated by file ownership (line 1562) but collection_name was
unchecked, so an authenticated user could append content from a file they
own into another user's knowledge-base collection by passing the victim's
KB UUID as collection_name. Identical pattern to the process_text and
process_web gaps that ba83613ff closed.

Apply the same one-line gate as the sibling endpoints: when
collection_name is user-supplied (not the default file-{file.id} fallback),
require write access via _validate_collection_access. The shared validator
delegates to filter_accessible_collections, which already correctly
handles file-* prefixes (via has_access_to_file) and KB UUIDs
(via Knowledges.check_access_by_user_id) — admins bypass.

Reported by tenbbughunters (Tenable) in GHSA-4g37-7p2c-38r9 (the
comprehensive write-path filing covering process_text / process_file /
process_web / process_youtube and the _validate_collection_access UUID
root cause), and independently re-identified for the missed process_file
call site by kodareef5 in GHSA-4m74-3cmc-293g.

Co-authored-by: tenbbughunters <tenbbughunters@users.noreply.github.com>
Co-authored-by: kodareef5 <kodareef5@users.noreply.github.com>

* fix: enforce collection write access on process_files_batch endpoint

Cohort follow-up to ba83613ff and the prior process_file fix on this
branch. process_files_batch (line 2604) is the third write endpoint in
the same router that accepts a user-supplied collection_name; it was
covered in the same Tenable filing as process_file and was missed by
the same cohort fix. The endpoint validates per-file ownership at line
2642 but does not check whether the caller has write access to the
target collection_name before save_docs_to_vector_db writes into it
at line 2683-2690 with add=True.

Apply the same one-line gate as the sibling endpoints. Validate only
when collection_name is user-supplied (truthy) so the existing fall
through behavior for the None case is unchanged.

Same Tenable / kodareef5 cohort as the previous commit.

Co-authored-by: tenbbughunters <tenbbughunters@users.noreply.github.com>
Co-authored-by: kodareef5 <kodareef5@users.noreply.github.com>

---------

Co-authored-by: nayakchinmohan <nayakchinmohan@users.noreply.github.com>
Co-authored-by: tenbbughunters <tenbbughunters@users.noreply.github.com>
Co-authored-by: kodareef5 <kodareef5@users.noreply.github.com>
2026-05-11 01:09:15 +09:00
Timothy Jaeryang Baek
6cc799b1bb chore: format 2026-04-21 15:52:00 +09:00
Timothy Jaeryang Baek
5127354b3e refac 2026-04-20 09:21:30 +09:00
Timothy Jaeryang Baek
128cf41fce refac 2026-04-17 11:12:42 +09:00
Timothy Jaeryang Baek
4866bec0f2 refac 2026-04-14 10:55:11 -05:00
Timothy Jaeryang Baek
2ddcb30b9a refac 2026-04-13 14:29:27 -05:00
Timothy Jaeryang Baek
25898116ea chore: format 2026-04-12 18:12:59 -05:00
Timothy Jaeryang Baek
27169124f2 refac: async db 2026-04-12 14:22:11 -05:00
Timothy Jaeryang Baek
1c25b06dca refac 2026-03-23 19:46:24 -05:00
Timothy Jaeryang Baek
de3317e26b refac 2026-03-17 17:58:01 -05:00
Timothy Jaeryang Baek
665f95eda3 refac 2026-01-28 01:18:39 +04:00
Timothy Jaeryang Baek
446cc0ac60 refac 2025-12-22 00:39:05 +04:00
Timothy Jaeryang Baek
f1bf4f20c5 feat: chat_file table 2025-12-21 23:17:53 +04:00
Timothy Jaeryang Baek
6fe737bf8f refac 2025-12-03 19:31:23 -05:00
Timothy Jaeryang Baek
4c28f19bdd enh/pref: convert markdown base64 images to urls
Co-Authored-By: Shirasawa <kaguyashirasawa@gmail.com>
2025-11-20 04:00:02 -05:00
Timothy Jaeryang Baek
8e5690aab4 refac: images 2025-11-04 13:30:59 -05:00
Timothy Jaeryang Baek
f1bbf3a91e refac 2025-09-23 03:05:38 -04:00
Timothy Jaeryang Baek
de7f7b3d85 refac 2025-09-23 02:40:59 -04:00