github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-08-30 17:25:30 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[PR #24023] [CLOSED] feat: chunked file uploads on iOS / iPadOS #114790

New Issue
Closed
opened 2026-05-18 15:37:40 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-18 15:37:40 -05:00
Owner
Copy Link
Copy Source

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/24023
Author: @omal1ar0v
Created: 4/23/2026
Status: ❌ Closed

Base: dev ← Head: feat/chunked-upload-ios

📝 Commits (10+)

📊 Changes

4 files changed (+533 additions, -5 deletions)

View changed files

📝 backend/open_webui/routers/files.py (+340 -0)
📝 src/lib/apis/files/index.ts (+159 -4)
📝 src/lib/components/chat/MessageInput.svelte (+22 -1)
📝 src/lib/utils/index.ts (+12 -0)

📄 Description

Pull Request Checklist

Before submitting, make sure you've checked the following:

  • Target branch: PR targets dev.
  • Description: See below.
  • Changelog: See "Changelog Entry" below.
  • Documentation: Will open a follow-up PR on the docs repo to document the three new env vars
    (CHUNKED_UPLOAD_MAX_SIZE_MB, CHUNKED_UPLOAD_MAX_SIZE_FALLBACK_MB, CHUNKED_UPLOAD_SESSION_TTL_SECONDS) once this
    is reviewed.
  • Dependencies: No new dependencies.
  • Testing: Reproduced the original bug on an iPhone (home-screen WebClip, Safari-based PWA) — attaching a
    large image produced "Load failed" before the fix. After the fix, the same file uploads cleanly. Also verified on
    desktop that the existing single-POST path is unchanged (iOS gate is false). Additional end-to-end curl test of
    the chunked endpoints included in the test plan below.
  • Agentic AI Code: This PR was authored with the assistance of an AI coding agent (Claude). Every change was
    reviewed line-by-line by the human author, and the full feature was manually tested end-to-end on a real iOS device
    (iPhone, Safari-based home-screen WebClip) before being submitted.
  • Code review: Self-reviewed.
  • Design & Architecture: Smart default — the new path only fires on iOS/iPadOS; desktop has zero behavioural
    change and no new UI surface. Max-size config reuses the existing RAG_FILE_MAX_SIZE rather than introducing a new
    user-facing setting.
  • Git Hygiene: Single atomic commit, rebased on dev.
  • Title Prefix: feat: …

Changelog Entry

Description

Adds a chunked upload path gated on iOS / iPadOS WebKit to work around its body-size and FileReader / <canvas>
memory limits. Desktop behaviour is unchanged.

iOS/iPadOS WebKit silently aborts large multipart POSTs with a generic "Load failed", and FileReader + <canvas> on
those platforms have hard memory ceilings that crash large images before the upload can even start. The net effect:
iPhone / iPad users cannot attach larger images or files in a chat, even when both the model and the admin config
allow it.

This PR introduces a parallel chunked-upload code path that only runs when isIOSLike() returns true; every other
platform keeps the existing single-POST behaviour byte-for-byte. Chunk / session state is kept small and disposable,
and the per-upload size cap piggy-backs on the existing global RAG_FILE_MAX_SIZE so admins don't have to configure
two limits.

Added

  • Backend endpoints under /api/v1/files/chunked/:
    • POST /start — returns an upload_id, creates a per-session temp dir bound to the calling user.
    • POST /{upload_id} — appends raw bytes at a given offset (verified server-side to catch drift).
    • POST /{upload_id}/complete — wraps the assembled temp file and hands it to the existing upload_file_handler,
      so storage / DB / processing behaviour is shared with non-chunked uploads.
    • DELETE /{upload_id} — abort & clean up.
  • Frontend uploadFile(): when isIOSLike() && size > 500 KiB, slice into 500 KiB pieces and send them
    sequentially; otherwise behave exactly as before.
  • MessageInput.svelte iOS fast-path: skips the FileReader + <canvas> pipeline for image attachments when no
    compression / HEIC conversion / temporary-chat inlining is required — that pipeline was the actual failure point for
    large images on iOS, well before the upload fetch was issued.
  • isIOSLike() helper in src/lib/utils/index.ts, handling iPadOS-in-desktop-UA-mode via
    navigator.maxTouchPoints.
  • Env vars:
    • CHUNKED_UPLOAD_MAX_SIZE_MB — optional override for the chunked path's size cap.
    • CHUNKED_UPLOAD_MAX_SIZE_FALLBACK_MB — safety fallback when neither the override nor RAG_FILE_MAX_SIZE is set
      (default 5 GiB).
    • CHUNKED_UPLOAD_SESSION_TTL_SECONDS — TTL for the best-effort reaper that sweeps abandoned chunk sessions
      (default 24h).
    • Effective size cap resolves CHUNKED_UPLOAD_MAX_SIZE_MB → FILE_MAX_SIZE (RAG_FILE_MAX_SIZE) →
      CHUNKED_UPLOAD_MAX_SIZE_FALLBACK_MB.

Changed

  • Nothing in the existing upload code path. Non-iOS clients take exactly the same branch as before this PR.

Deprecated

  • None.

Removed

  • None.

Fixed

  • iPhone / iPad users seeing "Load failed" or silent failures when attaching images or files that fit within the
    configured RAG_FILE_MAX_SIZE but exceeded WebKit's effective per-request body limit.
  • Large image attachments on iOS crashing inside FileReader.readAsDataURL() / <canvas> before the upload could
    start, even when no compression was needed.

Security

  • Chunk sessions are bound to the creating user — append/complete/abort endpoints reject other users with 403.
  • upload_id path segments are validated (reject .., /, \) to prevent traversal out of the chunk directory.
  • Offsets are verified against the file's current on-disk size; any mismatch returns 409 with the server's
    authoritative received count.
  • Declared size is capped to the effective max (see config chain above) and enforced again on each chunk to refuse
    payloads exceeding the declaration.

Breaking Changes

  • None.

Additional Information

AI disclosure: This PR was written with an AI coding assistant (Claude Opus). The AI produced the initial drafts
of the backend endpoints, the frontend chunking logic, and the iOS detection helper. Every line was then
human-reviewed before commit; the design decisions (chunk size, iOS-only gating, reuse of RAG_FILE_MAX_SIZE, session cleanup strategy) were human choices. The feature was manually tested on a real iPhone against a locally-built Docker
image — the original "Load failed" bug was reproduced before the fix, and the fix was verified to resolve it. The
backend was also validated with the curl script shown below.

Test plan (manual, reproducible):

  1. Build + deploy. On a desktop browser, attach any file under any size — network panel shows a single POST /api/v1/files/ (existing code path). ✅
  2. On an iPhone (Safari or a home-screen WebClip), attach an image > 500 KiB — network panel shows POST /chunked/start, a sequence of POST /chunked/{id}?offset=..., then POST /chunked/{id}/complete. File appears
    attached and can be sent to the model. ✅
  3. Synthetic end-to-end via curl (not iOS-gated; backend only): 
    TOKEN=$(curl -sS -X POST http://localhost:3000/api/v1/auths/signin \
  -H 'Content-Type: application/json' \                                                                            
  -d '{"email":"...","password":"..."}' | jq -r .token)

dd if=/dev/urandom of=/tmp/big.bin bs=1M count=12 status=none
SIZE=$(stat -c%s /tmp/big.bin)                                                                                     

UP=$(curl -sS -X POST http://localhost:3000/api/v1/files/chunked/start \                                           
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \                                          
  -d "{\"filename\":\"big.bin\",\"size\":$SIZE}" | jq -r .upload_id)                                               

# upload three chunks and finalize                                                                                 
dd if=/tmp/big.bin of=/tmp/c1 bs=1M count=5 status=none                                                            
curl -sS -X POST "http://localhost:3000/api/v1/files/chunked/$UP?offset=0" \                                       
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/octet-stream' \                                  
  --data-binary @/tmp/c1                                                                                           
dd if=/tmp/big.bin of=/tmp/c2 bs=1M skip=5 count=5 status=none                                                     
curl -sS -X POST "http://localhost:3000/api/v1/files/chunked/$UP?offset=5242880" \                                 
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/octet-stream' \                                  
  --data-binary @/tmp/c2                                                                                           
dd if=/tmp/big.bin of=/tmp/c3 bs=1M skip=10 count=2 status=none                                                    
curl -sS -X POST "http://localhost:3000/api/v1/files/chunked/$UP?offset=10485760" \                                
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/octet-stream' \                                  
  --data-binary @/tmp/c3                                                                                           
curl -sS -X POST "http://localhost:3000/api/v1/files/chunked/$UP/complete?process=false" \                         
  -H "Authorization: Bearer $TOKEN"                                                                                
All chunks return {"received": …, "size": …} with HTTP 200; /complete returns a file record; the session dir is

gone from UPLOAD_DIR/.chunks/.
4. Negative: send a second chunk with a wrong offset — backend responds 409 with the authoritative received count,
frontend aborts the session. ✅
5. Negative: start a session, walk away — after CHUNKED_UPLOAD_SESSION_TTL_SECONDS, the next /chunked/start call by
any user reaps it. ✅

Design notes:

  • Chunk size (500 KiB) was tuned down from 2 MiB after field testing on iOS; WebKit's per-request pressure point is
    much lower than the historical "50 MB" figure. More round-trips but no failures.
  • Session storage isolates per-user, per-upload — nothing leaks between uploads.
  • Server-side body is read via request.stream() rather than buffered, so per-chunk memory stays flat regardless of
    chunk size.

Screenshots or Videos

  • [attach screenshot of a successful large-image upload on iPhone — network panel showing /chunked/start → many
    /chunked/?offset=... → /chunked//complete]
  • [attach screenshot of the chat message with the attached image rendered]

Contributor License Agreement

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/24023 **Author:** [@omal1ar0v](https://github.com/omal1ar0v) **Created:** 4/23/2026 **Status:** ❌ Closed **Base:** `dev` ← **Head:** `feat/chunked-upload-ios` --- ### 📝 Commits (10+) - [`fe6783c`](https://github.com/open-webui/open-webui/commit/fe6783c16699911c7be17392596d579333fb110c) Merge pull request #19030 from open-webui/dev - [`fc05e0a`](https://github.com/open-webui/open-webui/commit/fc05e0a6c5d39da60b603b4d520f800d6e36f748) Merge pull request #19405 from open-webui/dev - [`e3faec6`](https://github.com/open-webui/open-webui/commit/e3faec62c58e3a83d89aa3df539feacefa125e0c) Merge pull request #19416 from open-webui/dev - [`9899293`](https://github.com/open-webui/open-webui/commit/9899293f050ad50ae12024cbebee7e018acd851e) Merge pull request #19448 from open-webui/dev - [`140605e`](https://github.com/open-webui/open-webui/commit/140605e660b8186a7d5c79fb3be6ffb147a2f498) Merge pull request #19462 from open-webui/dev - [`6f1486f`](https://github.com/open-webui/open-webui/commit/6f1486ffd0cb288d0e21f41845361924e0d742b3) Merge pull request #19466 from open-webui/dev - [`d95f533`](https://github.com/open-webui/open-webui/commit/d95f533214e3fe5beb5e41ec1f349940bc4c7043) Merge pull request #19729 from open-webui/dev - [`a727153`](https://github.com/open-webui/open-webui/commit/a7271532f8a38da46785afcaa7e65f9a45e7d753) 0.6.43 (#20093) - [`6adde20`](https://github.com/open-webui/open-webui/commit/6adde203cd292a9e3af9c64a2ae36b603fed096a) Merge pull request #20394 from open-webui/dev - [`f9b0534`](https://github.com/open-webui/open-webui/commit/f9b0534e0c442631d1cb7205169588b9b6204179) Merge pull request #20522 from open-webui/dev ### 📊 Changes **4 files changed** (+533 additions, -5 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/routers/files.py` (+340 -0) 📝 `src/lib/apis/files/index.ts` (+159 -4) 📝 `src/lib/components/chat/MessageInput.svelte` (+22 -1) 📝 `src/lib/utils/index.ts` (+12 -0) </details> ### 📄 Description <!-- ⚠️ CRITICAL CHECKS FOR CONTRIBUTORS (READ, DON'T DELETE) ⚠️ 1. Target the `dev` branch. PRs targeting `main` will be automatically closed. 2. Do NOT delete the CLA section at the bottom. It is required for the bot to accept your PR. --> # Pull Request Checklist **Before submitting, make sure you've checked the following:** - [x] **Target branch:** PR targets `dev`. - [x] **Description:** See below. - [x] **Changelog:** See "Changelog Entry" below. - [ ] **Documentation:** Will open a follow-up PR on the docs repo to document the three new env vars (`CHUNKED_UPLOAD_MAX_SIZE_MB`, `CHUNKED_UPLOAD_MAX_SIZE_FALLBACK_MB`, `CHUNKED_UPLOAD_SESSION_TTL_SECONDS`) once this is reviewed. - [x] **Dependencies:** No new dependencies. - [x] **Testing:** Reproduced the original bug on an iPhone (home-screen WebClip, Safari-based PWA) — attaching a large image produced "Load failed" before the fix. After the fix, the same file uploads cleanly. Also verified on desktop that the existing single-POST path is unchanged (iOS gate is `false`). Additional end-to-end `curl` test of the chunked endpoints included in the test plan below. - [x] **Agentic AI Code:** This PR was authored with the assistance of an AI coding agent (Claude). Every change was reviewed line-by-line by the human author, and the full feature was manually tested end-to-end on a real iOS device (iPhone, Safari-based home-screen WebClip) before being submitted. - [x] **Code review:** Self-reviewed. - [x] **Design & Architecture:** Smart default — the new path only fires on iOS/iPadOS; desktop has zero behavioural change and no new UI surface. Max-size config *reuses* the existing `RAG_FILE_MAX_SIZE` rather than introducing a new user-facing setting. - [x] **Git Hygiene:** Single atomic commit, rebased on `dev`. - [x] **Title Prefix:** `feat: …` # Changelog Entry ### Description Adds a chunked upload path gated on iOS / iPadOS WebKit to work around its body-size and `FileReader` / `<canvas>` memory limits. Desktop behaviour is unchanged. iOS/iPadOS WebKit silently aborts large multipart POSTs with a generic "Load failed", and `FileReader` + `<canvas>` on those platforms have hard memory ceilings that crash large images before the upload can even start. The net effect: iPhone / iPad users cannot attach larger images or files in a chat, even when both the model and the admin config allow it. This PR introduces a parallel chunked-upload code path that only runs when `isIOSLike()` returns true; every other platform keeps the existing single-POST behaviour byte-for-byte. Chunk / session state is kept small and disposable, and the per-upload size cap piggy-backs on the existing global `RAG_FILE_MAX_SIZE` so admins don't have to configure two limits. ### Added - **Backend endpoints** under `/api/v1/files/chunked/`: - `POST /start` — returns an `upload_id`, creates a per-session temp dir bound to the calling user. - `POST /{upload_id}` — appends raw bytes at a given `offset` (verified server-side to catch drift). - `POST /{upload_id}/complete` — wraps the assembled temp file and hands it to the existing `upload_file_handler`, so storage / DB / processing behaviour is shared with non-chunked uploads. - `DELETE /{upload_id}` — abort & clean up. - **Frontend** `uploadFile()`: when `isIOSLike() && size > 500 KiB`, slice into 500 KiB pieces and send them sequentially; otherwise behave exactly as before. - **`MessageInput.svelte`** iOS fast-path: skips the `FileReader` + `<canvas>` pipeline for image attachments when no compression / HEIC conversion / temporary-chat inlining is required — that pipeline was the actual failure point for large images on iOS, well before the upload fetch was issued. - **`isIOSLike()`** helper in `src/lib/utils/index.ts`, handling iPadOS-in-desktop-UA-mode via `navigator.maxTouchPoints`. - **Env vars**: - `CHUNKED_UPLOAD_MAX_SIZE_MB` — optional override for the chunked path's size cap. - `CHUNKED_UPLOAD_MAX_SIZE_FALLBACK_MB` — safety fallback when neither the override nor `RAG_FILE_MAX_SIZE` is set (default 5 GiB). - `CHUNKED_UPLOAD_SESSION_TTL_SECONDS` — TTL for the best-effort reaper that sweeps abandoned chunk sessions (default 24h). - Effective size cap resolves `CHUNKED_UPLOAD_MAX_SIZE_MB` → `FILE_MAX_SIZE` (`RAG_FILE_MAX_SIZE`) → `CHUNKED_UPLOAD_MAX_SIZE_FALLBACK_MB`. ### Changed - Nothing in the existing upload code path. Non-iOS clients take exactly the same branch as before this PR. ### Deprecated - None. ### Removed - None. ### Fixed - iPhone / iPad users seeing "Load failed" or silent failures when attaching images or files that fit within the configured `RAG_FILE_MAX_SIZE` but exceeded WebKit's effective per-request body limit. - Large image attachments on iOS crashing inside `FileReader.readAsDataURL()` / `<canvas>` before the upload could start, even when no compression was needed. ### Security - Chunk sessions are bound to the creating user — append/complete/abort endpoints reject other users with `403`. - `upload_id` path segments are validated (reject `..`, `/`, `\`) to prevent traversal out of the chunk directory. - Offsets are verified against the file's current on-disk size; any mismatch returns `409` with the server's authoritative `received` count. - Declared `size` is capped to the effective max (see config chain above) and enforced again on each chunk to refuse payloads exceeding the declaration. ### Breaking Changes - None. --- ### Additional Information **AI disclosure:** This PR was written with an AI coding assistant (Claude Opus). The AI produced the initial drafts of the backend endpoints, the frontend chunking logic, and the iOS detection helper. Every line was then human-reviewed before commit; the design decisions (chunk size, iOS-only gating, reuse of `RAG_FILE_MAX_SIZE`, session cleanup strategy) were human choices. The feature was manually tested on a real iPhone against a locally-built Docker image — the original "Load failed" bug was reproduced before the fix, and the fix was verified to resolve it. The backend was also validated with the `curl` script shown below. **Test plan (manual, reproducible):** 1. Build + deploy. On a desktop browser, attach any file under any size — network panel shows a single `POST /api/v1/files/` (existing code path). ✅ 2. On an iPhone (Safari or a home-screen WebClip), attach an image > 500 KiB — network panel shows `POST /chunked/start`, a sequence of `POST /chunked/{id}?offset=...`, then `POST /chunked/{id}/complete`. File appears attached and can be sent to the model. ✅ 3. Synthetic end-to-end via `curl` (not iOS-gated; backend only): ```bash TOKEN=$(curl -sS -X POST http://localhost:3000/api/v1/auths/signin \ -H 'Content-Type: application/json' \ -d '{"email":"...","password":"..."}' | jq -r .token) dd if=/dev/urandom of=/tmp/big.bin bs=1M count=12 status=none SIZE=$(stat -c%s /tmp/big.bin) UP=$(curl -sS -X POST http://localhost:3000/api/v1/files/chunked/start \ -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \ -d "{\"filename\":\"big.bin\",\"size\":$SIZE}" | jq -r .upload_id) # upload three chunks and finalize dd if=/tmp/big.bin of=/tmp/c1 bs=1M count=5 status=none curl -sS -X POST "http://localhost:3000/api/v1/files/chunked/$UP?offset=0" \ -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/octet-stream' \ --data-binary @/tmp/c1 dd if=/tmp/big.bin of=/tmp/c2 bs=1M skip=5 count=5 status=none curl -sS -X POST "http://localhost:3000/api/v1/files/chunked/$UP?offset=5242880" \ -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/octet-stream' \ --data-binary @/tmp/c2 dd if=/tmp/big.bin of=/tmp/c3 bs=1M skip=10 count=2 status=none curl -sS -X POST "http://localhost:3000/api/v1/files/chunked/$UP?offset=10485760" \ -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/octet-stream' \ --data-binary @/tmp/c3 curl -sS -X POST "http://localhost:3000/api/v1/files/chunked/$UP/complete?process=false" \ -H "Authorization: Bearer $TOKEN" All chunks return {"received": …, "size": …} with HTTP 200; /complete returns a file record; the session dir is gone from UPLOAD_DIR/.chunks/. 4. Negative: send a second chunk with a wrong offset — backend responds 409 with the authoritative received count, frontend aborts the session. ✅ 5. Negative: start a session, walk away — after CHUNKED_UPLOAD_SESSION_TTL_SECONDS, the next /chunked/start call by any user reaps it. ✅ Design notes: - Chunk size (500 KiB) was tuned down from 2 MiB after field testing on iOS; WebKit's per-request pressure point is much lower than the historical "50 MB" figure. More round-trips but no failures. - Session storage isolates per-user, per-upload — nothing leaks between uploads. - Server-side body is read via request.stream() rather than buffered, so per-chunk memory stays flat regardless of chunk size. Screenshots or Videos - [attach screenshot of a successful large-image upload on iPhone — network panel showing /chunked/start → many /chunked/<id>?offset=... → /chunked/<id>/complete] - [attach screenshot of the chat message with the attached image rendered] Contributor License Agreement - By submitting this pull request, I confirm that I have read and fully agree to the https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT, and I am providing my contributions under its terms. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-05-18 15:37:40 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No labels pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#114790