[GH-ISSUE #24235] feat: file upload hooks for custom pre-processing before storage/extraction #90973

New Issue

GiteaMirror · 2026-05-15T16:16:20-05:00

GiteaMirror commented

2026-05-15 16:16:20 -05:00

Originally created by @kirill-sch on GitHub (Apr 29, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/24235

Check Existing Issues

I have searched for all existing open AND closed issues and discussions for similar requests. I have found none that is comparable to my request.

Verify Feature Scope

I have read through and understood the scope definition for feature requests in the Issues section. I believe my feature request meets the definition and belongs in the Issues section instead of the Discussions.

Problem Description

Open WebUI has no extension point in the file upload pipeline. Once a user uploads a file, it goes staright to storage and extraction. There's no oppurtinity to inspect, transform, or reject first. A user-defined hook running at upload time would enable:

Security: virus and malware scanning before files are persisted
Compliance: DLP enforcement, PII redaction, content policy checks, audit logging
File transformation: format normalization, image preprocessing

The possible use cases are effectively unlimited, and a user-defined hook function would let each team form the upload pipeline to their specific requirements, without waiting for every individual case to be merged upstream as a one-off feature.

Desired Solution you'd like

A new section in the admin UI (alongside the existing Functions tab) where users can register Python functions that run during file upload. These hooks would be optional. If no upload functions are configured, the current upload flow is unchanged, preserving full backward compatibility.

Alternatives Considered

No response

Additional Context

If the Open WebUI team considers this a useful addition, I'd be happy to implement it and open a PR. Some initial guidance on the preferred approach would help ensure the implementation aligns with the project's direction.

Originally created by @kirill-sch on GitHub (Apr 29, 2026). Original GitHub issue: https://github.com/open-webui/open-webui/issues/24235 ### Check Existing Issues - [x] I have searched for all existing **open AND closed** issues and discussions for similar requests. I have found none that is comparable to my request. ### Verify Feature Scope - [x] I have read through and understood the scope definition for feature requests in the Issues section. I believe my feature request meets the definition and belongs in the Issues section instead of the Discussions. ### Problem Description Open WebUI has no extension point in the file upload pipeline. Once a user uploads a file, it goes staright to storage and extraction. There's no oppurtinity to inspect, transform, or reject first. A user-defined hook running at upload time would enable: - Security: virus and malware scanning before files are persisted - Compliance: DLP enforcement, PII redaction, content policy checks, audit logging - File transformation: format normalization, image preprocessing The possible use cases are effectively unlimited, and a user-defined hook function would let each team form the upload pipeline to their specific requirements, without waiting for every individual case to be merged upstream as a one-off feature. ### Desired Solution you'd like A new section in the admin UI (alongside the existing Functions tab) where users can register Python functions that run during file upload. These hooks would be optional. If no upload functions are configured, the current upload flow is unchanged, preserving full backward compatibility. ### Alternatives Considered _No response_ ### Additional Context If the Open WebUI team considers this a useful addition, I'd be happy to implement it and open a PR. Some initial guidance on the preferred approach would help ensure the implementation aligns with the project's direction.

GiteaMirror closed this issue

2026-05-15 16:16:20 -05:00

GiteaMirror commented

2026-05-15 16:16:20 -05:00

@Classic298 commented on GitHub (Apr 29, 2026):

You can do this via custom external document extraction server that's exactly this custom pipeline you want

@Classic298 commented on GitHub (Apr 29, 2026): You can do this via custom external document extraction server that's exactly this custom pipeline you want

GiteaMirror commented

2026-05-15 16:16:21 -05:00

@kirill-sch commented on GitHub (Apr 29, 2026):

@Classic298 I would respectfully disagree with this. The external document extraction runs after storage, not before:

Storage.upload_file
DB row inserted
extraction runs

So by the time my extraction server sees the bytes, the file is already persisted in the configured storage and rejecting at the extraction stage doesn't remove it. That defeats the use cases this issue is about. A malicious or sensitive file is already in the storage before my code can look at it.

@kirill-sch commented on GitHub (Apr 29, 2026): @Classic298 I would respectfully disagree with this. The external document extraction runs after storage, not before: - `Storage.upload_file` - DB row inserted - extraction runs So by the time my extraction server sees the bytes, the file is already persisted in the configured storage and rejecting at the extraction stage doesn't remove it. That defeats the use cases this issue is about. A malicious or sensitive file is already in the storage before my code can look at it.

GiteaMirror commented

2026-05-15 16:16:21 -05:00

@Classic298 commented on GitHub (Apr 29, 2026):

Walking through your claims one by one, because none of them survive contact with the actual code:

"Rejecting at the extraction stage doesn't remove it."

False. DELETE /files/{id} exists at backend/open_webui/routers/files.py:775 and removes the storage object, the DB row, the vector DB entries, and any KB associations. Your extraction server has full agency to call it back when it rejects. (If you don't do it, then yes it doesn't get deleted but there's NOTHING holding you back from deleting the file on rejection). The file id is recoverable from the X-Filename header today (storage filename is {id}_{name}), and exposing it as an explicit header is a one-line change if you want it cleaner. So the entire premise of "the file is already persisted and I can't do anything about it" is wrong - your extraction server can absolutely make Open WebUI disregard the file. Make a service account in Open WebUI with admin and use that api key in the extraction server. Easy.

Malware/virus scanning.

Open WebUI has three deployment tiers: solo hobbyist, family-and-friends, and enterprise. A solo user trusts their own machine — if they don't, they have bigger problems than an upload hook. Family-and-friends is the same scope. Enterprise usually has EDR/XDR or anti-malware on every endpoint and on every server, mandated by policy and usually by the cyber insurance contract. The moment malicious bytes hit the storage volume, the at-rest scanner quarantines them — independent of whether Open WebUI cooperated. A pre-upload AV hook is redundant defense-in-depth at best, and it's redundant on a layer that doesn't depend on the application running it correctly. This is not a real gap.

PII / DLP / compliance.

The leak vector that's actually specific to an AI tool is "did sensitive content reach the model, the embeddings, or the vector DB." That vector is fully closed by your extraction server returning an error — no page_content means nothing reaches save_docs_to_vector_db, nothing reaches the LLM prompt, nothing is retrievable.

If you also want the bytes gone from storage, see point one above again: call DELETE /files/{id}. The remaining concern — "the bytes briefly existed in the Open WebUI storage bucket" — is the same data-governance problem as the same file sitting on SharePoint, OneDrive, a fileshare, or the user's laptop. That's a corporate storage policy issue, not an Open WebUI feature gap. Open WebUI is not the right control point for "this file is not allowed to exist anywhere in the company". If the file is not allowed to exist anywhere in the company then why did the user have it on his laptop?

And even IF you accidentally don't catch the PII data during content extraction you can have an additional layer of protection using a Filter. And if you don't use a filter you're leaving half the lunch on the table.

Users can copy paste content without the data going through the RAG Pipeline. So you need a filter to check in the inlet() if there's any PII besides the uploaded files.

Format normalization / image preprocessing.

The only thing the downstream pipeline consumes is the extracted text returned by your extraction engine. HEIC→JPEG, DOCX→PDF, OCR preprocessing, encoding fixes — all of that belongs inside the extraction engine, before you return page_content. The original bytes sitting in storage are inert; nothing reads them after extraction.

Pre-storage normalization solves a problem that doesn't exist. why would this EVER be needed?

Audit logging.

The DB row already captures user_id, filename, size, content_type, timestamp, and (post-extraction) hash. Plus you can enable the specific existing audit logging features of Open WebUI (see the docs).

Content-level audit happens against the extraction output — your extraction server already sees every byte and can log whatever it wants, with whatever retention and SIEM integration your org requires. There is no audit signal available to a pre-storage hook that isn't already available to your extraction server plus the existing DB record.

"The possible use cases are effectively unlimited."

This is exactly the kind of framing that motivates feature requests that don't survive scrutiny. When you enumerate the actual cases, every one of them is either covered by the external extraction engine + DELETE /files/{id} callback, by infrastructure controls that exist outside the application, or by data-governance policy that isn't Open WebUI's job. "Unlimited" use cases means "I haven't named a real one yet."

Closing this stays closed. The extension point you're asking for already exists — it's the external content extraction engine, with a callback to the file deletion API for the cases where you want the bytes gone. If you want a worked example of the rejection-and-delete flow, that's a docs request, not a new hook surface.

Feel free to name any more cases. But the ones you named so far are already covered or an inherent issue of another existing governance issue in your company.

@Classic298 commented on GitHub (Apr 29, 2026): Walking through your claims one by one, because none of them survive contact with the actual code: **"Rejecting at the extraction stage doesn't remove it."** False. `DELETE /files/{id}` exists at `backend/open_webui/routers/files.py:775` and removes the storage object, the DB row, the vector DB entries, and any KB associations. Your extraction server has full agency to call it back when it rejects. (If you don't do it, then yes it doesn't get deleted but there's NOTHING holding you back from deleting the file on rejection). The file id is recoverable from the `X-Filename` header today (storage filename is `{id}_{name}`), and exposing it as an explicit header is a one-line change if you want it cleaner. So the entire premise of "the file is already persisted and I can't do anything about it" is wrong - your extraction server can absolutely make Open WebUI disregard the file. Make a service account in Open WebUI with admin and use that api key in the extraction server. Easy. **Malware/virus scanning.** Open WebUI has three deployment tiers: solo hobbyist, family-and-friends, and enterprise. A solo user trusts their own machine — if they don't, they have bigger problems than an upload hook. Family-and-friends is the same scope. Enterprise usually has EDR/XDR or anti-malware on every endpoint and on every server, mandated by policy and usually by the cyber insurance contract. The moment malicious bytes hit the storage volume, the at-rest scanner quarantines them — independent of whether Open WebUI cooperated. A pre-upload AV hook is redundant defense-in-depth at best, and it's redundant on a layer that doesn't depend on the application running it correctly. This is not a real gap. **PII / DLP / compliance.** The leak vector that's actually specific to an AI tool is "did sensitive content reach the model, the embeddings, or the vector DB." That vector is fully closed by your extraction server returning an error — no `page_content` means nothing reaches `save_docs_to_vector_db`, nothing reaches the LLM prompt, nothing is retrievable. If you also want the bytes gone from storage, see point one above again: call `DELETE /files/{id}`. The remaining concern — "the bytes briefly existed in the Open WebUI storage bucket" — is the same data-governance problem as the same file sitting on SharePoint, OneDrive, a fileshare, or the user's laptop. That's a corporate storage policy issue, not an Open WebUI feature gap. Open WebUI is not the right control point for "this file is not allowed to exist anywhere in the company". If the file is not allowed to exist anywhere in the company then why did the user have it on his laptop? And even IF you accidentally don't catch the PII data during content extraction you can have an additional layer of protection using a Filter. And if you don't use a filter you're leaving half the lunch on the table. Users can copy paste content without the data going through the RAG Pipeline. So you need a filter to check in the inlet() if there's any PII besides the uploaded files. **Format normalization / image preprocessing.** The only thing the downstream pipeline consumes is the extracted text returned by your extraction engine. HEIC→JPEG, DOCX→PDF, OCR preprocessing, encoding fixes — all of that belongs inside the extraction engine, before you return `page_content`. **The original bytes sitting in storage are inert; nothing reads them after extraction.** <ins>**Pre-storage normalization solves a problem that doesn't exist.**</ins> why would this EVER be needed? **Audit logging.** The DB row already captures user_id, filename, size, content_type, timestamp, and (post-extraction) hash. Plus you can enable the specific existing audit logging features of Open WebUI (see the docs). Content-level audit happens against the extraction output — your extraction server already sees every byte and can log whatever it wants, with whatever retention and SIEM integration your org requires. There is no audit signal available to a pre-storage hook that isn't already available to your extraction server plus the existing DB record. **"The possible use cases are effectively unlimited."** This is exactly the kind of framing that motivates feature requests that don't survive scrutiny. When you enumerate the actual cases, every one of them is either covered by the external extraction engine + `DELETE /files/{id}` callback, by infrastructure controls that exist outside the application, or by data-governance policy that isn't Open WebUI's job. "Unlimited" use cases means "I haven't named a real one yet." Closing this stays closed. The extension point you're asking for already exists — it's the external content extraction engine, with a callback to the file deletion API for the cases where you want the bytes gone. If you want a worked example of the rejection-and-delete flow, that's a docs request, not a new hook surface. Feel free to name any more cases. But the ones you named so far are already covered or an inherent issue of another existing governance issue in your company.

GiteaMirror commented

2026-05-15 16:16:21 -05:00

@Classic298 commented on GitHub (Apr 29, 2026):

name a deployment that the extraction-server + DELETE pattern actually can't serve, and this can be reconsidered

@Classic298 commented on GitHub (Apr 29, 2026): name a deployment that the extraction-server + DELETE pattern actually can't serve, and this can be reconsidered

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#90973