[PR #22681] [CLOSED] fix: bypass RAG and directly inject full context files to eliminate message latency #49860

New Issue

GiteaMirror · 2026-04-30T02:15:06-05:00

GiteaMirror commented

2026-04-30 02:15:06 -05:00

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/22681
Author: @a86582751
Created: 3/14/2026
Status: ❌ Closed

Base: dev ← Head: fix-full-context-bypass-rag

📝 Commits (1)

149f186 fix: bypass RAG and directly inject full context files to eliminate message latency

📊 Changes

1 file changed (+97 additions, -34 deletions)

View changed files

📝 backend/open_webui/utils/middleware.py (+97 -34)

📄 Description

Pull Request Checklist

Before submitting, make sure you've checked the following:

Target branch: Verify that the pull request targets the dev branch. PRs targeting main will be immediately closed.
Description: Provide a concise description of the changes made in this pull request down below.
Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
Documentation: Add docs in Open WebUI Docs Repository. Document user-facing behavior, environment variables, public APIs/interfaces, or deployment steps.
Dependencies: Are there any new or upgraded dependencies? If so, explain why, update the changelog/docs, and include any compatibility notes.
Testing: Perform manual tests to verify the implemented fix/feature works as intended AND does not break any other functionality.
Agentic AI Code: Confirm this Pull Request is not written by any AI Agent or has at least gone through additional human review AND manual testing.
Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
Design & Architecture: Prefer smart defaults over adding new settings; use local state for ephemeral UI logic.
Git Hygiene: Keep PRs atomic (one logical change). Clean up commits and rebase on dev to ensure no unrelated commits (e.g. from main) are included.

Changelog Entry

Description

Fixed significant delay (up to 40 seconds) when sending messages in "Full Context" mode by directly loading file contents from the database and injecting them into the user message, fully bypassing the RAG pipeline and frontend serialization of large files.

Fixed

Fixed the issue where get_sources_from_items was still being called even when all_full_context or bypass_embedding was true.
Fixed severe frontend serialization latency caused by passing full file contents (especially large 200KB+ documents) in the JSON request body.
Fixed confusing UI state where "sources_retrieved" was emitted even when retrieval was skipped.

Additional Information

This fix drastically improves performance for Role-Play scenarios and large document contexts.
Performance test (264KB DOCX): Before this fix, frontend serialization took ~36s and RAG took ~2-5s (Total latency ~40s). After this fix, backend loading takes <100ms and injection takes <10ms (Total latency <200ms). This is an improvement of nearly 200x.
Supports all file types handled by Open WebUI Loaders (DOCX, PDF, TXT/MD, Images). Normal RAG retrieval and mixed-mode (partial full + partial retrieval) remain fully functional.
Changes made in backend/open_webui/utils/middleware.py: Added checks for all_full_context and bypass_embedding to directly load content and append it as [Attached Files Context] before returning, skipping generation and embedding stages entirely.

Screenshots or Videos

before:
Use Focused Retrieval

use Full Contextm，But the response is slow.:

after:
The response speed for chatting with large files has significantly improved.

The RAG functionality remains unaffected.：

Thank you for the welcome! I completely understand the community's concerns regarding untested PRs. I can explicitly confirm that I have personally, manually, and exhaustively tested this fix.

Here is the detailed context of my troubleshooting journey and exactly how I verified the solution:

🔍 The Background & Troubleshooting Process

I was trying to migrate my role-play chat history from another AI client to Open WebUI. I uploaded a character card file (校园恋爱角色卡定制.md, containing about 91,480 characters). The model response became extremely slow.

I toggled the "Full Context" mode for the file and also enabled "Bypass Embedding and Retrieval" in Admin Panel -> Settings -> Documents.
Despite this, the UI still showed the "Retrieving..." and "Found 1 source" steps. Once the conversation exceeded 3 turns, the chat completely froze, likely due to the massive context combined with the RAG overhead.

To isolate the issue:

I tried pasting the chat history directly into the chatbox. Open WebUI automatically converted it into a TXT file, and the exact same "Retrieving" delay occurred.
I then went to settings and turned off the "Paste large text as file" option. When I pasted the raw text directly into the chatbox again, the "Retrieving" process did not trigger, and the conversation was perfectly smooth.
This definitively proved that the bottleneck was not my backend LLM API, but rather Open WebUI's file retrieval (RAG) pipeline.

According to the official docs, enabling "File Context" in the model settings triggers RAG. If disabled, the model can't read files at all. The "Full Context" toggle seemed to just run the entire RAG process anyway before dumping all the content, failing to actually bypass the heavy RAG workload. This severely impacted conversation performance.

🛠️ The Investigation & AI Transparency

My intuition told me there was a flaw in the file processing logic. To be fully transparent, I used AI to help me navigate the codebase and investigate my hypothesis. Sure enough, we found the logical gap: the RAG pipeline was never truly bypassed even when full context was requested.

✅ How I Tested It (Verification)

While AI helped find the logic flaw, the testing was 100% manual and rigorous:

I deployed a test container equipped with this modified code on my personal Alibaba Cloud server.
I repeated the exact scenario: uploaded the 91k-character RP document, toggled "Full Context", and chatted for multiple turns.
The "Retrieving..." step is now completely bypassed. The file content is directly injected, and the chat is blazing fast.
I also tested normal RAG files to ensure nothing was broken for standard use cases.

Visual evidence (Before/After comparison screenshots) is already attached in my original post at the top of this PR.

This PR fixes a genuine performance bottleneck for RP and large-document users. It is fully tested on a real server and ready for your review! 🙏

Contributor License Agreement

By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.

Note

Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/22681 **Author:** [@a86582751](https://github.com/a86582751) **Created:** 3/14/2026 **Status:** ❌ Closed **Base:** `dev` ← **Head:** `fix-full-context-bypass-rag` --- ### 📝 Commits (1) - [`149f186`](https://github.com/open-webui/open-webui/commit/149f1868ce346167b724adc01bb5f95b85ef4245) fix: bypass RAG and directly inject full context files to eliminate message latency ### 📊 Changes **1 file changed** (+97 additions, -34 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/utils/middleware.py` (+97 -34) </details> ### 📄 Description  # Pull Request Checklist **Before submitting, make sure you've checked the following:** - [x] **Target branch:** Verify that the pull request targets the `dev` branch. **PRs targeting `main` will be immediately closed.** - [x] **Description:** Provide a concise description of the changes made in this pull request down below. - [x] **Changelog:** Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description. - [ ] **Documentation:** Add docs in Open WebUI Docs Repository. Document user-facing behavior, environment variables, public APIs/interfaces, or deployment steps. - [ ] **Dependencies:** Are there any new or upgraded dependencies? If so, explain why, update the changelog/docs, and include any compatibility notes. - [x] **Testing:** Perform manual tests to **verify the implemented fix/feature works as intended AND does not break any other functionality**. - [x] **Agentic AI Code:** Confirm this Pull Request is **not written by any AI Agent** or has at least **gone through additional human review AND manual testing**. - [x] **Code review:** Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards? - [x] **Design & Architecture:** Prefer smart defaults over adding new settings; use local state for ephemeral UI logic. - [x] **Git Hygiene:** Keep PRs atomic (one logical change). Clean up commits and rebase on `dev` to ensure no unrelated commits (e.g. from `main`) are included. # Changelog Entry ### Description - Fixed significant delay (up to 40 seconds) when sending messages in "Full Context" mode by directly loading file contents from the database and injecting them into the user message, fully bypassing the RAG pipeline and frontend serialization of large files. ### Fixed - Fixed the issue where `get_sources_from_items` was still being called even when `all_full_context` or `bypass_embedding` was true. - Fixed severe frontend serialization latency caused by passing full file contents (especially large 200KB+ documents) in the JSON request body. - Fixed confusing UI state where "sources_retrieved" was emitted even when retrieval was skipped. --- ### Additional Information - This fix drastically improves performance for Role-Play scenarios and large document contexts. - **Performance test (264KB DOCX):** Before this fix, frontend serialization took ~36s and RAG took ~2-5s (Total latency ~40s). After this fix, backend loading takes <100ms and injection takes <10ms (Total latency <200ms). This is an improvement of nearly 200x. - Supports all file types handled by Open WebUI Loaders (DOCX, PDF, TXT/MD, Images). Normal RAG retrieval and mixed-mode (partial full + partial retrieval) remain fully functional. - Changes made in `backend/open_webui/utils/middleware.py`: Added checks for `all_full_context` and `bypass_embedding` to directly load content and append it as `[Attached Files Context]` before returning, skipping generation and embedding stages entirely. ### Screenshots or Videos - before: Use Focused Retrieval <img width="2630" height="1585" alt="默认设置1" src="https://github.com/user-attachments/assets/bc8d7b98-c975-446c-ae83-9923268a4339" /> <img width="2481" height="1324" alt="默认设置2" src="https://github.com/user-attachments/assets/2b200d93-2b5a-44aa-8826-f8a4e6389584" /> use Full Contextm，But the response is slow.: <img width="2569" height="1287" alt="image" src="https://github.com/user-attachments/assets/90f4a2b0-9081-42fe-9659-9403812f9a0e" /> - after: The response speed for chatting with large files has significantly improved. <img width="2762" height="1533" alt="image" src="https://github.com/user-attachments/assets/cd56c61e-ab3f-4e24-aecd-e9d73343718d" /> <img width="2640" height="1261" alt="image" src="https://github.com/user-attachments/assets/ee4aaebb-c999-402c-82dc-6349f93cff08" /> The RAG functionality remains unaffected.： <img width="2994" height="1703" alt="image" src="https://github.com/user-attachments/assets/a148e6b7-ef30-43e2-aeb4-018b4b612631" /> Thank you for the welcome! I completely understand the community's concerns regarding untested PRs. I can explicitly confirm that I have personally, manually, and exhaustively tested this fix. Here is the detailed context of my troubleshooting journey and exactly how I verified the solution: ### 🔍 The Background & Troubleshooting Process I was trying to migrate my role-play chat history from another AI client to Open WebUI. I uploaded a character card file (`校园恋爱角色卡定制.md`, containing about 91,480 characters). The model response became extremely slow. - I toggled the **"Full Context"** mode for the file and also enabled **"Bypass Embedding and Retrieval"** in `Admin Panel -> Settings -> Documents`. - Despite this, the UI still showed the **"Retrieving..."** and **"Found 1 source"** steps. Once the conversation exceeded 3 turns, the chat completely froze, likely due to the massive context combined with the RAG overhead. **To isolate the issue:** 1. I tried pasting the chat history directly into the chatbox. Open WebUI automatically converted it into a TXT file, and the exact same "Retrieving" delay occurred. 2. I then went to settings and **turned off the "Paste large text as file"** option. When I pasted the raw text directly into the chatbox again, the "Retrieving" process did not trigger, and the conversation was perfectly smooth. *This definitively proved that the bottleneck was not my backend LLM API, but rather Open WebUI's file retrieval (RAG) pipeline.* According to the official docs, enabling "File Context" in the model settings triggers RAG. If disabled, the model can't read files at all. The "Full Context" toggle seemed to just run the entire RAG process anyway before dumping all the content, failing to actually bypass the heavy RAG workload. This severely impacted conversation performance. ### 🛠️ The Investigation & AI Transparency My intuition told me there was a flaw in the file processing logic. **To be fully transparent, I used AI to help me navigate the codebase and investigate my hypothesis.** Sure enough, we found the logical gap: the RAG pipeline was never truly bypassed even when full context was requested. ### ✅ How I Tested It (Verification) While AI helped find the logic flaw, **the testing was 100% manual and rigorous**: 1. I deployed a test container equipped with this modified code on my personal Alibaba Cloud server. 2. I repeated the exact scenario: uploaded the 91k-character RP document, toggled "Full Context", and chatted for multiple turns. 3. The "Retrieving..." step is now completely bypassed. The file content is directly injected, and the chat is blazing fast. 4. I also tested normal RAG files to ensure nothing was broken for standard use cases. **Visual evidence (Before/After comparison screenshots) is already attached in my original post at the top of this PR.** This PR fixes a genuine performance bottleneck for RP and large-document users. It is fully tested on a real server and ready for your review! 🙏 ### Contributor License Agreement  - [x] By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms. > [!NOTE] > Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-04-30 02:15:06 -05:00

GiteaMirror closed this issue

2026-04-30 02:15:08 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#49860