github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-07-16 14:39:31 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[PR #20648] [CLOSED] feat: hybrid RAG context placement for KV cache optimization #80591

New Issue
Closed
opened 2026-05-13 14:43:41 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-13 14:43:41 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/20648
Author: @Classic298
Created: 1/13/2026
Status: ❌ Closed

Base: dev ← Head: patch-2

📝 Commits (3)

📊 Changes

1 file changed (+67 additions, -25 deletions)

View changed files

📝 backend/open_webui/utils/middleware.py (+67 -25)

📄 Description

feat: hybrid RAG context placement for KV cache optimization

PR Description

Summary

Implements hybrid placement of RAG context to optimize KV prefix caching when RAG_SYSTEM_CONTEXT is enabled.

Problem

When RAG_SYSTEM_CONTEXT=True, ALL RAG content is placed in the system message. This breaks KV caching for chunked retrieval because per-query chunks change each turn, invalidating the cached prefix.

Solution

Separate sources by stability:

Source Type Placement Rationale
Full documents (context="full") System message Stable → cached
Global BYPASS_EMBEDDING_AND_RETRIEVAL System message All files treated as full
Global RAG_FULL_CONTEXT System message All files treated as full
Per-query chunks User message Changes each turn
Tool results User message Turn-specific

Changes

  • Modified apply_source_context_to_messages in utils/middleware.py
  • Added is_static_source helper to classify sources based on file metadata and global config
  • Static sources → system message (stable prefix for KV caching)
  • Dynamic sources → user message (changes per turn)

Backward Compatibility

When RAG_SYSTEM_CONTEXT=False: unchanged behavior (all sources → user message)

Contributor License Agreement

By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.

Note

Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/20648 **Author:** [@Classic298](https://github.com/Classic298) **Created:** 1/13/2026 **Status:** ❌ Closed **Base:** `dev` ← **Head:** `patch-2` --- ### 📝 Commits (3) - [`240c613`](https://github.com/open-webui/open-webui/commit/240c6131fea776db2f82ea56e7660575c78a4593) Update middleware.py - [`14d0556`](https://github.com/open-webui/open-webui/commit/14d0556729fbc782e75ec73fc6035517792ea3e6) Update middleware.py - [`c6b3f2f`](https://github.com/open-webui/open-webui/commit/c6b3f2fd13628423658b991dcb3f8daeee7f32fb) Update middleware.py ### 📊 Changes **1 file changed** (+67 additions, -25 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/utils/middleware.py` (+67 -25) </details> ### 📄 Description # feat: hybrid RAG context placement for KV cache optimization ## PR Description ### Summary Implements hybrid placement of RAG context to optimize KV prefix caching when RAG_SYSTEM_CONTEXT is enabled. ### Problem When RAG_SYSTEM_CONTEXT=True, ALL RAG content is placed in the system message. This breaks KV caching for chunked retrieval because per-query chunks change each turn, invalidating the cached prefix. ### Solution Separate sources by stability: | Source Type | Placement | Rationale | |------------|-----------|-----------| | Full documents (context="full") | System message | Stable → cached | | Global BYPASS_EMBEDDING_AND_RETRIEVAL | System message | All files treated as full | | Global RAG_FULL_CONTEXT | System message | All files treated as full | | Per-query chunks | User message | Changes each turn | | Tool results | User message | Turn-specific | ### Changes - Modified apply_source_context_to_messages in utils/middleware.py - Added is_static_source helper to classify sources based on file metadata and global config - Static sources → system message (stable prefix for KV caching) - Dynamic sources → user message (changes per turn) ### Backward Compatibility When RAG_SYSTEM_CONTEXT=False: unchanged behavior (all sources → user message) ### Contributor License Agreement By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms. > [!NOTE] > Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-05-13 14:43:41 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#80591
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?