feat: hybrid RAG context placement for KV cache optimization
PR Description
Summary
Implements hybrid placement of RAG context to optimize KV prefix caching when RAG_SYSTEM_CONTEXT is enabled.
Problem
When RAG_SYSTEM_CONTEXT=True, ALL RAG content is placed in the system message. This breaks KV caching for chunked retrieval because per-query chunks change each turn, invalidating the cached prefix.
Solution
Separate sources by stability:
Source Type
Placement
Rationale
Full documents (context="full")
System message
Stable → cached
Global BYPASS_EMBEDDING_AND_RETRIEVAL
System message
All files treated as full
Global RAG_FULL_CONTEXT
System message
All files treated as full
Per-query chunks
User message
Changes each turn
Tool results
User message
Turn-specific
Changes
Modified apply_source_context_to_messages in utils/middleware.py
Added is_static_source helper to classify sources based on file metadata and global config
Static sources → system message (stable prefix for KV caching)
Dynamic sources → user message (changes per turn)
Backward Compatibility
When RAG_SYSTEM_CONTEXT=False: unchanged behavior (all sources → user message)
Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
Note
Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/20648
**Author:** [@Classic298](https://github.com/Classic298)
**Created:** 1/13/2026
**Status:** ❌ Closed
**Base:** `dev` ← **Head:** `patch-2`
---
### 📝 Commits (3)
- [`240c613`](https://github.com/open-webui/open-webui/commit/240c6131fea776db2f82ea56e7660575c78a4593) Update middleware.py
- [`14d0556`](https://github.com/open-webui/open-webui/commit/14d0556729fbc782e75ec73fc6035517792ea3e6) Update middleware.py
- [`c6b3f2f`](https://github.com/open-webui/open-webui/commit/c6b3f2fd13628423658b991dcb3f8daeee7f32fb) Update middleware.py
### 📊 Changes
**1 file changed** (+67 additions, -25 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/open_webui/utils/middleware.py` (+67 -25)
</details>
### 📄 Description
# feat: hybrid RAG context placement for KV cache optimization
## PR Description
### Summary
Implements hybrid placement of RAG context to optimize KV prefix caching when RAG_SYSTEM_CONTEXT is enabled.
### Problem
When RAG_SYSTEM_CONTEXT=True, ALL RAG content is placed in the system message. This breaks KV caching for chunked retrieval because per-query chunks change each turn, invalidating the cached prefix.
### Solution
Separate sources by stability:
| Source Type | Placement | Rationale |
|------------|-----------|-----------|
| Full documents (context="full") | System message | Stable → cached |
| Global BYPASS_EMBEDDING_AND_RETRIEVAL | System message | All files treated as full |
| Global RAG_FULL_CONTEXT | System message | All files treated as full |
| Per-query chunks | User message | Changes each turn |
| Tool results | User message | Turn-specific |
### Changes
- Modified apply_source_context_to_messages in utils/middleware.py
- Added is_static_source helper to classify sources based on file metadata and global config
- Static sources → system message (stable prefix for KV caching)
- Dynamic sources → user message (changes per turn)
### Backward Compatibility
When RAG_SYSTEM_CONTEXT=False: unchanged behavior (all sources → user message)
### Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms.
> [!NOTE]
> Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/20648
Author: @Classic298
Created: 1/13/2026
Status: ❌ Closed
Base:
dev← Head:patch-2📝 Commits (3)
240c613Update middleware.py14d0556Update middleware.pyc6b3f2fUpdate middleware.py📊 Changes
1 file changed (+67 additions, -25 deletions)
View changed files
📝
backend/open_webui/utils/middleware.py(+67 -25)📄 Description
feat: hybrid RAG context placement for KV cache optimization
PR Description
Summary
Implements hybrid placement of RAG context to optimize KV prefix caching when RAG_SYSTEM_CONTEXT is enabled.
Problem
When RAG_SYSTEM_CONTEXT=True, ALL RAG content is placed in the system message. This breaks KV caching for chunked retrieval because per-query chunks change each turn, invalidating the cached prefix.
Solution
Separate sources by stability:
Changes
Backward Compatibility
When RAG_SYSTEM_CONTEXT=False: unchanged behavior (all sources → user message)
Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.