github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-03-22 14:13:08 -05:00
Code Issues 308 Packages Projects Releases 100 Wiki Activity

[PR #14730] feat: Adding individual RAG config settings #10365

New Issue
Open
opened 2025-11-11 19:03:03 -06:00 by GiteaMirror · 0 comments
GiteaMirror commented 2025-11-11 19:03:03 -06:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/14730
Author: @Maytown
Created: 6/6/2025
Status: 🔄 Open

Base: dev ← Head: Dev-Individual-RAG-Config

📝 Commits (10+)

  • b46da1a Added DEFAULT_RAG_SETTINGS option for individual rag configuration
  • 45ee9c3 Function: Added get knowledge by collection name and adjusted insert new knowledge to insert rag config to DB
  • 220ad37 Function: Adjusted create new knowledge to send rag config
  • e3a93b2 Feature: Adjusted all necessary functions to handle individual rag configuration - fallback to default configuration if individual config is not used
  • 4f93577 Feature: Adjusted process_chat_payload to handle individual rag config (send individual rag template)
  • 6e2155d Feature: adjusted createNewKnowledge to pass individual rag config
  • 4f1c9a8 Feature: adjusted retrieval restpoints to handle indivudal rag config - passing rag config and changing from get to post
  • 79ff995 Feature: Added Rag Configuration to knowledge creation - if disabled falls back to default/general rag settings from documentsettings
  • 9233f1f Merge remote-tracking branch 'upstream/dev' into Individual-RAG-Config
  • 49e4375 FIX: adjusted to send knowledge base id for saving docs - fixed update_reranking_config to handle knowledge config by collection name

📊 Changes

19 files changed (+4936 additions, -1033 deletions)

View changed files

📝 backend/open_webui/config.py (+38 -0)
➕ backend/open_webui/internal/migrations/019_add_rag_config_to_knowledge.py (+84 -0)
📝 backend/open_webui/main.py (+77 -58)
📝 backend/open_webui/migrations/versions/6a39f3d8e55c_add_knowledge_table.py (+1 -0)
📝 backend/open_webui/models/knowledge.py (+47 -0)
📝 backend/open_webui/retrieval/utils.py (+68 -9)
📝 backend/open_webui/routers/files.py (+22 -7)
📝 backend/open_webui/routers/knowledge.py (+63 -0)
📝 backend/open_webui/routers/memories.py (+5 -5)
📝 backend/open_webui/routers/retrieval.py (+1269 -807)
➕ backend/open_webui/test/apps/webui/routers/test_individual_rag_config.py (+338 -0)
📝 backend/open_webui/utils/middleware.py (+33 -21)
📝 src/lib/apis/files/index.ts (+4 -1)
📝 src/lib/apis/knowledge/index.ts (+35 -3)
📝 src/lib/apis/retrieval/index.ts (+14 -6)
📝 src/lib/components/common/Modal.svelte (+10 -0)
📝 src/lib/components/workspace/Knowledge/CreateKnowledgeBase.svelte (+1389 -107)
📝 src/lib/components/workspace/Knowledge/KnowledgeBase.svelte (+79 -9)
➕ src/lib/components/workspace/common/RagConfigModal.svelte (+1360 -0)

📄 Description

  • # Pull Request Checklist

Before submitting, make sure you've checked the following:

  • Target branch: Please verify that the pull request targets the dev branch.
  • Description: Provide a concise description of the changes made in this pull request.
  • Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
  • Documentation: Have you updated relevant documentation Open WebUI Docs, or other documentation sources?
  • Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
  • Testing: Have you written and run sufficient tests to validate the changes?
  • Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
  • Prefix: To clearly categorize this pull request, prefix the pull request title using one of the following:
    • BREAKING CHANGE: Significant changes that may affect compatibility
    • build: Changes that affect the build system or external dependencies
    • ci: Changes to our continuous integration processes or workflows
    • chore: Refactor, cleanup, or other non-functional code changes
    • docs: Documentation update or addition
    • feat: Introduces a new feature or enhancement to the codebase
    • fix: Bug fix or error correction
    • i18n: Internationalization or localization changes
    • perf: Performance improvement
    • refactor: Code restructuring for better maintainability, readability, or scalability
    • style: Changes that do not affect the meaning of the code (white space, formatting, missing semi-colons, etc.)
    • test: Adding missing tests or correcting existing tests
    • WIP: Work in progress, a temporary label for incomplete or ongoing work

Changelog Entry

Description

  • Moving the RAG settings from the general panel to an optional indivudal settings panel when creating the knowledge base - except Danger Zone functions
  • Backend distinguishes between the individual RAG config and the default (general -> documents)
    -Admin users can download new models whereas non-admin users can select embedder/reranker from dropdown; dropdown shows already downloaded models

Added

  • RAG config settings in the create Knowledgebase setting with a switch to cahnge between individual and default rag config
  • RAG settings panel in the according Knowledgebase page to change settings later (only shows up if individual rag config is enabled)

Changed

  • Main changes are in retrieval.py to distinguish between default/individual; get/update rag/embedding config looks up if default rag config is used or not
  • Query doc handlers and chat completion file handlers changed to use the according settings
  • File upload minimal changes to not change the logic
  • Changed how rerankers and embedders are loaded in main.py - loads now from all currently loaded models (saved in DB)

Deprecated

  • [List any deprecated functionality or features that have been removed]

Removed

  • [List any removed features, files, or functionalities]

Fixed

  • [List any fixes, corrections, or bug fixes]

Security

  • [List any new or updated security-related changes, including vulnerability fixes]

Breaking Changes

  • BREAKING CHANGE: [List any breaking changes affecting compatibility or functionality]

Additional Information

  • [Insert any additional context, notes, or explanations for the changes]
    • [Reference any related issues, commits, or other relevant information]

Screenshots or Videos

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/14730 **Author:** [@Maytown](https://github.com/Maytown) **Created:** 6/6/2025 **Status:** 🔄 Open **Base:** `dev` ← **Head:** `Dev-Individual-RAG-Config` --- ### 📝 Commits (10+) - [`b46da1a`](https://github.com/open-webui/open-webui/commit/b46da1ae868c932fd60c3c9d15b743af87e39ae3) Added DEFAULT_RAG_SETTINGS option for individual rag configuration - [`45ee9c3`](https://github.com/open-webui/open-webui/commit/45ee9c33ade1a37f9d7283cb28a98fdcf5054900) Function: Added get knowledge by collection name and adjusted insert new knowledge to insert rag config to DB - [`220ad37`](https://github.com/open-webui/open-webui/commit/220ad3723eecd15ed5ce392be6b87f42b9071cb4) Function: Adjusted create new knowledge to send rag config - [`e3a93b2`](https://github.com/open-webui/open-webui/commit/e3a93b24a01d3084f162af5edd3c43f8c5927be3) Feature: Adjusted all necessary functions to handle individual rag configuration - fallback to default configuration if individual config is not used - [`4f93577`](https://github.com/open-webui/open-webui/commit/4f93577d3dd21559ba828a9fb334ebfd501c5640) Feature: Adjusted process_chat_payload to handle individual rag config (send individual rag template) - [`6e2155d`](https://github.com/open-webui/open-webui/commit/6e2155dd8c9fd8a1bd2f255aabc4c2bcb731f1a6) Feature: adjusted createNewKnowledge to pass individual rag config - [`4f1c9a8`](https://github.com/open-webui/open-webui/commit/4f1c9a8c4f6de0e1e40a03eab6383d75e4c6c30d) Feature: adjusted retrieval restpoints to handle indivudal rag config - passing rag config and changing from get to post - [`79ff995`](https://github.com/open-webui/open-webui/commit/79ff99550c16b18ea76dd6402edbb465402646f9) Feature: Added Rag Configuration to knowledge creation - if disabled falls back to default/general rag settings from documentsettings - [`9233f1f`](https://github.com/open-webui/open-webui/commit/9233f1f848b5d04b60dac465fd99284b3a2a1fea) Merge remote-tracking branch 'upstream/dev' into Individual-RAG-Config - [`49e4375`](https://github.com/open-webui/open-webui/commit/49e43752637a436977f1d69e9ab6006ee8bba407) FIX: adjusted to send knowledge base id for saving docs - fixed update_reranking_config to handle knowledge config by collection name ### 📊 Changes **19 files changed** (+4936 additions, -1033 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/config.py` (+38 -0) ➕ `backend/open_webui/internal/migrations/019_add_rag_config_to_knowledge.py` (+84 -0) 📝 `backend/open_webui/main.py` (+77 -58) 📝 `backend/open_webui/migrations/versions/6a39f3d8e55c_add_knowledge_table.py` (+1 -0) 📝 `backend/open_webui/models/knowledge.py` (+47 -0) 📝 `backend/open_webui/retrieval/utils.py` (+68 -9) 📝 `backend/open_webui/routers/files.py` (+22 -7) 📝 `backend/open_webui/routers/knowledge.py` (+63 -0) 📝 `backend/open_webui/routers/memories.py` (+5 -5) 📝 `backend/open_webui/routers/retrieval.py` (+1269 -807) ➕ `backend/open_webui/test/apps/webui/routers/test_individual_rag_config.py` (+338 -0) 📝 `backend/open_webui/utils/middleware.py` (+33 -21) 📝 `src/lib/apis/files/index.ts` (+4 -1) 📝 `src/lib/apis/knowledge/index.ts` (+35 -3) 📝 `src/lib/apis/retrieval/index.ts` (+14 -6) 📝 `src/lib/components/common/Modal.svelte` (+10 -0) 📝 `src/lib/components/workspace/Knowledge/CreateKnowledgeBase.svelte` (+1389 -107) 📝 `src/lib/components/workspace/Knowledge/KnowledgeBase.svelte` (+79 -9) ➕ `src/lib/components/workspace/common/RagConfigModal.svelte` (+1360 -0) </details> ### 📄 Description - [x] # Pull Request Checklist **Before submitting, make sure you've checked the following:** - [x] **Target branch:** Please verify that the pull request targets the `dev` branch. - [x] **Description:** Provide a concise description of the changes made in this pull request. - [x] **Changelog:** Ensure a changelog entry following the format of [Keep a Changelog](https://keepachangelog.com/) is added at the bottom of the PR description. - [ ] **Documentation:** Have you updated relevant documentation [Open WebUI Docs](https://github.com/open-webui/docs), or other documentation sources? - [ ] **Dependencies:** Are there any new dependencies? Have you updated the dependency versions in the documentation? - [x] **Testing:** Have you written and run sufficient tests to validate the changes? - [x] **Code review:** Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards? - [x] **Prefix:** To clearly categorize this pull request, prefix the pull request title using one of the following: - **BREAKING CHANGE**: Significant changes that may affect compatibility - **build**: Changes that affect the build system or external dependencies - **ci**: Changes to our continuous integration processes or workflows - **chore**: Refactor, cleanup, or other non-functional code changes - **docs**: Documentation update or addition - **feat**: Introduces a new feature or enhancement to the codebase - **fix**: Bug fix or error correction - **i18n**: Internationalization or localization changes - **perf**: Performance improvement - **refactor**: Code restructuring for better maintainability, readability, or scalability - **style**: Changes that do not affect the meaning of the code (white space, formatting, missing semi-colons, etc.) - **test**: Adding missing tests or correcting existing tests - **WIP**: Work in progress, a temporary label for incomplete or ongoing work # Changelog Entry ### Description - Moving the RAG settings from the general panel to an optional indivudal settings panel when creating the knowledge base - except Danger Zone functions - Backend distinguishes between the individual RAG config and the default (general -> documents) -Admin users can download new models whereas non-admin users can select embedder/reranker from dropdown; dropdown shows already downloaded models ### Added - RAG config settings in the create Knowledgebase setting with a switch to cahnge between individual and default rag config - RAG settings panel in the according Knowledgebase page to change settings later (only shows up if individual rag config is enabled) ### Changed - Main changes are in retrieval.py to distinguish between default/individual; get/update rag/embedding config looks up if default rag config is used or not - Query doc handlers and chat completion file handlers changed to use the according settings - File upload minimal changes to not change the logic - Changed how rerankers and embedders are loaded in main.py - loads now from all currently loaded models (saved in DB) ### Deprecated - [List any deprecated functionality or features that have been removed] ### Removed - [List any removed features, files, or functionalities] ### Fixed - [List any fixes, corrections, or bug fixes] ### Security - [List any new or updated security-related changes, including vulnerability fixes] ### Breaking Changes - **BREAKING CHANGE**: [List any breaking changes affecting compatibility or functionality] --- ### Additional Information - [Insert any additional context, notes, or explanations for the changes] - [Reference any related issues, commits, or other relevant information] ### Screenshots or Videos --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2025-11-11 19:03:03 -06:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#10365
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?