feat: RAG Prompt for Knowledge #4614

New Issue

2025-11-11T15:58:41-06:00

GiteaMirror commented

2025-11-11 15:58:41 -06:00

Originally created by @etlweather on GitHub (Mar 31, 2025).

Check Existing Issues

I have searched the existing issues and discussions.

Problem Description

Currently, there is a system-wide prompt for RAG with Knowledge. It would be of great use to be able to have a prompt per knowledge.

Desired Solution you'd like

A system-wide prompt

That can be overridden on a Knowledge basis.
Perhaps also at the Model level

Usage example:

Generally, I like the RAG prompt. "If there is nothing in the knowledge, say so." But I have a knowledge which isn't the absolute. And even if there is no knowledge, it does not need to say that it does not have the answer.

Alternatives Considered

No response

Additional Context

No response

Originally created by @etlweather on GitHub (Mar 31, 2025). ### Check Existing Issues - [x] I have searched the existing issues and discussions. ### Problem Description Currently, there is a system-wide prompt for RAG with Knowledge. It would be of great use to be able to have a prompt per knowledge. ### Desired Solution you'd like A system-wide prompt - That can be overridden on a Knowledge basis. - Perhaps also at the Model level Usage example: Generally, I like the RAG prompt. "If there is nothing in the knowledge, say so." But I have a knowledge which isn't the absolute. And even if there is no knowledge, it does not need to say that it does not have the answer. ### Alternatives Considered _No response_ ### Additional Context _No response_

GiteaMirror commented

2025-11-11 15:58:41 -06:00

@mahenning commented on GitHub (Mar 31, 2025):

Do you mean the "RAG_TEMPLATE" (link), the one you set in the admin document settings? If yes, I'm not sure it is that easy to set it per knowledge base. What if the model has multiple knowledge bases? As the template is used in the last step to generate an answer in the chat, it would be very convoluted (if not impossible) to use different templates for different knowledge bases and merge them into one coherent answer.

A per-model knowledge base RAG template would be possible in my opinion, like the model system prompt.

@mahenning commented on GitHub (Mar 31, 2025): Do you mean the "RAG_TEMPLATE" ([link](https://docs.openwebui.com/getting-started/env-configuration#rag_template)), the one you set in the admin document settings? If yes, I'm not sure it is that easy to set it per knowledge base. What if the model has multiple knowledge bases? As the template is used in the last step to generate an answer in the chat, it would be very convoluted (if not impossible) to use different templates for different knowledge bases and merge them into one coherent answer. A per-model knowledge base RAG template would be possible in my opinion, like the model system prompt.

GiteaMirror commented

2025-11-11 15:58:42 -06:00

@etlweather commented on GitHub (Mar 31, 2025):

Yes, I mean RAG_TEMPLATE and I agree, as part of the Model make a lot more sense than in the Knowledge for the reasons you said.

@etlweather commented on GitHub (Mar 31, 2025): Yes, I mean RAG_TEMPLATE and I agree, as part of the Model make a lot more sense than in the Knowledge for the reasons you said.

GiteaMirror commented

2025-11-11 15:58:42 -06:00