Customization of rag_messages Function with Local API Endpoint for Enhanced Flexibility #557

New Issue

2025-11-11T14:25:58-06:00

GiteaMirror commented

2025-11-11 14:25:58 -06:00

Originally created by @sir3mat on GitHub (Mar 28, 2024).

Hello,

I've been exploring the rag_messages function within the open-webui repository, and I find it to be a powerful feature for generating responses based on user messages and contextual documents. Specifically, I am referring to the logic implemented in this section of the code.

However, for my specific use case, I believe there's an opportunity to enhance its flexibility and adaptability by allowing for the integration of a custom local API endpoint. This endpoint would implement the logic for querying relevant contexts (e.g., using alternative tools like Haystack or LlamaIndex) based on the user's input and return the final compiled rag template. This could enable more versatile and customized handling of queries and document retrieval strategies.

Proposed Feature:
I propose adding a configuration option to the config.py here that allows users to specify a URL to a custom local API endpoint. When this option is set, the rag_messages function would send a request to the specified endpoint with the necessary data (e.g., user query, documents) and receive the relevant contexts directly from this service, instead of using the current hardcoded querying strategies.

Benefits:
Customizability: Users can tailor the document retrieval and context generation process to their specific needs, using any technology or approach they prefer.
Versatility: This feature could make the rag_messages function applicable to a broader range of applications and use cases, as users are not limited to the built-in querying mechanisms.
Innovation: Encourages experimentation with different retrieval models and algorithms, potentially leading to new insights and improvements in the handling of conversational contexts.

I believe this enhancement could significantly increase the utility and flexibility of the rag_messages function, and I'm keen to hear your thoughts on this proposal.

Thank you for considering my feature request.

Originally created by @sir3mat on GitHub (Mar 28, 2024). Hello, I've been exploring the rag_messages function within the [open-webui](https://github.com/open-webui/open-webui) repository, and I find it to be a powerful feature for generating responses based on user messages and contextual documents. Specifically, I am referring to the logic implemented in [this section of the code](https://github.com/open-webui/open-webui/blob/ac294a74e7bc5f080970c049c656506e36f9407b/backend/apps/rag/utils.py#L103). However, for my specific use case, I believe there's an opportunity to enhance its flexibility and adaptability by allowing for the integration of a custom local API endpoint. This endpoint would implement the logic for querying relevant contexts (e.g., using alternative tools like Haystack or LlamaIndex) based on the user's input and return the final compiled rag template. This could enable more versatile and customized handling of queries and document retrieval strategies. **Proposed Feature:** I propose adding a configuration option to the config.py [here](https://github.com/open-webui/open-webui/blob/main/backend/config.py#L364) that allows users to specify a URL to a custom local API endpoint. When this option is set, the rag_messages function would send a request to the specified endpoint with the necessary data (e.g., user query, documents) and receive the relevant contexts directly from this service, instead of using the current hardcoded querying strategies. **Benefits:** Customizability: Users can tailor the document retrieval and context generation process to their specific needs, using any technology or approach they prefer. Versatility: This feature could make the rag_messages function applicable to a broader range of applications and use cases, as users are not limited to the built-in querying mechanisms. Innovation: Encourages experimentation with different retrieval models and algorithms, potentially leading to new insights and improvements in the handling of conversational contexts. I believe this enhancement could significantly increase the utility and flexibility of the rag_messages function, and I'm keen to hear your thoughts on this proposal. Thank you for considering my feature request.

GiteaMirror closed this issue

2025-11-11 14:25:59 -06:00

GiteaMirror commented

2025-11-11 14:25:59 -06:00