Aggregate responses from multiple LLMs #1958

Closed
opened 2025-11-11 14:57:23 -06:00 by GiteaMirror · 3 comments
Owner

Originally created by @odefta on GitHub (Sep 1, 2024).

Current Situation
In a chat session, we can select multiple LLMs to respond to a prompt. Each LLM provides its own response, so we end up with multiple separate answers.

Proposal
It would be great to have the option to use another LLM model to automatically aggregate these responses into a single, unified answer. This would be particularly useful for tackling complex logical problems, even though it may result in higher token usage.

Background
I’ve tested the "Mixture of Agents" function, but it seems to only work with local models, limiting its flexibility.

Originally created by @odefta on GitHub (Sep 1, 2024). **Current Situation** In a chat session, we can select multiple LLMs to respond to a prompt. Each LLM provides its own response, so we end up with multiple separate answers. **Proposal** It would be great to have the option to use another LLM model to automatically aggregate these responses into a single, unified answer. This would be particularly useful for tackling complex logical problems, even though it may result in higher token usage. **Background** I’ve tested the "Mixture of Agents" function, but it seems to only work with local models, limiting its flexibility.
Author
Owner

@cheahjs commented on GitHub (Sep 1, 2024):

Already a feature introduced in v0.3.14.

image
@cheahjs commented on GitHub (Sep 1, 2024): Already a feature introduced in v0.3.14. <img width="136" alt="image" src="https://github.com/user-attachments/assets/7d5b0d84-9a63-4c16-b93b-ff27eadddba3">
Author
Owner

@odefta commented on GitHub (Sep 1, 2024):

Thanks, awesome! I had an earlier version 0.3.12. Do you know which model will be used to display the (aggregated) final answer?

@odefta commented on GitHub (Sep 1, 2024): Thanks, awesome! I had an earlier version 0.3.12. Do you know which model will be used to display the (aggregated) final answer?
Author
Owner

@odefta commented on GitHub (Sep 1, 2024):

Can we choose which model is used for aggregation and / or which prompt is used for instructions?

@odefta commented on GitHub (Sep 1, 2024): Can we choose which model is used for aggregation and / or which prompt is used for instructions?
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#1958