mirror of
https://github.com/open-webui/open-webui.git
synced 2026-03-25 04:24:30 -05:00
feat: multiple draft generations and rlhf data collection #96
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/open-webui#96
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @nivibilla on GitHub (Dec 15, 2023).
Is your feature request related to a problem? Please describe.
Recent DPO models being so good shows the impact of having good rlhf datasets.
Describe the solution you'd like
Similar to lymsys's arena battle. The ability to make multiple draft generations per prompt generations (from the same model, or different models/endpoints) and then the user would be able to click which one is better(possibly even edit the answer and fix). This would provide very rich and useful conversation trees along with rlhf data.
Describe alternatives you've considered
Lymsys ui kind of does this. With the battle mode but limited to one prompt, no follow up. I was looking for the ability to make multiple drafts every turn in the conversation.
@tjbck commented on GitHub (Dec 15, 2023):
Hi, I'm not sure if I understand what your feature request, could you elaborate a bit more? You can regenerate the responses and pick the one you like the most to continue the conversation if that's the feature you're requesting. Thanks.
@nivibilla commented on GitHub (Dec 15, 2023):
Hi, yeah sorry im not explaining properly.
In addition to be able to generate multiple drafts like you have already. I would like a way to keep record of that data. As in be able to export human preference data for prompt response based on which one is better. (I see you already have a thumbs up button but not sure where this data is recorded)
@tjbck commented on GitHub (Dec 15, 2023):
You can export all the chat logs if thats the feature you're looking for!
@nivibilla commented on GitHub (Dec 16, 2023):
Does that include if multiple generations were made for the same prompt and the thumbs up flag?
@tjbck commented on GitHub (Dec 23, 2023):
Yep! It should have everything, let me know if it doesn't, I'll reopen this issue. Thanks!
@briancleland commented on GitHub (Dec 27, 2023):
Hi @tjbck , can you point me to where the thumbs-up/thumbs-down is recorded in the exported json please?
@tjbck commented on GitHub (Dec 27, 2023):
@briancleland, I'd suggest you wait for #216 to get merged to main. I think there might've been a bug with the message rating annotation feature. Stay tuned!
@briancleland commented on GitHub (Dec 28, 2023):
Perfect, thanks!