feat: a mode for the UI for rapidly tweaking prompts #647

Closed
opened 2025-11-11 14:28:12 -06:00 by GiteaMirror · 3 comments
Owner

Originally created by @jeremiahsb on GitHub (Apr 13, 2024).

Is your feature request related to a problem? Please describe.
Related: #619 & #1391
Not a problem per say, just a way to turn open webui into a tool for prompt tweaking.

Describe the solution you'd like
It would be great to have the ability to have a screen which the user could:

  • test multiple variations of system prompts, holding other attributes constant
  • test multiple models holding other attributes constant
  • test multiple temperatures holding other attributes constant

Once the user hits a "test" button, it would execute the configurations and display the response and highlight differences.

Describe alternatives you've considered
Having something like promptfoo, but integrated directly in the open webui.
https://github.com/promptfoo/promptfoo

Originally created by @jeremiahsb on GitHub (Apr 13, 2024). **Is your feature request related to a problem? Please describe.** Related: #619 & #1391 Not a problem per say, just a way to turn open webui into a tool for prompt tweaking. **Describe the solution you'd like** It would be great to have the ability to have a screen which the user could: - test multiple variations of system prompts, holding other attributes constant - test multiple models holding other attributes constant - test multiple temperatures holding other attributes constant Once the user hits a "test" button, it would execute the configurations and display the response and highlight differences. **Describe alternatives you've considered** Having something like promptfoo, but integrated directly in the open webui. https://github.com/promptfoo/promptfoo
Author
Owner

@justinh-rahb commented on GitHub (Apr 14, 2024):

This is sorta the intention behind our "Playground" feature, though admittedly it's a first-draft that really needs some more work, including a quick settings sidebar for parameters, and saved profiles, similar to what you see in other platforms' "Playground" UIs now. Extending this even further with some of the ideas you suggest sounds like a useful feature to me as well, and one I'd fully support a PR for.

@justinh-rahb commented on GitHub (Apr 14, 2024): This is sorta the intention behind our "Playground" feature, though admittedly it's a first-draft that really needs some more work, including a quick settings sidebar for parameters, and saved profiles, similar to what you see in other platforms' "Playground" UIs now. Extending this even further with some of the ideas you suggest sounds like a useful feature to me as well, and one I'd fully support a PR for.
Author
Owner

@tjbck commented on GitHub (Apr 14, 2024):

This would be a great addition to our playground feature, feel free to make a PR! Once I clear out our backlogs I'll also actively start looking into this.

@tjbck commented on GitHub (Apr 14, 2024): This would be a great addition to our playground feature, feel free to make a PR! Once I clear out our backlogs I'll also actively start looking into this.
Author
Owner

@tjbck commented on GitHub (May 6, 2024):

Closing in favour of #2000, Let's continue our discussion there!

@tjbck commented on GitHub (May 6, 2024): Closing in favour of #2000, Let's continue our discussion there!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#647