[GH-ISSUE #9796] Categories in leaderboard and server-side search #102645

New Issue

GiteaMirror · 2026-05-17T23:57:32-05:00

GiteaMirror commented

2026-05-17 23:57:32 -05:00

Originally created by @Fetnak on GitHub (Feb 11, 2025).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/9796

Feature Request

Is your feature request related to a problem? Please describe.

The current tag system often generates unrelated tags. Moreover, when I manually add my own tags, they are not suggested in subsequent evaluations.
Additionally, the client-side leaderboard search functionality is not working well, particularly on weaker devices where it often fails to work altogether.

Describe the solution you'd like

I propose adding a separate section in the settings to create your own custom categories, such as: C++ Coding, Python Coding, Translate and General as fallback. Each evaluation should be required to have a category assigned to it, or General if category was not selected.
Also, I suggest moving the search functionality to the server-side.

Additional context

Without proper evaluations, it's difficult to determine which model doesn't meet my requirements and should be removed. I don't want to accidentally delete a model that performs poorly across the board but excels in translations, for example, even if it ends up at the bottom of the leaderboard. Additionally, it's challenging to select the right model for a specific task.
I hope you implement a more convenient evaluation system. Otherwise, I will have to host several Open WebUI instances just to have a normal leaderboard, because right now, I don't see any other solution.

Originally created by @Fetnak on GitHub (Feb 11, 2025). Original GitHub issue: https://github.com/open-webui/open-webui/issues/9796 # Feature Request **Is your feature request related to a problem? Please describe.** The current tag system often generates unrelated tags. Moreover, when I manually add my own tags, they are not suggested in subsequent evaluations. Additionally, the client-side leaderboard search functionality is not working well, particularly on weaker devices where it often fails to work altogether. **Describe the solution you'd like** I propose adding a separate section in the settings to create your own custom categories, such as: `C++ Coding`, `Python Coding`, `Translate` and `General` as fallback. Each evaluation should be required to have a category assigned to it, or `General` if category was not selected. Also, I suggest moving the search functionality to the server-side. **Additional context** Without proper evaluations, it's difficult to determine which model doesn't meet my requirements and should be removed. I don't want to accidentally delete a model that performs poorly across the board but excels in translations, for example, even if it ends up at the bottom of the leaderboard. Additionally, it's challenging to select the right model for a specific task. I hope you implement a more convenient evaluation system. Otherwise, I will have to host several Open WebUI instances just to have a normal leaderboard, because right now, I don't see any other solution.

GiteaMirror closed this issue

2026-05-17 23:57:33 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#102645