Change global presets for models #3496

New Issue

2025-11-11T15:32:52-06:00

GiteaMirror commented

2025-11-11 15:32:52 -06:00

Originally created by @machiav3lli on GitHub (Jan 30, 2025).

Feature Request

Is your feature request related to a problem? Please describe.

As an admin of small instance for some friends, I'd like to be able to change parameters for all models at once (e.g. temperature, top P, context length). Naturally respecting the limits of the respective model when overriding.

Describe the solution you'd like

In admin panel → settings → models: add an option for global settings of (advanced) parameters of models.

Furthermore, a hierarchy of overwrites should be kept (from higher to lower priority): Chat controls → User's general settings → Model settings → Global settings.

Describe alternatives you've considered

Alternative address: admin panel → settings → general

Additional context

For example: depending on the list of the models you'd be providing, some models have high hallucination probability even with 0.8 temperature. So if you have 8 models, 6 of which come from model families that require lower temperature (e.g. qwen 0.5-0.7), while the other two are fine with the standard (0.8). Currently you'll have to manually change the temperature of all the 6 models and their respective family updates (usually with new names/identifiers). Users have currently the option in settings → general to set such parameter, but for such managed instances, it's usually so that the users don't really know why/how/what to change such (advanced) parameters and the admins are the usual managers of such defaults.

Originally created by @machiav3lli on GitHub (Jan 30, 2025). # Feature Request **Is your feature request related to a problem? Please describe.** As an admin of small instance for some friends, I'd like to be able to change parameters for all models at once (e.g. temperature, top P, context length). Naturally respecting the limits of the respective model when overriding. **Describe the solution you'd like** In admin panel → settings → models: add an option for global settings of (advanced) parameters of models. Furthermore, a hierarchy of overwrites should be kept (from higher to lower priority): Chat controls → User's general settings → Model settings → Global settings. **Describe alternatives you've considered** Alternative address: admin panel → settings → general **Additional context** For example: depending on the list of the models you'd be providing, some models have high hallucination probability even with 0.8 temperature. So if you have 8 models, 6 of which come from model families that require lower temperature (e.g. qwen 0.5-0.7), while the other two are fine with the standard (0.8). Currently you'll have to manually change the temperature of all the 6 models and their respective family updates (usually with new names/identifiers). Users have currently the option in settings → general to set such parameter, but for such managed instances, it's usually so that the users don't really know why/how/what to change such (advanced) parameters and the admins are the usual managers of such defaults.

GiteaMirror closed this issue