Originally created by @aleixdorca on GitHub (Oct 14, 2024).
Hi everyone!
I was thinking it might be helpful to add a feature that allows Admins to control which model parameters users can adjust.
Currently, users have the freedom to change any parameter, which can sometimes lead to performance issues on the local server.
For instance, if a model is set up with a specific context size to manage VRAM usage efficiently, a user may change this setting. The result might be that the model starts using the CPU instead of the GPU, potentially causing disruptions in the server's quality of service (QoS).
What do you think?
Best regards.
Originally created by @aleixdorca on GitHub (Oct 14, 2024).
Hi everyone!
I was thinking it might be helpful to add a feature that allows Admins to control which model parameters users can adjust.
Currently, users have the freedom to change any parameter, which can sometimes lead to performance issues on the local server.
For instance, if a model is set up with a specific context size to manage VRAM usage efficiently, a user may change this setting. The result might be that the model starts using the CPU instead of the GPU, potentially causing disruptions in the server's quality of service (QoS).
What do you think?
Best regards.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @aleixdorca on GitHub (Oct 14, 2024).
Hi everyone!
I was thinking it might be helpful to add a feature that allows Admins to control which model parameters users can adjust.
Currently, users have the freedom to change any parameter, which can sometimes lead to performance issues on the local server.
For instance, if a model is set up with a specific context size to manage VRAM usage efficiently, a user may change this setting. The result might be that the model starts using the CPU instead of the GPU, potentially causing disruptions in the server's quality of service (QoS).
What do you think?
Best regards.