Is your feature request related to a problem? Please describe.
Continuing the issue from #4983, I only realized today that Context Length is completely ineffective for API models and only works for Ollama models...
We need the max_token parameter to manage the context to ensure API costs are consumed within the expected range
Describe the solution you'd like
Add max_token parameter support for API models
Describe alternatives you've considered
Functions like Context Clip Filter cannot replace this functionality in API parameters, and even if some other functions implement the feature to trim the context to only the specified number of tokens, many people won't visit the Open WebUI community to load it.
Additional context
I really think that many features of Open WebUI should not rely on functions or plugins to implement. They are really basic (such as max_token and Max Message Count in Context, etc.), and not everyone has the patience or can discover that the community has the functions they need... Please seriously consider this point.
Originally created by @CookSleep on GitHub (Sep 15, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/5446
**Is your feature request related to a problem? Please describe.**
Continuing the issue from #4983, I only realized today that `Context Length` is completely ineffective for API models and only works for Ollama models...
We need the `max_token` parameter to manage the context to ensure API costs are consumed within the expected range
**Describe the solution you'd like**
Add `max_token` parameter support for API models
**Describe alternatives you've considered**
Functions like `Context Clip Filter` cannot replace this functionality in API parameters, and even if some other functions implement the feature to trim the context to only the specified number of tokens, many people won't visit the Open WebUI community to load it.
**Additional context**
I really think that many features of Open WebUI **should not** rely on functions or plugins to implement. They are really basic (such as `max_token` and `Max Message Count in Context`, etc.), and not everyone has the patience or can discover that the community has the functions they need... Please seriously consider this point.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @CookSleep on GitHub (Sep 15, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/5446
Is your feature request related to a problem? Please describe.
Continuing the issue from #4983, I only realized today that
Context Lengthis completely ineffective for API models and only works for Ollama models...We need the
max_tokenparameter to manage the context to ensure API costs are consumed within the expected rangeDescribe the solution you'd like
Add
max_tokenparameter support for API modelsDescribe alternatives you've considered
Functions like
Context Clip Filtercannot replace this functionality in API parameters, and even if some other functions implement the feature to trim the context to only the specified number of tokens, many people won't visit the Open WebUI community to load it.Additional context
I really think that many features of Open WebUI should not rely on functions or plugins to implement. They are really basic (such as
max_tokenandMax Message Count in Context, etc.), and not everyone has the patience or can discover that the community has the functions they need... Please seriously consider this point.