Commercial LLM providers like Google and SambaNova are recently trying to offer OpenAI endpoint compatibility, but they often forget to fully implement the /models endpoint used by OWUI to dynamically list available models. If the Connections admin panel had some sort of optional "model list override", we could start using these providers even was they drag their feet building out fill OpenAI compatibility.
When the user clicks the "Verify connection" button when an override list is non-empty, then OWUI would instead try to get information only for the first item on the override list rather than trying to dynamically load the list.
Originally created by @rndmcnlly on GitHub (Nov 9, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/6821
Originally assigned to: @tjbck on GitHub.
# Feature Request
Commercial LLM providers like Google and SambaNova are recently trying to offer OpenAI endpoint compatibility, but they often forget to fully implement the `/models` endpoint used by OWUI to dynamically list available models. If the Connections admin panel had some sort of optional "model list override", we could start using these providers even was they drag their feet building out fill OpenAI compatibility.
When the user clicks the "Verify connection" button when an override list is non-empty, then OWUI would instead try to get information only for the first item on the override list rather than trying to dynamically load the list.
Had the same issue for LitServe, as they don't implement /models in their openAI-API standard as well :( luckily I could extend their API via a workaround. So 👍🏼 +1 from my side, would be a great feature.
<!-- gh-comment-id:2466828514 -->
@fabigr8 commented on GitHub (Nov 10, 2024):
Had the same issue for LitServe, as they don't implement `/models` in their openAI-API standard as well :( luckily I could extend their API via a workaround. So 👍🏼 +1 from my side, would be a great feature.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @rndmcnlly on GitHub (Nov 9, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/6821
Originally assigned to: @tjbck on GitHub.
Feature Request
Commercial LLM providers like Google and SambaNova are recently trying to offer OpenAI endpoint compatibility, but they often forget to fully implement the
/modelsendpoint used by OWUI to dynamically list available models. If the Connections admin panel had some sort of optional "model list override", we could start using these providers even was they drag their feet building out fill OpenAI compatibility.When the user clicks the "Verify connection" button when an override list is non-empty, then OWUI would instead try to get information only for the first item on the override list rather than trying to dynamically load the list.
@tjbck commented on GitHub (Nov 9, 2024):
Been on my mind lately, investigating.
@fabigr8 commented on GitHub (Nov 10, 2024):
Had the same issue for LitServe, as they don't implement
/modelsin their openAI-API standard as well :( luckily I could extend their API via a workaround. So 👍🏼 +1 from my side, would be a great feature.@tjbck commented on GitHub (Nov 12, 2024):
Added to dev, Testing wanted here!