Is your feature request related to a problem? Please describe.
TabbyAPI and text-gen-webui are popular interfaces for run performant gpu quantizations, specifically the exl2 quantizations.
I tried running each, text-gen-webui with no API key, and tabbyAPI with its key.
I'm not able to see text-gen-webui models or get something working there, but for tabbyAPI I can see the available models, and can select the current one loaded. However, the response I get it always a single word:
Originally created by @BarfingLemurs on GitHub (Apr 24, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1735
**Is your feature request related to a problem? Please describe.**
TabbyAPI and text-gen-webui are popular interfaces for run performant gpu quantizations, specifically the exl2 quantizations.
I tried running each, text-gen-webui with no API key, and tabbyAPI with its key.
I'm not able to see text-gen-webui models or get something working there, but for tabbyAPI I can see the available models, and can select the current one loaded. However, the response I get it always a single word:
https://imgur.com/a/9hjoRFh
### Info
After login, go to Newchat > settings connections > show openai API http://192.168.1.4:5000/v1 & key
Standard cpu docker installation, then went to localhost:3000
tabbyAPI log: https://pastebin.com/KnBmZtp3
linux, Firefox browser
OpenAI API endpoint configurations need to have an API key or it won't activate.
<!-- gh-comment-id:2076053109 -->
@justinh-rahb commented on GitHub (Apr 24, 2024):
OpenAI API endpoint configurations need to have an API key or it won't activate.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @BarfingLemurs on GitHub (Apr 24, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/1735
Is your feature request related to a problem? Please describe.
TabbyAPI and text-gen-webui are popular interfaces for run performant gpu quantizations, specifically the exl2 quantizations.
I tried running each, text-gen-webui with no API key, and tabbyAPI with its key.
I'm not able to see text-gen-webui models or get something working there, but for tabbyAPI I can see the available models, and can select the current one loaded. However, the response I get it always a single word:
https://imgur.com/a/9hjoRFh
Info
After login, go to Newchat > settings connections > show openai API http://192.168.1.4:5000/v1 & key
Standard cpu docker installation, then went to localhost:3000
tabbyAPI log: https://pastebin.com/KnBmZtp3
linux, Firefox browser
@tjbck commented on GitHub (Apr 24, 2024):
A
@tjbck commented on GitHub (Apr 24, 2024):
I'm just kidding 😂 That's strange, I'll take a look!
@justinh-rahb commented on GitHub (Apr 24, 2024):
OpenAI API endpoint configurations need to have an API key or it won't activate.