mirror of
https://github.com/open-webui/open-webui.git
synced 2026-06-05 16:28:32 -05:00
llama-cpp-python server not working with "OpenAI API Connections" #999
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/open-webui#999
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @nick-tonjum on GitHub (May 22, 2024).
Hello!
I'm having an issue with the 'OpenAI API Connections' in the settings.
I currently have my OpenAI api key working there perfectly, but when I try to add the address for a llama-cpp-python server it isn't listing the models whatsoever, despite saying it successfully added.
My open-webui instance is running in a docker container, so this is what I have entered:
http://host.docker.internal:4883/v1
no API key since it's internal, but still doesn't work if I have one set. I also tried localhost, 0.0.0.0, 127.0.0.1, and even tried opening it up to my public IP and it didn't work. I can manually GET the models at http://host.docker.internal:4883/v1/models and it works without issue.
I also know it's not an issue with host.docker.internal, as my ollama server itself is running on it and I have my ollama server pointed to host.docker.interal:11434.
I'm stuck.
Thanks!