llama-cpp-python server not working with "OpenAI API Connections" #999

New Issue

2025-11-11T14:35:18-06:00

GiteaMirror commented

2025-11-11 14:35:18 -06:00

Originally created by @nick-tonjum on GitHub (May 22, 2024).

Hello!
I'm having an issue with the 'OpenAI API Connections' in the settings.
I currently have my OpenAI api key working there perfectly, but when I try to add the address for a llama-cpp-python server it isn't listing the models whatsoever, despite saying it successfully added.

My open-webui instance is running in a docker container, so this is what I have entered:
http://host.docker.internal:4883/v1

no API key since it's internal, but still doesn't work if I have one set. I also tried localhost, 0.0.0.0, 127.0.0.1, and even tried opening it up to my public IP and it didn't work. I can manually GET the models at http://host.docker.internal:4883/v1/models and it works without issue.

I also know it's not an issue with host.docker.internal, as my ollama server itself is running on it and I have my ollama server pointed to host.docker.interal:11434.

I'm stuck.

Thanks!

Originally created by @nick-tonjum on GitHub (May 22, 2024). Hello! I'm having an issue with the 'OpenAI API Connections' in the settings. I currently have my OpenAI api key working there perfectly, but when I try to add the address for a llama-cpp-python server it isn't listing the models whatsoever, despite saying it successfully added. My open-webui instance is running in a docker container, so this is what I have entered: http://host.docker.internal:4883/v1 no API key since it's internal, but still doesn't work if I have one set. I also tried localhost, 0.0.0.0, 127.0.0.1, and even tried opening it up to my public IP and it didn't work. I can manually GET the models at http://host.docker.internal:4883/v1/models and it works without issue. I also know it's not an issue with host.docker.internal, as my ollama server itself is running on it and I have my ollama server pointed to host.docker.interal:11434. I'm stuck. Thanks!

GiteaMirror closed this issue

2025-11-11 14:35:19 -06:00

GiteaMirror referenced this issue

2026-04-19 19:11:19 -05:00

[GH-ISSUE #999] Support Ollama's OpenAI Compatible APIs #12288

GiteaMirror referenced this issue

2026-04-25 02:35:00 -05:00

[GH-ISSUE #999] Support Ollama's OpenAI Compatible APIs #27816

GiteaMirror referenced this issue

2026-05-05 11:37:45 -05:00

[GH-ISSUE #999] Support Ollama's OpenAI Compatible APIs #50953

GiteaMirror referenced this issue