issue: Custom models not finding LiteLLM base models (404) #4838

New Issue

2025-11-11T16:04:20-06:00

GiteaMirror commented

2025-11-11 16:04:20 -06:00

Originally created by @thibmaek on GitHub (Apr 14, 2025).

Check Existing Issues

I have searched the existing issues and discussions.
I am using the latest version of Open WebUI.

Installation Method

Docker

Open WebUI Version

v0.6.4

Ollama Version (if applicable)

v0.6.4

Operating System

Debian 12

Browser (if applicable)

Arc (latest)

Confirmation

I have read and followed all instructions in README.md.
I am using the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have listed steps to reproduce the bug in detail.

Expected Behavior

Create a new assistant model CTO from base models (litellm)
@ tag the model in chat UI
model uses the system prompt on top of the base model to answer

Actual Behavior

Chat gives 404 Model not found when using custom model
Using the models from LiteLLM directly works fine
Dropdown is truncating text for available models

Steps to Reproduce

Add LiteLLM base url in Settings > Connections
Verify that you can talk to any model from LiteLLM directly from the model list
Create a new custom model from any LiteLLM base model & save
Talk to the new custom model

Logs & Screenshots

Docker logs:

2025-04-14 09:13:34.940 | INFO     | uvicorn.protocols.http.httptools_impl:send:476 - x.x.x.x:55376 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 - {}
2025-04-14 09:14:04.259 | INFO     | uvicorn.protocols.http.httptools_impl:send:476 - x.x.x.x:55459 - "POST /api/v1/chats/cb15f623-bc4e-436a-a685-dae2634bdfd0 HTTP/1.1" 200 - {}
2025-04-14 09:14:04.270 | INFO     | uvicorn.protocols.http.httptools_impl:send:476 - x.x.x.x:55459 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 - {}
2025-04-14 09:14:04.383 | INFO     | uvicorn.protocols.http.httptools_impl:send:476 - x.x.x.x:55459 - "POST /api/v1/memories/query HTTP/1.1" 200 - {}
2025-04-14 09:14:05.139 | INFO     | open_webui.routers.openai:get_all_models:389 - get_all_models() - {}
2025-04-14 09:14:05.140 | INFO     | uvicorn.protocols.http.httptools_impl:send:476 - x.x.x.x:55459 - "POST /api/chat/completions HTTP/1.1" 400 - {}

Browser logs:

LiteLLM configuration:

Custom model config:

Error:

Additional Information

This works fine if I configure the base model as a local Ollama model (e.g mistral-small:latest)
- I'm also using a remote Ollama instance on my local network, so not on the same host as Open WebUI is running in Docker
LiteLLM models work when using directly, only not in custom models.
- Also is set up according to https://docs.litellm.ai/docs/tutorials/openweb_ui

Docker compose config:

services:
  openwebui:
    container_name: openwebui
    image: ghcr.io/open-webui/open-webui
    env_file: openwebui.env
    volumes:
      - ./data:/app/backend/data
    ports:
      - 6390:8080
    restart: unless-stopped
    extra_hosts:
      - host.docker.internal:host-gateway

env file:

OLLAMA_BASE_URL=http://x.x.x.x:11434
OPENAI_API_BASE_URL=http://172.17.0.1:4000/v1

Originally created by @thibmaek on GitHub (Apr 14, 2025). ### Check Existing Issues - [x] I have searched the existing issues and discussions. - [x] I am using the latest version of Open WebUI. ### Installation Method Docker ### Open WebUI Version v0.6.4 ### Ollama Version (if applicable) v0.6.4 ### Operating System Debian 12 ### Browser (if applicable) Arc (latest) ### Confirmation - [x] I have read and followed all instructions in `README.md`. - [x] I am using the latest version of **both** Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have listed steps to reproduce the bug in detail. ### Expected Behavior - Create a new assistant model CTO from base models (litellm) - @ tag the model in chat UI - model uses the system prompt on top of the base model to answer ### Actual Behavior - Chat gives 404 Model not found when using custom model - Using the models from LiteLLM directly works fine - Dropdown is truncating text for available models ### Steps to Reproduce 1. Add LiteLLM base url in Settings > Connections 2. Verify that you can talk to any model from LiteLLM directly from the model list 3. Create a new custom model from any LiteLLM base model & save 4. Talk to the new custom model ### Logs & Screenshots **Docker logs:** ```plaintext 2025-04-14 09:13:34.940 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - x.x.x.x:55376 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 - {} 2025-04-14 09:14:04.259 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - x.x.x.x:55459 - "POST /api/v1/chats/cb15f623-bc4e-436a-a685-dae2634bdfd0 HTTP/1.1" 200 - {} 2025-04-14 09:14:04.270 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - x.x.x.x:55459 - "GET /api/v1/chats/?page=1 HTTP/1.1" 200 - {} 2025-04-14 09:14:04.383 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - x.x.x.x:55459 - "POST /api/v1/memories/query HTTP/1.1" 200 - {} 2025-04-14 09:14:05.139 | INFO | open_webui.routers.openai:get_all_models:389 - get_all_models() - {} 2025-04-14 09:14:05.140 | INFO | uvicorn.protocols.http.httptools_impl:send:476 - x.x.x.x:55459 - "POST /api/chat/completions HTTP/1.1" 400 - {} ``` **Browser logs:** ![Image](https://github.com/user-attachments/assets/bf8572da-5ff9-4c9e-b636-548307fb4b75) **LiteLLM configuration:** ![Image](https://github.com/user-attachments/assets/6ae912aa-a851-42f4-baaa-b149a34c17b7) ![Image](https://github.com/user-attachments/assets/cd02a1c1-a4e2-44c8-9293-b192ebcde349) **Custom model config:** ![Image](https://github.com/user-attachments/assets/944831be-1ca8-4baf-b61b-24fa0e3509b2) **Error:** ![Image](https://github.com/user-attachments/assets/b6a90d22-317e-41e6-ae3f-0ac0ebba44b9) ### Additional Information - This works fine if I configure the base model as a local Ollama model (e.g mistral-small:latest) - I'm also using a remote Ollama instance on my local network, so not on the same host as Open WebUI is running in Docker - LiteLLM models work when using directly, only not in custom models. - Also is set up according to https://docs.litellm.ai/docs/tutorials/openweb_ui Docker compose config: ```yaml services: openwebui: container_name: openwebui image: ghcr.io/open-webui/open-webui env_file: openwebui.env volumes: - ./data:/app/backend/data ports: - 6390:8080 restart: unless-stopped extra_hosts: - host.docker.internal:host-gateway ``` env file: ```env OLLAMA_BASE_URL=http://x.x.x.x:11434 OPENAI_API_BASE_URL=http://172.17.0.1:4000/v1 ```

GiteaMirror added the bug label 2025-11-11 16:04:21 -06:00

GiteaMirror closed this issue

2025-11-11 16:04:21 -06:00

GiteaMirror commented

2025-11-11 16:04:21 -06:00

@thibmaek commented on GitHub (Apr 14, 2025):

I redacted my local IP for Ollama host as x.x.x.x

@thibmaek commented on GitHub (Apr 14, 2025): I redacted my local IP for Ollama host as x.x.x.x

GiteaMirror commented

2025-11-11 16:04:22 -06:00

@miversen33 commented on GitHub (Apr 14, 2025):

I am noticing the same behavior when using https://openwebui.com/m/hub/based-dolphin-mixtral:latest

I click get, I add it to my instance, I open a chat and see

Looking at the logs, it seems that /api/chat/completions is throwing a 400

Nothing terribly interesting in the logs either

@miversen33 commented on GitHub (Apr 14, 2025): I am noticing the same behavior when using https://openwebui.com/m/hub/based-dolphin-mixtral:latest I click `get`, I add it to my instance, I open a chat and see ![Image](https://github.com/user-attachments/assets/8dabd317-5442-45ff-a926-90e8bf64bd15) Looking at the logs, it seems that `/api/chat/completions` is throwing a 400 ![Image](https://github.com/user-attachments/assets/4222969d-0949-4911-804d-b052a4a77cd6) Nothing terribly interesting in the logs either ![Image](https://github.com/user-attachments/assets/4d51c6b7-f8df-4cc9-8cca-9c6da502b02b)

GiteaMirror commented

2025-11-11 16:04:22 -06:00