[GH-ISSUE #8043] Open WebUI has problems with VLLM OpenAI compatible API #30504

New Issue

GiteaMirror · 2026-04-25T04:41:36-05:00

GiteaMirror commented

2026-04-25 04:41:36 -05:00

Originally created by @JohnConnor123 on GitHub (Dec 24, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/8043

Bug Report

Installation Method

Docker

Environment

Open WebUI Version:: latest (image id: 795de2b7c3e0)
Operating System:: Ubuntu 24.10

Confirmation:

I have read and followed all the instructions provided in the README.md.
I am on the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.

Expected Behavior:

I want the web ui to give me the option to choose a model

Actual Behavior:

WebUI does not see openai compatible api

Description

I have vLLM running in docker with locally downloaded LLM at /mnt/weights/saiga_nemo_12b-Q6_K.gguf. I want open web ui to have the option to select this LLM. My command to run open webui is the following:

start-webui.sh

docker run --name=open-webui --rm \
--env=ENABLE_RAG_WEB_SEARCH=true \
--env=RAG_WEB_SEARCH_ENGINE=duckduckgo \
--env=COMFYUI_BASE_URL=http://<IP>:52002 \
--env=ENABLE_IMAGE_GENERATION=true \
--env=IMAGE_SIZE=1024x1024 \
--env='IMAGE_GENERATION_MODEL=Stable Diffusion 3 - Medium' \
--env=WHISPER_MODEL=medium \
--volume=/home/thinclient/llm-server/webui-data:/app/backend/data \
-p 52001:8080 \
ghcr.io/open-webui/open-webui:main

start-vllm.sh:

docker run --gpus '"device=0"' --rm -d \
    --name vllm \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    -v /home/thinclient/llm-server/weights:/mnt/weights \
    --env "HUGGING_FACE_HUB_TOKEN=<HF_API_TOKEN>" \
    -p 52000:8000 \
    --ipc host \
    vllm/vllm-openai:latest \
    --model facebook/opt-125m \
    --tensor-parallel-size 1 \
    --pipeline-parallel-size 1 \
    --gpu-memory-utilization 0.97 \
    --max_model_len 10000 \

But I can't select any model at all in webui. How to fix this? I can provide all aditional information

Webui post request by this address:

Address is reachable:

open web-ui docker logs:

And there is no requests to vllm api (vllm docker logs):

Originally created by @JohnConnor123 on GitHub (Dec 24, 2024). Original GitHub issue: https://github.com/open-webui/open-webui/issues/8043 # Bug Report --- ## Installation Method Docker ## Environment - **Open WebUI Version:**: latest (image id: 795de2b7c3e0) - **Operating System:**: Ubuntu 24.10 **Confirmation:** - [x] I have read and followed all the instructions provided in the README.md. - [x] I am on the latest version of both Open WebUI and Ollama. - [x] I have included the browser console logs. - [x] I have included the Docker container logs. - [x] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below. ## Expected Behavior: I want the web ui to give me the option to choose a model ## Actual Behavior: WebUI does not see openai compatible api ## Description I have vLLM running in docker with locally downloaded LLM at /mnt/weights/saiga_nemo_12b-Q6_K.gguf. I want open web ui to have the option to select this LLM. My command to run open webui is the following: start-webui.sh ``` docker run --name=open-webui --rm \ --env=ENABLE_RAG_WEB_SEARCH=true \ --env=RAG_WEB_SEARCH_ENGINE=duckduckgo \ --env=COMFYUI_BASE_URL=http://<IP>:52002 \ --env=ENABLE_IMAGE_GENERATION=true \ --env=IMAGE_SIZE=1024x1024 \ --env='IMAGE_GENERATION_MODEL=Stable Diffusion 3 - Medium' \ --env=WHISPER_MODEL=medium \ --volume=/home/thinclient/llm-server/webui-data:/app/backend/data \ -p 52001:8080 \ ghcr.io/open-webui/open-webui:main ``` start-vllm.sh: ``` docker run --gpus '"device=0"' --rm -d \ --name vllm \ -v ~/.cache/huggingface:/root/.cache/huggingface \ -v /home/thinclient/llm-server/weights:/mnt/weights \ --env "HUGGING_FACE_HUB_TOKEN=<HF_API_TOKEN>" \ -p 52000:8000 \ --ipc host \ vllm/vllm-openai:latest \ --model facebook/opt-125m \ --tensor-parallel-size 1 \ --pipeline-parallel-size 1 \ --gpu-memory-utilization 0.97 \ --max_model_len 10000 \ ``` But I can't select any model at all in webui. How to fix this? I can provide all aditional information ![image](https://github.com/user-attachments/assets/759db567-ea5b-4a88-a59e-09bc0bbc74b0) Webui post request by this address: ![image](https://github.com/user-attachments/assets/87bad668-d56c-4448-9973-e70ab0496d70) Address is reachable: ![image](https://github.com/user-attachments/assets/e294a2f1-6205-4ee2-8ce7-482365a006b4) open web-ui docker logs: ![image](https://github.com/user-attachments/assets/2b6e32e7-84cb-48dc-b17d-633c4ef29f79) And there is no requests to vllm api (vllm docker logs): ![image](https://github.com/user-attachments/assets/f59a4cab-b17c-4a85-8fd8-6b4e791ff249)

GiteaMirror closed this issue

2026-04-25 04:41:36 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#30504