Open WebUI Version:: latest (image id: 795de2b7c3e0)
Operating System:: Ubuntu 24.10
Confirmation:
I have read and followed all the instructions provided in the README.md.
I am on the latest version of both Open WebUI and Ollama.
I have included the browser console logs.
I have included the Docker container logs.
I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
Expected Behavior:
I want the web ui to give me the option to choose a model
Actual Behavior:
WebUI does not see openai compatible api
Description
I have vLLM running in docker with locally downloaded LLM at /mnt/weights/saiga_nemo_12b-Q6_K.gguf. I want open web ui to have the option to select this LLM. My command to run open webui is the following:
But I can't select any model at all in webui. How to fix this? I can provide all aditional information
Webui post request by this address:
Address is reachable:
open web-ui docker logs:
And there is no requests to vllm api (vllm docker logs):
Originally created by @JohnConnor123 on GitHub (Dec 24, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/8043
# Bug Report
---
## Installation Method
Docker
## Environment
- **Open WebUI Version:**: latest (image id: 795de2b7c3e0)
- **Operating System:**: Ubuntu 24.10
**Confirmation:**
- [x] I have read and followed all the instructions provided in the README.md.
- [x] I am on the latest version of both Open WebUI and Ollama.
- [x] I have included the browser console logs.
- [x] I have included the Docker container logs.
- [x] I have provided the exact steps to reproduce the bug in the "Steps to Reproduce" section below.
## Expected Behavior:
I want the web ui to give me the option to choose a model
## Actual Behavior:
WebUI does not see openai compatible api
## Description
I have vLLM running in docker with locally downloaded LLM at /mnt/weights/saiga_nemo_12b-Q6_K.gguf. I want open web ui to have the option to select this LLM. My command to run open webui is the following:
start-webui.sh
```
docker run --name=open-webui --rm \
--env=ENABLE_RAG_WEB_SEARCH=true \
--env=RAG_WEB_SEARCH_ENGINE=duckduckgo \
--env=COMFYUI_BASE_URL=http://<IP>:52002 \
--env=ENABLE_IMAGE_GENERATION=true \
--env=IMAGE_SIZE=1024x1024 \
--env='IMAGE_GENERATION_MODEL=Stable Diffusion 3 - Medium' \
--env=WHISPER_MODEL=medium \
--volume=/home/thinclient/llm-server/webui-data:/app/backend/data \
-p 52001:8080 \
ghcr.io/open-webui/open-webui:main
```
start-vllm.sh:
```
docker run --gpus '"device=0"' --rm -d \
--name vllm \
-v ~/.cache/huggingface:/root/.cache/huggingface \
-v /home/thinclient/llm-server/weights:/mnt/weights \
--env "HUGGING_FACE_HUB_TOKEN=<HF_API_TOKEN>" \
-p 52000:8000 \
--ipc host \
vllm/vllm-openai:latest \
--model facebook/opt-125m \
--tensor-parallel-size 1 \
--pipeline-parallel-size 1 \
--gpu-memory-utilization 0.97 \
--max_model_len 10000 \
```
But I can't select any model at all in webui. How to fix this? I can provide all aditional information
Webui post request by this address:
Address is reachable:
open web-ui docker logs:
And there is no requests to vllm api (vllm docker logs):
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @JohnConnor123 on GitHub (Dec 24, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/8043
Bug Report
Installation Method
Docker
Environment
Open WebUI Version:: latest (image id: 795de2b7c3e0)
Operating System:: Ubuntu 24.10
Confirmation:
Expected Behavior:
I want the web ui to give me the option to choose a model
Actual Behavior:
WebUI does not see openai compatible api
Description
I have vLLM running in docker with locally downloaded LLM at /mnt/weights/saiga_nemo_12b-Q6_K.gguf. I want open web ui to have the option to select this LLM. My command to run open webui is the following:
start-webui.sh
start-vllm.sh:
But I can't select any model at all in webui. How to fix this? I can provide all aditional information
Webui post request by this address:
Address is reachable:
open web-ui docker logs:
And there is no requests to vllm api (vllm docker logs):