github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-08-30 17:25:30 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

Local-AI: Empty model selections (when they shouldn't be) #1166

New Issue
Closed
opened 2025-11-11 14:39:07 -06:00 by GiteaMirror · 1 comment
GiteaMirror commented 2025-11-11 14:39:07 -06:00
Owner
Copy Link
Copy Source

Originally created by @senpro-ingwersenk on GitHub (Jun 7, 2024).

Bug Report

Description

Bug Summary:
I configured my LocalAI instance as the OpenAI API endpoint; when I ose curl to verify, I see the models just fine:

curl.exe http://192.168.28.174/localai/v1/models
{"object":"list","data":[{"id":"LocalAI-llama3-8b-function-call-v0.2","object":"model"},{"id":"bert-embeddings","object":"model"},{"id":"llama-3-sauerkrautlm-8b-instruct","object":"model"},{"id":"llama3-8b-instruct","object":"model"},{"id":"llava-1.6-vicuna","object":"model"},{"id":"mirai-nova-llama3-LocalAI-8b-v0.1","object":"model"},{"id":"mistral-7b-instruct-v0.3","object":"model"},{"id":"moondream2","object":"model"},{"id":"openvino-multilingual-e5-base","object":"model"},{"id":"openvino-phi3","object":"model"},{"id":"phi-3-mini-4k-instruct","object":"model"},{"id":"whisper-tiny","object":"model"},{"id":"mmproj-vicuna7b-f16.gguf","object":"model"},{"id":"moondream2-mmproj-f16.gguf","object":"model"}]}

However, the same URL (minus /models of course) doesn't result in any models being loaded at all. But it also still reports the connection as successful.

Steps to Reproduce:
You may wish to borrow my Caddyfile for proper reproduction:

:80 {
  #handle_path /chroma* {}
  handle_path /localai* {
    reverse_proxy * http://localhost:8080 # Serves LocalAI
  }
  handle /* {
    reverse_proxy * http://localhost:3000 # Serves Open WebUI
  }
}

Launch the Docker containers accordingly. Here is a snippet from my docker-compose.yml:

  localai:
    image: quay.io/go-skynet/local-ai:latest-gpu-nvidia-cuda-12
    restart: always
    ports: [ 0.0.0.0:8080:8080 ]
    volumes:
      - ./models:/build/models
    environment:
      LOCALAI_LOG_LEVEL: "debug"
      LOCALAI_THREADS: "20"
      LOCALAI_SINGLE_ACTIVE_BACKEND: "true"
      REBUILD: "true"
      BUILD_TYPE: "cublas"
      BUILD_PARALLELISM: 24
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

  openwebui:
    #image: ghcr.io/open-webui/open-webui:main
    image: ghcr.io/open-webui/open-webui:ollama
    volumes:
      - ./openwebui:/app/backend/data
      - ./ollama:/root/.ollama
    extra_hosts:
      - "host.docker.internal:host-gateway"
    ports:
      - 0.0.0.0:3000:8080
    environment:
      OPENAI_API_BASE_URL: http://192.168.28.174/localai/v1
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

Adjust this and the Caddyfile accordingly, then run.
When you open the UI, there will be no available models to chose from:
grafik

Expected Behavior:
I expected to be able to find and see models to chat with or run RAG.

Actual Behavior:
An empty list while the API clearly reports them all.

Environment

  • Open WebUI Version: ghcr.io/open-webui/open-webui:ollama (Chosen, so I can potentially utilize both LocalAI and Ollama models situationally.)

  • Ollama (if applicable): As embedded in the container.

  • Operating System: Debian Bookworm, amd64

  • Browser (if applicable): Firefox

Reproduction Details

Confirmation:

  • I have read and followed all the instructions provided in the README.md.
  • I am on the latest version of both Open WebUI and Ollama.
  • I have included the browser console logs.
  • I have included the Docker container logs.

Logs and Screenshots

Browser Console Logs:
grafik

Docker Container Logs:

openwebui_1  | user f54c42d4-6a26-454c-be87-ce3967387692 disconnected with session ID F2SE_IH1K53atWxoAAAL
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:     192.168.30.195:0 - "GET / HTTP/1.1" 304 Not Modified
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:     192.168.30.195:0 - "GET /api/config HTTP/1.1" 200 OK
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:     192.168.30.195:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=O_n-lDW HTTP/1.1" 200 OK
openwebui_1  | INFO:     192.168.30.195:0 - "GET /api/v1/auths/ HTTP/1.1" 200 OK
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | connect  NXQo8FR0wgUyiQ36AAAN
openwebui_1  | user Kevin Ingwersen(f54c42d4-6a26-454c-be87-ce3967387692) connected with session ID NXQo8FR0wgUyiQ36AAAN
openwebui_1  | 1
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:     ('192.168.30.195', 0) - "WebSocket /ws/socket.io/?EIO=4&transport=websocket&sid=AHnf-DBGXoLh5XabAAAM" [accepted]
openwebui_1  | INFO:     connection open
openwebui_1  | Models in use: []
openwebui_1  | INFO:     192.168.30.195:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=O_n-lD_&sid=AHnf-DBGXoLh5XabAAAM HTTP/1.1" 200 OK
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:     192.168.30.195:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=O_n-lE0&sid=AHnf-DBGXoLh5XabAAAM HTTP/1.1" 200 OK
openwebui_1  | INFO:     192.168.30.195:0 - "GET /api/changelog HTTP/1.1" 200 OK
openwebui_1  | INFO:     192.168.30.195:0 - "GET /api/v1/users/user/settings HTTP/1.1" 200 OK
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:     192.168.30.195:0 - "GET /api/v1/prompts/ HTTP/1.1" 200 OK
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:     192.168.30.195:0 - "GET /api/v1/configs/banners HTTP/1.1" 200 OK
openwebui_1  | INFO:     192.168.30.195:0 - "GET /api/models HTTP/1.1" 200 OK
openwebui_1  | INFO:     192.168.30.195:0 - "GET /api/v1/documents/ HTTP/1.1" 200 OK
openwebui_1  | INFO:     192.168.30.195:0 - "GET /api/v1/chats/tags/all HTTP/1.1" 200 OK
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:apps.ollama.main:get_all_models()
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:     192.168.30.195:0 - "GET /ollama/api/version HTTP/1.1" 200 OK
openwebui_1  | INFO:     192.168.30.195:0 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
openwebui_1  | INFO:     192.168.30.195:0 - "GET /api/v1/users/user/settings HTTP/1.1" 200 OK

This happens during the connection test:

openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:config:Saving 'OPENAI_API_BASE_URLS' to config.json
openwebui_1  | INFO:     192.168.30.195:0 - "POST /openai/urls/update HTTP/1.1" 200 OK
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:     192.168.30.195:0 - "POST /openai/keys/update HTTP/1.1" 200 OK
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:apps.openai.main:get_all_models()
localai_1    | 9:16AM INF Success ip=172.18.0.1 latency="309.172µs" method=GET status=200 url=/v1/models
openwebui_1  | INFO:     192.168.30.195:0 - "GET /openai/models/0 HTTP/1.1" 200 OK
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:apps.openai.main:get_all_models()
openwebui_1  | INFO:     192.168.30.195:0 - "GET /api/models HTTP/1.1" 200 OK

Screenshots (if applicable):
See above.

Installation Method

Docker Compose; the service configurations are above.

Additional Information

I am running this as a temporary setup on an old IBM server with 2x Intel Xeon, 160GB RAM and a RTX 3060. Because of the way the network is built and restricted, I would like to use Caddy to simplify access to the service. Due to the abundance of RAM, it'd be neat to take advantage of it at some point - which is why I use both LocalAI and Ollama; let one manage the GPU and the other the CPU. That is the plan, at least.

Note

If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!

Originally created by @senpro-ingwersenk on GitHub (Jun 7, 2024). # Bug Report ## Description **Bug Summary:** I configured my LocalAI instance as the OpenAI API endpoint; when I ose curl to verify, I see the models just fine: ``` curl.exe http://192.168.28.174/localai/v1/models {"object":"list","data":[{"id":"LocalAI-llama3-8b-function-call-v0.2","object":"model"},{"id":"bert-embeddings","object":"model"},{"id":"llama-3-sauerkrautlm-8b-instruct","object":"model"},{"id":"llama3-8b-instruct","object":"model"},{"id":"llava-1.6-vicuna","object":"model"},{"id":"mirai-nova-llama3-LocalAI-8b-v0.1","object":"model"},{"id":"mistral-7b-instruct-v0.3","object":"model"},{"id":"moondream2","object":"model"},{"id":"openvino-multilingual-e5-base","object":"model"},{"id":"openvino-phi3","object":"model"},{"id":"phi-3-mini-4k-instruct","object":"model"},{"id":"whisper-tiny","object":"model"},{"id":"mmproj-vicuna7b-f16.gguf","object":"model"},{"id":"moondream2-mmproj-f16.gguf","object":"model"}]} ``` However, the same URL (minus `/models` of course) doesn't result in any models being loaded at all. But it also still reports the connection as successful. **Steps to Reproduce:** You may wish to borrow my Caddyfile for proper reproduction: ``` :80 { #handle_path /chroma* {} handle_path /localai* { reverse_proxy * http://localhost:8080 # Serves LocalAI } handle /* { reverse_proxy * http://localhost:3000 # Serves Open WebUI } } ``` Launch the Docker containers accordingly. Here is a snippet from my `docker-compose.yml`: ```yaml localai: image: quay.io/go-skynet/local-ai:latest-gpu-nvidia-cuda-12 restart: always ports: [ 0.0.0.0:8080:8080 ] volumes: - ./models:/build/models environment: LOCALAI_LOG_LEVEL: "debug" LOCALAI_THREADS: "20" LOCALAI_SINGLE_ACTIVE_BACKEND: "true" REBUILD: "true" BUILD_TYPE: "cublas" BUILD_PARALLELISM: 24 deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu] openwebui: #image: ghcr.io/open-webui/open-webui:main image: ghcr.io/open-webui/open-webui:ollama volumes: - ./openwebui:/app/backend/data - ./ollama:/root/.ollama extra_hosts: - "host.docker.internal:host-gateway" ports: - 0.0.0.0:3000:8080 environment: OPENAI_API_BASE_URL: http://192.168.28.174/localai/v1 deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu] ``` Adjust this and the Caddyfile accordingly, then run. When you open the UI, there will be no available models to chose from: ![grafik](https://github.com/open-webui/open-webui/assets/144339473/d2f2ec5a-f4d0-4efb-9951-c9916ae9ca43) **Expected Behavior:** I expected to be able to find and see models to chat with or run RAG. **Actual Behavior:** An empty list while the API clearly reports them all. ## Environment - **Open WebUI Version:** `ghcr.io/open-webui/open-webui:ollama` (Chosen, so I can potentially utilize both LocalAI and Ollama models situationally.) - **Ollama (if applicable):** As embedded in the container. - **Operating System:** Debian Bookworm, amd64 - **Browser (if applicable):** Firefox ## Reproduction Details **Confirmation:** - [x] I have read and followed all the instructions provided in the README.md. - [x] I am on the latest version of both Open WebUI and Ollama. - [x] I have included the browser console logs. - [X] I have included the Docker container logs. ## Logs and Screenshots **Browser Console Logs:** ![grafik](https://github.com/open-webui/open-webui/assets/144339473/bbcc2286-955c-497b-b2a6-99d96d22dc59) **Docker Container Logs:** ``` openwebui_1 | user f54c42d4-6a26-454c-be87-ce3967387692 disconnected with session ID F2SE_IH1K53atWxoAAAL openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO: 192.168.30.195:0 - "GET / HTTP/1.1" 304 Not Modified openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO: 192.168.30.195:0 - "GET /api/config HTTP/1.1" 200 OK openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO: 192.168.30.195:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=O_n-lDW HTTP/1.1" 200 OK openwebui_1 | INFO: 192.168.30.195:0 - "GET /api/v1/auths/ HTTP/1.1" 200 OK openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | connect NXQo8FR0wgUyiQ36AAAN openwebui_1 | user Kevin Ingwersen(f54c42d4-6a26-454c-be87-ce3967387692) connected with session ID NXQo8FR0wgUyiQ36AAAN openwebui_1 | 1 openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO: ('192.168.30.195', 0) - "WebSocket /ws/socket.io/?EIO=4&transport=websocket&sid=AHnf-DBGXoLh5XabAAAM" [accepted] openwebui_1 | INFO: connection open openwebui_1 | Models in use: [] openwebui_1 | INFO: 192.168.30.195:0 - "POST /ws/socket.io/?EIO=4&transport=polling&t=O_n-lD_&sid=AHnf-DBGXoLh5XabAAAM HTTP/1.1" 200 OK openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO: 192.168.30.195:0 - "GET /ws/socket.io/?EIO=4&transport=polling&t=O_n-lE0&sid=AHnf-DBGXoLh5XabAAAM HTTP/1.1" 200 OK openwebui_1 | INFO: 192.168.30.195:0 - "GET /api/changelog HTTP/1.1" 200 OK openwebui_1 | INFO: 192.168.30.195:0 - "GET /api/v1/users/user/settings HTTP/1.1" 200 OK openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO: 192.168.30.195:0 - "GET /api/v1/prompts/ HTTP/1.1" 200 OK openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO: 192.168.30.195:0 - "GET /api/v1/configs/banners HTTP/1.1" 200 OK openwebui_1 | INFO: 192.168.30.195:0 - "GET /api/models HTTP/1.1" 200 OK openwebui_1 | INFO: 192.168.30.195:0 - "GET /api/v1/documents/ HTTP/1.1" 200 OK openwebui_1 | INFO: 192.168.30.195:0 - "GET /api/v1/chats/tags/all HTTP/1.1" 200 OK openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO:apps.ollama.main:get_all_models() openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO: 192.168.30.195:0 - "GET /ollama/api/version HTTP/1.1" 200 OK openwebui_1 | INFO: 192.168.30.195:0 - "GET /api/v1/chats/ HTTP/1.1" 200 OK openwebui_1 | INFO: 192.168.30.195:0 - "GET /api/v1/users/user/settings HTTP/1.1" 200 OK ``` This happens during the connection test: ``` openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO:config:Saving 'OPENAI_API_BASE_URLS' to config.json openwebui_1 | INFO: 192.168.30.195:0 - "POST /openai/urls/update HTTP/1.1" 200 OK openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO: 192.168.30.195:0 - "POST /openai/keys/update HTTP/1.1" 200 OK openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO:apps.openai.main:get_all_models() localai_1 | 9:16AM INF Success ip=172.18.0.1 latency="309.172µs" method=GET status=200 url=/v1/models openwebui_1 | INFO: 192.168.30.195:0 - "GET /openai/models/0 HTTP/1.1" 200 OK openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO:apps.openai.main:get_all_models() openwebui_1 | INFO: 192.168.30.195:0 - "GET /api/models HTTP/1.1" 200 OK ``` **Screenshots (if applicable):** See above. ## Installation Method Docker Compose; the service configurations are above. ## Additional Information I am running this as a temporary setup on an old IBM server with 2x Intel Xeon, 160GB RAM and a RTX 3060. Because of the way the network is built and restricted, I would like to use Caddy to simplify access to the service. Due to the abundance of RAM, it'd be neat to take advantage of it at some point - which is why I use both LocalAI and Ollama; let one manage the GPU and the other the CPU. That is the plan, at least. ## Note If the bug report is incomplete or does not follow the provided instructions, it may not be addressed. Please ensure that you have followed the steps outlined in the README.md and troubleshooting.md documents, and provide all necessary information for us to reproduce and address the issue. Thank you!
GiteaMirror commented 2025-11-11 14:39:08 -06:00
Author
Owner
Copy Link
Copy Source

@senpro-ingwersenk commented on GitHub (Jun 7, 2024):

I also tried with the non-ollama embedded model; same result, no models show up in the list.

@senpro-ingwersenk commented on GitHub (Jun 7, 2024): I also tried with the non-ollama embedded model; same result, no models show up in the list.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
GiteaMirror ninjasurge (Kaehlen Rog)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#1166