Originally created by @av on GitHub (Aug 4, 2024). Original GitHub issue: https://github.com/open-webui/open-webui/issues/4348
Bug Summary: Open WebUI doesn't load models from OpenAI-compatible backends that do not require an API key
Steps to Reproduce:
{ "openai": { "api_base_urls": [ "http://litellm:4000/v1" ], "api_keys": [ "" ], "enabled": true } }
Expected Behavior: GET /api/models - loads models from the endpoint
GET /api/models
Actual Behavior: GET /api/models - responds with an empty list
Confirmation:
Note, that adjusting the config to:
{ "openai": { "api_base_urls": [ "http://litellm:4000/v1" ], "api_keys": [ "sk-fake-api-key" ], "enabled": true } }
And then the model list is populated correctly
Browser Console Logs:
Backend config: {status: true, name: 'Harbor (Open WebUI)', version: '0.3.10', default_locale: '', default_models: null, …} 0.04e6f1aa.js:1 user-count {count: 1} 0.04e6f1aa.js:1 usage {models: Array(0)}models: [][[Prototype]]: Object
Docker Container Logs:
harbor.webui | INFO:config:GLOBAL_LOG_LEVEL: DEBUG harbor.webui | INFO:config:AUDIO_LOG_LEVEL: DEBUG harbor.webui | INFO:config:COMFYUI_LOG_LEVEL: DEBUG harbor.webui | INFO:config:CONFIG_LOG_LEVEL: DEBUG harbor.webui | INFO:config:DB_LOG_LEVEL: DEBUG harbor.webui | INFO:config:IMAGES_LOG_LEVEL: DEBUG harbor.webui | INFO:config:MAIN_LOG_LEVEL: DEBUG harbor.webui | INFO:config:MODELS_LOG_LEVEL: DEBUG harbor.webui | INFO:config:OLLAMA_LOG_LEVEL: DEBUG harbor.webui | INFO:config:OPENAI_LOG_LEVEL: DEBUG harbor.webui | INFO:config:RAG_LOG_LEVEL: DEBUG harbor.webui | INFO:config:WEBHOOK_LOG_LEVEL: DEBUG harbor.webui | INFO:config:'OPENAI_API_KEYS' loaded from config.json harbor.webui | INFO:config:'OPENAI_API_BASE_URLS' loaded from config.json harbor.webui | INFO:config:Embedding model set: sentence-transformers/all-MiniLM-L6-v2 harbor.webui | DEBUG:chromadb.config:Starting component System harbor.webui | DEBUG:chromadb.config:Starting component Posthog harbor.webui | DEBUG:chromadb.config:Starting component OpenTelemetryClient harbor.webui | DEBUG:chromadb.config:Starting component SqliteDB harbor.webui | DEBUG:chromadb.config:Starting component QuotaEnforcer harbor.webui | DEBUG:chromadb.config:Starting component LocalSegmentManager harbor.webui | DEBUG:chromadb.config:Starting component SegmentAPI harbor.webui | INFO:apps.webui.internal.wrappers:Connected to SQLite database harbor.webui | INFO:apps.webui.internal.db:Starting migrations harbor.webui | DEBUG:peewee:('CREATE TABLE IF NOT EXISTS "migratehistory" ("id" INTEGER NOT NULL PRIMARY KEY, "name" VARCHAR(255) NOT NULL, "migrated_at" DATETIME NOT NULL)', []) harbor.webui | DEBUG:peewee:('SELECT "t1"."id", "t1"."name", "t1"."migrated_at" FROM "migratehistory" AS "t1" ORDER BY "t1"."id"', []) harbor.webui | INFO:apps.webui.internal.db:There is nothing to migrate harbor.webui | DEBUG:passlib.utils.compat:loaded lazy attr 'SafeConfigParser': <class 'configparser.ConfigParser'> harbor.webui | DEBUG:passlib.utils.compat:loaded lazy attr 'NativeStringIO': <class '_io.StringIO'> harbor.webui | DEBUG:passlib.utils.compat:loaded lazy attr 'BytesIO': <class '_io.BytesIO'> harbor.webui | INFO:apps.audio.main:whisper_device_type: cpu harbor.webui | WARNING:langchain_community.utils.user_agent:USER_AGENT environment variable not set, consider setting it to identify your requests. harbor.webui | DEBUG:apps.rag.utils:model: sentence-transformers/all-MiniLM-L6-v2 harbor.webui | DEBUG:apps.rag.utils:snapshot_kwargs: {'cache_dir': '/app/backend/data/cache/embedding/models', 'local_files_only': True} harbor.webui | DEBUG:apps.rag.utils:model_repo_path: /app/backend/data/cache/embedding/models/models--sentence-transformers--all-MiniLM-L6-v2/snapshots/8b3219a92973c328a8e22fadcfa821b5dc75636a harbor.webui | INFO:sentence_transformers.SentenceTransformer:Load pretrained SentenceTransformer: /app/backend/data/cache/embedding/models/models--sentence-transformers--all-MiniLM-L6-v2/snapshots/8b3219a92973c328a8e22fadcfa821b5dc75636a harbor.webui | INFO: Started server process [10] harbor.webui | INFO: Waiting for application startup. harbor.webui | harbor.webui | ___ __ __ _ _ _ ___ harbor.webui | / _ \ _ __ ___ _ __ \ \ / /__| |__ | | | |_ _| harbor.webui | | | | | '_ \ / _ \ '_ \ \ \ /\ / / _ \ '_ \| | | || | harbor.webui | | |_| | |_) | __/ | | | \ V V / __/ |_) | |_| || | harbor.webui | \___/| .__/ \___|_| |_| \_/\_/ \___|_.__/ \___/|___| harbor.webui | |_| harbor.webui | harbor.webui | harbor.webui | v0.3.10 - building the best open-source AI user interface. harbor.webui | harbor.webui | https://github.com/open-webui/open-webui harbor.webui | harbor.webui | INFO [alembic.runtime.migration] Context impl SQLiteImpl. harbor.webui | INFO [alembic.runtime.migration] Will assume non-transactional DDL. harbor.webui | INFO: Application startup complete. harbor.webui | INFO: Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit) harbor.webui | INFO [apps.openai.main] get_all_models() harbor.webui | INFO [apps.ollama.main] get_all_models() harbor.webui | ERROR [apps.ollama.main] Connection error: Cannot connect to host host.docker.internal:11434 ssl:default [Name or service not known] harbor.webui | DEBUG [main] Commit session after request harbor.webui | INFO: 172.31.0.1:34460 - "GET /ws/socket.io/?EIO=4&transport=polling&t=P4TUaWv HTTP/1.1" 200 OK harbor.webui | INFO [apps.openai.main] get_all_models() harbor.webui | INFO [apps.ollama.main] get_all_models() harbor.webui | INFO: ('172.31.0.1', 34470) - "WebSocket /ws/socket.io/?EIO=4&transport=websocket&sid=WW9mrSVpYo33cNimAAAA" [accepted] harbor.webui | INFO: connection open harbor.webui | INFO [apps.openai.main] get_all_models() harbor.webui | INFO [apps.ollama.main] get_all_models() harbor.webui | ERROR [apps.ollama.main] Connection error: Cannot connect to host host.docker.internal:11434 ssl:default [Name or service not known] harbor.webui | ERROR [apps.ollama.main] Connection error: Cannot connect to host host.docker.internal:11434 ssl:default [Name or service not known] harbor.webui | DEBUG [main] Commit session after request harbor.webui | user av(d4e2487c-acc8-4474-a682-d683b27dc2af) connected with session ID XNKFzbKCaT_JakCOAAAB harbor.webui | INFO: 172.31.0.1:34472 - "GET /ws/socket.io/?EIO=4&transport=polling&t=P4TUaY2&sid=WW9mrSVpYo33cNimAAAA HTTP/1.1" 200 OK harbor.webui | DEBUG [main] Commit session after request harbor.webui | INFO: 172.31.0.1:34460 - "POST /ws/socket.io/?EIO=4&transport=polling&t=P4TUaY1&sid=WW9mrSVpYo33cNimAAAA HTTP/1.1" 200 OK
Docker Compose installation
@tjbck commented on GitHub (Aug 4, 2024):
This is an expected behaviour!
@av commented on GitHub (Aug 4, 2024):
Understood, thank you for taking a look!
No dependencies set.
The note is not visible to the blocked user.
Originally created by @av on GitHub (Aug 4, 2024).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/4348
Bug Report
Description
Bug Summary:
Open WebUI doesn't load models from OpenAI-compatible backends that do not require an API key
Steps to Reproduce:
Expected Behavior:
GET /api/models- loads models from the endpointActual Behavior:
GET /api/models- responds with an empty listEnvironment
Reproduction Details
Confirmation:
Logs and Screenshots
Note, that adjusting the config to:
And then the model list is populated correctly
Browser Console Logs:
Docker Container Logs:
Details
Installation Method
Docker Compose installation
@tjbck commented on GitHub (Aug 4, 2024):
This is an expected behaviour!
@av commented on GitHub (Aug 4, 2024):
Understood, thank you for taking a look!