[PR #24547] fix: llamacpp load/unload indicator now detects loaded models via /slots #98756

New Issue

GiteaMirror · 2026-05-16T01:36:09-05:00

GiteaMirror commented

2026-05-16 01:36:09 -05:00

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/24547
Author: @kimjune01
Created: 5/10/2026
Status: 🔄 Open

Base: dev ← Head: fix-llamacpp-load-indicator-24544-clean

📝 Commits (1)

6ec42e7 fix: llamacpp load/unload indicator now detects loaded models via /slots

📊 Changes

1 file changed (+75 additions, -2 deletions)

View changed files

📝 backend/open_webui/utils/models.py (+75 -2)

📄 Description

The current load indicator treats a model as loaded only when /api/tags includes expires_at, which is standard Ollama behavior. llama.cpp-compatible servers can keep models loaded in /slots, but their /api/tags response does not include expires_at, so open-webui shows those models as unloaded.

This change adds per-backend detection for /slots support. During model refresh, open-webui fetches /api/tags and, unless the backend is already known not to support it, probes /slots concurrently. A 200 response caches that backend as supporting /slots; a 404 caches it as not supporting /slots; transient failures such as timeouts or 5xx responses are logged but not cached.

The load indicator now treats a model as loaded when either:

/api/tags includes expires_at
the same backend's /slots response contains that model ID

For llama.cpp --models-preset, this matches preset names directly when the same identifier appears in both /api/tags and /slots. When prefix_id is configured, both raw and prefixed model IDs are checked.

This preserves Ollama behavior while correctly showing llama.cpp slot-loaded models as loaded, without repeatedly probing /slots on backends that return 404.

Fixes #24544.

Contributor License Agreement

By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/24547 **Author:** [@kimjune01](https://github.com/kimjune01) **Created:** 5/10/2026 **Status:** 🔄 Open **Base:** `dev` ← **Head:** `fix-llamacpp-load-indicator-24544-clean` --- ### 📝 Commits (1) - [`6ec42e7`](https://github.com/open-webui/open-webui/commit/6ec42e7e9d67c49b1304d34e0689f2657deac2ff) fix: llamacpp load/unload indicator now detects loaded models via /slots ### 📊 Changes **1 file changed** (+75 additions, -2 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/utils/models.py` (+75 -2) </details> ### 📄 Description The current load indicator treats a model as loaded only when `/api/tags` includes `expires_at`, which is standard Ollama behavior. llama.cpp-compatible servers can keep models loaded in `/slots`, but their `/api/tags` response does not include `expires_at`, so open-webui shows those models as unloaded. This change adds per-backend detection for `/slots` support. During model refresh, open-webui fetches `/api/tags` and, unless the backend is already known not to support it, probes `/slots` concurrently. A `200` response caches that backend as supporting `/slots`; a `404` caches it as not supporting `/slots`; transient failures such as timeouts or 5xx responses are logged but not cached. The load indicator now treats a model as loaded when either: 1. `/api/tags` includes `expires_at` 2. the same backend's `/slots` response contains that model ID For llama.cpp `--models-preset`, this matches preset names directly when the same identifier appears in both `/api/tags` and `/slots`. When `prefix_id` is configured, both raw and prefixed model IDs are checked. This preserves Ollama behavior while correctly showing llama.cpp slot-loaded models as loaded, without repeatedly probing `/slots` on backends that return `404`. Fixes #24544. ### Contributor License Agreement - [x] By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-05-16 01:36:09 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#98756