github-starred/open-webui
2
0
Fork 0
mirror of https://github.com/open-webui/open-webui.git synced 2026-07-17 08:21:12 -05:00
Code Issues 2.4k Packages Projects Releases 129 Wiki Activity

[GH-ISSUE #24115] feat: GPU Cloud sidebar tab — live instance monitoring for Modal, Salad, and Fal.ai #35723

New Issue
Open
opened 2026-04-25 09:53:54 -05:00 by GiteaMirror · 7 comments
GiteaMirror commented 2026-04-25 09:53:54 -05:00
Owner
Copy Link

Originally created by @githb-ac on GitHub (Apr 24, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/24115

Check Existing Issues

  • I have searched for all existing open AND closed issues and discussions for similar requests. I have found none that is comparable to my request.

Verify Feature Scope

  • I have read through and understood the scope definition for feature requests in the Issues section. I believe my feature request meets the definition and belongs in the Issues section instead of the Discussions.

Problem Description

Users running Open WebUI with GPU-backed inference have no visibility into their
GPU resources from within the interface. There is currently no way to see which
GPU instances are active, their utilisation, cost per hour, or total spend —
without switching to each provider's separate dashboard.

This ticket covers three leading serverless GPU providers that Open WebUI users
are most likely to be running:

  • Modal (https://modal.com/) — serverless GPU and compute platform with a native
    Python SDK and a first-party MCP server, popular for running LLM inference,
    fine-tuning, and batch workloads on-demand.

  • Salad (https://salad.com/) — distributed cloud GPU network offering some of
    the lowest-cost GPU compute available, widely used for inference workloads
    including image generation and LLM serving.

  • Fal.ai (https://fal.ai/pricing) — serverless inference platform that is
    particularly well known in the AI community for its image generation API
    endpoints (Flux, SDXL, and others). Fal.ai is widely used by LLM agent users —
    including Claude, Hermes Agent, and Open WebUI pipelines — to connect image
    generation capabilities via API. It is now expanding into general-purpose
    serverless GPU compute, making it directly comparable to Modal and Salad for
    agent workloads.

Open WebUI already tracks API costs and VPS health, but GPU infrastructure —
the most expensive and time-sensitive part of many setups — is completely
invisible inside the interface. Users must log into three separate provider
dashboards to get a complete picture of their running costs and active instances.

Desired Solution you'd like

Add a GPU Cloud sidebar tab with live monitoring and management for GPU cloud
providers, starting with Modal, Salad, and Fal.ai.

Provider selector and connection panel:

  • Dropdown / tab selector: Modal / Salad / Fal.ai (extensible to future providers)
  • API key input per provider (stored in settings, masked)
  • Connection status indicator per provider

Live instance metrics per active GPU:

  • GPU model (A100, H100, RTX 4090 etc)
  • VRAM used / total
  • GPU utilisation percentage
  • Cost per hour
  • Total spend today and this month

Instance management:

  • Start / stop / restart instances
  • Spin up new instance with model and region selector
  • View per-instance logs

Provider-specific panels:

Modal (https://modal.com/):

  • Active function runs, queue depth, cold start times, per-function cost
  • Note: Modal ships a native MCP server — if Open WebUI MCP integration is
    available, Modal requires no custom API work at all

Fal.ai (https://fal.ai/pricing):

  • Active inference endpoints, request throughput, queue status
  • Image generation endpoint health (Flux, SDXL, etc)

Salad (https://salad.com/):

  • Container group status, node health, replica count

API references:

Settings required:

  • gpu_provider — active provider
  • modal_api_key, salad_api_key, fal_api_key
  • gpu_tab_enabled — toggle to show/hide the tab

Alternatives Considered

Currently users must log into each provider dashboard separately to monitor GPU
usage. There is no unified cross-provider GPU view anywhere in Open WebUI, and
no way to correlate GPU costs with agent activity or API spend visible in the
same interface.

Additional Context

Modal: https://modal.com/
Modal MCP: https://github.com/modal-labs/modal-client
Modal API docs: https://modal.com/docs

Salad: https://salad.com/
Salad API docs: https://docs.salad.com/reference/saladcloud-api

Fal.ai: https://fal.ai/pricing
Fal.ai docs: https://fal.ai/docs

Attached dashboard shows the proposed GPU Cloud tab layout including the
provider selector, KPI cards, and live instance table.

owui-aegis-app-data-v5.html

Originally created by @githb-ac on GitHub (Apr 24, 2026). Original GitHub issue: https://github.com/open-webui/open-webui/issues/24115 ### Check Existing Issues - [x] I have searched for all existing **open AND closed** issues and discussions for similar requests. I have found none that is comparable to my request. ### Verify Feature Scope - [x] I have read through and understood the scope definition for feature requests in the Issues section. I believe my feature request meets the definition and belongs in the Issues section instead of the Discussions. ### Problem Description Users running Open WebUI with GPU-backed inference have no visibility into their GPU resources from within the interface. There is currently no way to see which GPU instances are active, their utilisation, cost per hour, or total spend — without switching to each provider's separate dashboard. This ticket covers three leading serverless GPU providers that Open WebUI users are most likely to be running: - Modal (https://modal.com/) — serverless GPU and compute platform with a native Python SDK and a first-party MCP server, popular for running LLM inference, fine-tuning, and batch workloads on-demand. - Salad (https://salad.com/) — distributed cloud GPU network offering some of the lowest-cost GPU compute available, widely used for inference workloads including image generation and LLM serving. - Fal.ai (https://fal.ai/pricing) — serverless inference platform that is particularly well known in the AI community for its image generation API endpoints (Flux, SDXL, and others). Fal.ai is widely used by LLM agent users — including Claude, Hermes Agent, and Open WebUI pipelines — to connect image generation capabilities via API. It is now expanding into general-purpose serverless GPU compute, making it directly comparable to Modal and Salad for agent workloads. Open WebUI already tracks API costs and VPS health, but GPU infrastructure — the most expensive and time-sensitive part of many setups — is completely invisible inside the interface. Users must log into three separate provider dashboards to get a complete picture of their running costs and active instances. ### Desired Solution you'd like Add a GPU Cloud sidebar tab with live monitoring and management for GPU cloud providers, starting with Modal, Salad, and Fal.ai. Provider selector and connection panel: - Dropdown / tab selector: Modal / Salad / Fal.ai (extensible to future providers) - API key input per provider (stored in settings, masked) - Connection status indicator per provider Live instance metrics per active GPU: - GPU model (A100, H100, RTX 4090 etc) - VRAM used / total - GPU utilisation percentage - Cost per hour - Total spend today and this month Instance management: - Start / stop / restart instances - Spin up new instance with model and region selector - View per-instance logs Provider-specific panels: Modal (https://modal.com/): - Active function runs, queue depth, cold start times, per-function cost - Note: Modal ships a native MCP server — if Open WebUI MCP integration is available, Modal requires no custom API work at all Fal.ai (https://fal.ai/pricing): - Active inference endpoints, request throughput, queue status - Image generation endpoint health (Flux, SDXL, etc) Salad (https://salad.com/): - Container group status, node health, replica count API references: - Modal: native MCP server preferred; fallback to Modal REST API (https://modal.com/docs) - Salad: https://docs.salad.com/reference/saladcloud-api - Fal.ai: https://fal.ai/docs Settings required: - gpu_provider — active provider - modal_api_key, salad_api_key, fal_api_key - gpu_tab_enabled — toggle to show/hide the tab ### Alternatives Considered Currently users must log into each provider dashboard separately to monitor GPU usage. There is no unified cross-provider GPU view anywhere in Open WebUI, and no way to correlate GPU costs with agent activity or API spend visible in the same interface. ### Additional Context Modal: https://modal.com/ Modal MCP: https://github.com/modal-labs/modal-client Modal API docs: https://modal.com/docs Salad: https://salad.com/ Salad API docs: https://docs.salad.com/reference/saladcloud-api Fal.ai: https://fal.ai/pricing Fal.ai docs: https://fal.ai/docs Attached dashboard shows the proposed GPU Cloud tab layout including the provider selector, KPI cards, and live instance table. [owui-aegis-app-data-v5.html](https://github.com/user-attachments/files/27069231/owui-aegis-app-data-v5.html)
GiteaMirror commented 2026-04-25 09:53:55 -05:00
Author
Owner
Copy Link

@Classic298 commented on GitHub (Apr 25, 2026):

Open WebUI already tracks API costs and VPS health, but GPU infrastructure —
the most expensive and time-sensitive part of many setups — is completely
invisible inside the interface. Users must log into three separate provider
dashboards to get a complete picture of their running costs and active instances.

Completely false besides you can use a filter or action described to do what you want to do

<!-- gh-comment-id:4318573696 --> @Classic298 commented on GitHub (Apr 25, 2026): > Open WebUI already tracks API costs and VPS health, but GPU infrastructure — the most expensive and time-sensitive part of many setups — is completely invisible inside the interface. Users must log into three separate provider dashboards to get a complete picture of their running costs and active instances. Completely false besides you can use a filter or action described to do what you want to do
GiteaMirror commented 2026-04-25 09:53:55 -05:00
Author
Owner
Copy Link

@githb-ac commented on GitHub (Apr 25, 2026):

Thanks for the response — but I'd respectfully push back. Filters and Actions in Open WebUI extend what the LLM can do inside a chat. They cannot display live GPU instance metrics — active GPU utilisation, VRAM usage, cost per hour, running container status — from external providers like Modal, Salad, and Fal.ai.

What's being requested is a dedicated monitoring panel that connects to provider APIs and surfaces live infrastructure data inside the Open WebUI interface. That is a UI feature, not something a Tool or Action can replicate. There is no mechanism in Open WebUI today that lets you view live GPU instance data from any provider without leaving the interface entirely.

Would appreciate reconsideration of this one.

<!-- gh-comment-id:4318618793 --> @githb-ac commented on GitHub (Apr 25, 2026): Thanks for the response — but I'd respectfully push back. Filters and Actions in Open WebUI extend what the LLM can do inside a chat. They cannot display live GPU instance metrics — active GPU utilisation, VRAM usage, cost per hour, running container status — from external providers like Modal, Salad, and Fal.ai. What's being requested is a dedicated monitoring panel that connects to provider APIs and surfaces live infrastructure data inside the Open WebUI interface. That is a UI feature, not something a Tool or Action can replicate. There is no mechanism in Open WebUI today that lets you view live GPU instance data from any provider without leaving the interface entirely. Would appreciate reconsideration of this one.
GiteaMirror commented 2026-04-25 09:53:56 -05:00
Author
Owner
Copy Link

@Classic298 commented on GitHub (Apr 25, 2026):

Yes plugins can do this see docs and my comment on the other issue

<!-- gh-comment-id:4319266471 --> @Classic298 commented on GitHub (Apr 25, 2026): Yes plugins can do this see docs and my comment on the other issue
GiteaMirror commented 2026-04-25 09:53:56 -05:00
Author
Owner
Copy Link

@githb-ac commented on GitHub (Apr 25, 2026):

Plugins in Open WebUI are Python functions that run in the chat pipeline — they can call external APIs and return text or structured data into a chat message. They cannot render a persistent sidebar panel with live updating GPU metrics, instance management controls, start/stop buttons, and per-provider tabs that exist independently of any chat session.

What's being requested is a first-class UI panel — the same way the existing VPS health tab works. That requires a frontend component, not a plugin. A plugin that prints GPU stats into a chat message is a fundamentally different thing to a monitoring dashboard. Could you point to a specific plugin or documentation that achieves what's described here? If one exists I'd be genuinely happy to use it.

<!-- gh-comment-id:4319506583 --> @githb-ac commented on GitHub (Apr 25, 2026): Plugins in Open WebUI are Python functions that run in the chat pipeline — they can call external APIs and return text or structured data into a chat message. They cannot render a persistent sidebar panel with live updating GPU metrics, instance management controls, start/stop buttons, and per-provider tabs that exist independently of any chat session. What's being requested is a first-class UI panel — the same way the existing VPS health tab works. That requires a frontend component, not a plugin. A plugin that prints GPU stats into a chat message is a fundamentally different thing to a monitoring dashboard. Could you point to a specific plugin or documentation that achieves what's described here? If one exists I'd be genuinely happy to use it.
GiteaMirror commented 2026-04-25 09:53:56 -05:00
Author
Owner
Copy Link

@Classic298 commented on GitHub (Apr 25, 2026):

Yes they can.

And there are different types of plugins also

But yes they CAN render persistent sidebars in the chat.

<!-- gh-comment-id:4319512916 --> @Classic298 commented on GitHub (Apr 25, 2026): Yes they can. And there are different types of plugins also But yes they CAN render persistent sidebars in the chat.
GiteaMirror commented 2026-04-25 09:53:57 -05:00
Author
Owner
Copy Link

@githb-ac commented on GitHub (Apr 25, 2026):

Plugins in Open WebUI are Python functions that run in the chat pipeline — they can call external APIs and return text or structured data into a chat message. They cannot render a persistent sidebar panel with live updating GPU metrics, instance management controls, start/stop buttons, and per-provider tabs that exist independently of any chat session.

What's being requested is a first-class UI panel — the same way the existing VPS health tab works. That requires a frontend component, not a plugin. A plugin that prints GPU stats into a chat message is a fundamentally different thing to a monitoring dashboard. Could you point to a specific plugin or documentation that achieves what's described here? If one exists I'd be genuinely happy to use it.

<!-- gh-comment-id:4319540645 --> @githb-ac commented on GitHub (Apr 25, 2026): Plugins in Open WebUI are Python functions that run in the chat pipeline — they can call external APIs and return text or structured data into a chat message. They cannot render a persistent sidebar panel with live updating GPU metrics, instance management controls, start/stop buttons, and per-provider tabs that exist independently of any chat session. What's being requested is a first-class UI panel — the same way the existing VPS health tab works. That requires a frontend component, not a plugin. A plugin that prints GPU stats into a chat message is a fundamentally different thing to a monitoring dashboard. Could you point to a specific plugin or documentation that achieves what's described here? If one exists I'd be genuinely happy to use it.
GiteaMirror commented 2026-04-25 09:53:57 -05:00
Author
Owner
Copy Link

@Classic298 commented on GitHub (Apr 25, 2026):

Yes they can render persistent sidebar

<!-- gh-comment-id:4319549293 --> @Classic298 commented on GitHub (Apr 25, 2026): Yes they can render persistent sidebar
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
confirmed
confirmed issue
core
documentation
enhancement
good first issue
help wanted
non-core
pull-request
Mirrored from GitHub Pull Request
 python
question
testing wanted
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#35723
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?