[PR #23752] [CLOSED] perf(mcp): pool bearer-auth connections and emit progress status events #42984

New Issue

GiteaMirror · 2026-04-25T14:43:40-05:00

GiteaMirror commented

2026-04-25 14:43:40 -05:00

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/23752
Author: @DSavaliya-gh
Created: 4/15/2026
Status: ❌ Closed

Base: dev ← Head: fix/mcp-connection-pool-status-events

📝 Commits (1)

9a466c2 perf(mcp): pool bearer-auth connections and emit status events

📊 Changes

1 file changed (+54 additions, -5 deletions)

View changed files

📝 backend/open_webui/utils/middleware.py (+54 -5)

📄 Description

Summary

Fixes #23751 — MCP tool servers reconnect from scratch on every chat message with zero user feedback, causing a 15–20 second silent blinking-dot phase before the model starts.

What Changed

One file modified: backend/open_webui/utils/middleware.py

Before

mcp_clients[server_id] = MCPClient()          # fresh every message
await mcp_clients[server_id].connect(...)     # full TCP+TLS+HTTP+MCP handshake
tool_specs = await mcp_clients[server_id].list_tool_specs()  # list_tools round-trip

No status event. User sees blinking dot for 15-20 s with zero feedback.

After

1 — Progress status events

await event_emitter({"type": "status", "data": {
    "action": "mcp_connect",
    "description": f"Connecting to '{server_id}'...",
    "done": False,
}})
# ... connect ...
await event_emitter({"type": "status", "data": {
    "action": "mcp_connect",
    "description": f"Connected to '{server_id}'",
    "done": True,
}})

The existing StatusHistory component renders these immediately — same UX as web search status.

2 — Connection pool for static bearer-auth servers

pool = getattr(request.app.state, 'mcp_client_pool', {})
request.app.state.mcp_client_pool = pool

if auth_type == 'bearer' and server_id in pool and pool[server_id].session:
    mcp_clients[server_id] = pool[server_id]   # reuse
else:
    mcp_clients[server_id] = MCPClient()
    await mcp_clients[server_id].connect(...)
    if auth_type == 'bearer':
        pool[server_id] = mcp_clients[server_id]

Per-user auth types (session, oauth_2.1) are never pooled — they always get a fresh connection to preserve per-user token isolation. Stale pool entries are evicted on any connection error.

Pull Request Checklist

Target branch: dev
Description: Provided above
Testing: Manually verified — status events render in StatusHistory; warm connections skip the handshake
No AI Agent: Changes written by human author with manual review and testing
Code review: Self-reviewed for correctness and adherence to existing patterns
Design & Architecture: Uses existing app.state pattern (same as TOOL_SERVERS); no new settings added
Git Hygiene: Single atomic commit, rebased on dev
Documentation: No user-facing behaviour change beyond a visible status indicator; no env var added

Changelog Entry

Description

Fixed a 15–20 second silent "blinking dot" delay that appeared before every chat message when MCP tool servers were configured. The fix adds visible progress status events and introduces a connection pool for bearer-auth MCP servers so the full TCP/HTTP/MCP handshake is not repeated on every message.

Fixed

MCP tool server connection is now reused across messages for static bearer-auth servers instead of reconnecting from scratch on every chat message

Changed

MCP server connect/disconnect now emits type: 'status' events (action: 'mcp_connect') so the user sees activity in the StatusHistory component during any connection wait

Additional Information

Fixes [PR #15195] [MERGED] [PERF] Proper default pooling settings (#23751)
The mcp_client_pool key on app.state follows the same pattern as app.state.TOOL_SERVERS used in utils/tools.py
Pool is process-scoped (single-instance); multi-replica deployments behind a load balancer each maintain their own pools, which is correct behaviour

By submitting this pull request, I confirm that my contribution is made under the terms of the MIT license, and I agree to the Contributor License Agreement.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/23752 **Author:** [@DSavaliya-gh](https://github.com/DSavaliya-gh) **Created:** 4/15/2026 **Status:** ❌ Closed **Base:** `dev` ← **Head:** `fix/mcp-connection-pool-status-events` --- ### 📝 Commits (1) - [`9a466c2`](https://github.com/open-webui/open-webui/commit/9a466c2eeb4b5160ec9871c1587735f23ff4000f) perf(mcp): pool bearer-auth connections and emit status events ### 📊 Changes **1 file changed** (+54 additions, -5 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/utils/middleware.py` (+54 -5) </details> ### 📄 Description  ## Summary Fixes **#23751** — MCP tool servers reconnect from scratch on every chat message with zero user feedback, causing a **15–20 second silent blinking-dot phase** before the model starts. --- ## What Changed One file modified: `backend/open_webui/utils/middleware.py` ### Before ```python mcp_clients[server_id] = MCPClient() # fresh every message await mcp_clients[server_id].connect(...) # full TCP+TLS+HTTP+MCP handshake tool_specs = await mcp_clients[server_id].list_tool_specs() # list_tools round-trip ``` No status event. User sees blinking dot for 15-20 s with zero feedback. ### After **1 — Progress status events** ```python await event_emitter({"type": "status", "data": { "action": "mcp_connect", "description": f"Connecting to '{server_id}'...", "done": False, }}) # ... connect ... await event_emitter({"type": "status", "data": { "action": "mcp_connect", "description": f"Connected to '{server_id}'", "done": True, }}) ``` The existing `StatusHistory` component renders these immediately — same UX as web search status. **2 — Connection pool for static bearer-auth servers** ```python pool = getattr(request.app.state, 'mcp_client_pool', {}) request.app.state.mcp_client_pool = pool if auth_type == 'bearer' and server_id in pool and pool[server_id].session: mcp_clients[server_id] = pool[server_id] # reuse else: mcp_clients[server_id] = MCPClient() await mcp_clients[server_id].connect(...) if auth_type == 'bearer': pool[server_id] = mcp_clients[server_id] ``` Per-user auth types (`session`, `oauth_2.1`) are **never** pooled — they always get a fresh connection to preserve per-user token isolation. Stale pool entries are evicted on any connection error. --- ## Pull Request Checklist - [x] **Target branch:** `dev` - [x] **Description:** Provided above - [x] **Testing:** Manually verified — status events render in `StatusHistory`; warm connections skip the handshake - [x] **No AI Agent:** Changes written by human author with manual review and testing - [x] **Code review:** Self-reviewed for correctness and adherence to existing patterns - [x] **Design & Architecture:** Uses existing `app.state` pattern (same as `TOOL_SERVERS`); no new settings added - [x] **Git Hygiene:** Single atomic commit, rebased on `dev` - [ ] **Documentation:** No user-facing behaviour change beyond a visible status indicator; no env var added --- # Changelog Entry ### Description Fixed a 15–20 second silent "blinking dot" delay that appeared before every chat message when MCP tool servers were configured. The fix adds visible progress status events and introduces a connection pool for bearer-auth MCP servers so the full TCP/HTTP/MCP handshake is not repeated on every message. ### Fixed - MCP tool server connection is now reused across messages for static bearer-auth servers instead of reconnecting from scratch on every chat message ### Changed - MCP server connect/disconnect now emits `type: 'status'` events (`action: 'mcp_connect'`) so the user sees activity in the `StatusHistory` component during any connection wait --- ### Additional Information - Fixes #23751 - The `mcp_client_pool` key on `app.state` follows the same pattern as `app.state.TOOL_SERVERS` used in `utils/tools.py` - Pool is process-scoped (single-instance); multi-replica deployments behind a load balancer each maintain their own pools, which is correct behaviour --- By submitting this pull request, I confirm that my contribution is made under the terms of the MIT license, and I agree to the [Contributor License Agreement](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-04-25 14:43:40 -05:00

GiteaMirror closed this issue

2026-04-25 14:43:40 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#42984