Note to first-time contributors: Please open a discussion post in Discussions and describe your changes before submitting a pull request.
Before submitting, make sure you've checked the following:
Target branch: Please verify that the pull request targets the dev branch.
Description: Provide a concise description of the changes made in this pull request.
Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
Documentation: Have you updated relevant documentation Open WebUI Docs, or other documentation sources?
Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
Testing: Have you written and run sufficient tests to validate the changes?
Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
Prefix:feat: Introduces a new feature or enhancement to the codebase
Changelog Entry
Description
Implements Redis-based caching for Tool Servers to support multi-pod backend deployments. Previously, tool servers were cached in memory per pod, causing inconsistencies when multiple backend pods were running. This enhancement stores tool servers in Redis for shared access across all pods, with fallback to memory cache when Redis is unavailable.
We don't really need to keep the TOOL_SERVERS_CONNECTION in memory. We can derive the information from tool_servers.
Added
Redis-based tool server caching system (open_webui/utils/tools_cache.py)
get_cached_tool_servers(): Retrieves tool servers from Redis → memory cache
set_and_cache_tool_servers(): Immediately fetches and caches tool servers when configuration is set
invalidate_tool_servers_cache(): Clears both Redis and memory cache
Self-contained tool server data with access_control and __raw properties
cahcing invalidation when tool server configuration is updated
Changed
Tool server configuration flow now immediately fetches and caches servers instead of storing connections
Tool server data structure enhanced with access_control property and __raw original connection data
Tools router simplified to only read from cache, eliminating runtime config dependencies
Configuration endpoints now retrieve connections from cached tool servers' __raw property
Configuration flow change: Tool servers are now cached immediately when configuration is set, rather than on-demand
Removed
Dependency on request.app.state.config.TOOL_SERVER_CONNECTIONS for runtime tool server access
Import of TOOL_SERVER_CONNECTIONS from main.py
Complex conditional logic in cache retrieval functions
Fixed
Multi-pod tool server inconsistency - tool servers now shared via Redis cache across all backend pods
Tool server updates not propagating between pods in multi-instance deployments
Breaking Changesn/a
Additional Information
Technical Implementation:
Redis Key: open-webui:tool_servers with 5-minute TTL
Fallback Strategy: Redis → Memory → Empty list (no more on-demand fetching)
Data Structure: Tool servers now include access_control and __raw (original connection) properties
Multi-pod Safe: All pods share the same Redis cache for consistent tool server state
Migration Path:
Existing deployments will automatically migrate to the new caching system
Single-pod deployments continue to work with memory fallback
Multi-pod deployments now properly share tool server state via Redis
Files Modified:
open_webui/utils/tools_cache.py - New Redis caching module
open_webui/utils/tools.py - Enhanced tool server data structure
Verify tool server functionality with Redis enabled/disabled
Test multi-pod deployment tool server consistency
Validate cache refresh endpoint functionality
Confirm existing tool server configurations continue working
Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/open-webui/open-webui/pull/15617
**Author:** [@signorettif](https://github.com/signorettif)
**Created:** 7/9/2025
**Status:** ❌ Closed
**Base:** `dev` ← **Head:** `fs-redis-in-memory-tools`
---
### 📝 Commits (4)
- [`8820ac1`](https://github.com/open-webui/open-webui/commit/8820ac163b4cc4f9e810b16d62f41fdb133063f5) feat: cache tool servers in redis when available
- [`3ec77bf`](https://github.com/open-webui/open-webui/commit/3ec77bfe0d178a68996882098d477ee2a9422ec2) accept initialised values from env variable
- [`91707ef`](https://github.com/open-webui/open-webui/commit/91707efc8b9b0dca5e376469123fd9236db9c4a2) do not return in-memory stuff if it is not current spec
- [`1442424`](https://github.com/open-webui/open-webui/commit/1442424bc572245d8db1966c173e38b7d593ef4d) fix circularity
### 📊 Changes
**6 files changed** (+222 additions, -111 deletions)
<details>
<summary>View changed files</summary>
📝 `backend/open_webui/functions.py` (+1 -1)
📝 `backend/open_webui/routers/configs.py` (+81 -36)
📝 `backend/open_webui/routers/tools.py` (+25 -34)
📝 `backend/open_webui/utils/middleware.py` (+1 -1)
📝 `backend/open_webui/utils/tools.py` (+22 -39)
➕ `backend/open_webui/utils/tools_cache.py` (+92 -0)
</details>
### 📄 Description
# Pull Request Checklist
### Note to first-time contributors: Please open a discussion post in [Discussions](https://github.com/open-webui/open-webui/discussions) and describe your changes before submitting a pull request.
**Before submitting, make sure you've checked the following:**
- [x] **Target branch:** Please verify that the pull request targets the `dev` branch.
- [x] **Description:** Provide a concise description of the changes made in this pull request.
- [x] **Changelog:** Ensure a changelog entry following the format of [Keep a Changelog](https://keepachangelog.com/) is added at the bottom of the PR description.
- [ ] **Documentation:** Have you updated relevant documentation [Open WebUI Docs](https://github.com/open-webui/docs), or other documentation sources?
- [x] **Dependencies:** Are there any new dependencies? Have you updated the dependency versions in the documentation?
- [ ] **Testing:** Have you written and run sufficient tests to validate the changes?
- [x] **Code review:** Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
- [x] **Prefix:** **feat**: Introduces a new feature or enhancement to the codebase
# Changelog Entry
### Description
Implements Redis-based caching for Tool Servers to support multi-pod backend deployments. Previously, tool servers were cached in memory per pod, causing inconsistencies when multiple backend pods were running. This enhancement stores tool servers in Redis for shared access across all pods, with fallback to memory cache when Redis is unavailable.
We don't really need to keep the TOOL_SERVERS_CONNECTION in memory. We can derive the information from `tool_servers`.
### Added
- **Redis-based tool server caching system** (`open_webui/utils/tools_cache.py`)
- `get_cached_tool_servers()`: Retrieves tool servers from Redis → memory cache
- `set_and_cache_tool_servers()`: Immediately fetches and caches tool servers when configuration is set
- `invalidate_tool_servers_cache()`: Clears both Redis and memory cache
- **Self-contained tool server data** with `access_control` and `__raw` properties
- **cahcing invalidation** when tool server configuration is updated
### Changed
- **Tool server configuration flow** now immediately fetches and caches servers instead of storing connections
- **Tool server data structure** enhanced with `access_control` property and `__raw` original connection data
- **Tools router** simplified to only read from cache, eliminating runtime config dependencies
- **Configuration endpoints** now retrieve connections from cached tool servers' `__raw` property
- **Configuration flow change**: Tool servers are now cached immediately when configuration is set, rather than on-demand
### Removed
- **Dependency on `request.app.state.config.TOOL_SERVER_CONNECTIONS`** for runtime tool server access
- **Import of `TOOL_SERVER_CONNECTIONS`** from main.py
- **Complex conditional logic** in cache retrieval functions
### Fixed
- **Multi-pod tool server inconsistency** - tool servers now shared via Redis cache across all backend pods
- **Tool server updates not propagating** between pods in multi-instance deployments
### Breaking Changesn/a
---
### Additional Information
**Technical Implementation:**
- **Redis Key**: `open-webui:tool_servers` with 5-minute TTL
- **Fallback Strategy**: Redis → Memory → Empty list (no more on-demand fetching)
- **Data Structure**: Tool servers now include `access_control` and `__raw` (original connection) properties
- **Multi-pod Safe**: All pods share the same Redis cache for consistent tool server state
**Migration Path:**
- Existing deployments will automatically migrate to the new caching system
- Single-pod deployments continue to work with memory fallback
- Multi-pod deployments now properly share tool server state via Redis
**Files Modified:**
- `open_webui/utils/tools_cache.py` - New Redis caching module
- `open_webui/utils/tools.py` - Enhanced tool server data structure
- `open_webui/routers/configs.py` - Updated configuration endpoints
- `open_webui/routers/tools.py` - Simplified cache-only access + refresh endpoint
- `open_webui/main.py` - Removed TOOL_SERVER_CONNECTIONS dependency
**Testing Recommendations:**
- Verify tool server functionality with Redis enabled/disabled
- Test multi-pod deployment tool server consistency
- Validate cache refresh endpoint functionality
- Confirm existing tool server configurations continue working
### Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/open-webui/open-webui/pull/15617
Author: @signorettif
Created: 7/9/2025
Status: ❌ Closed
Base:
dev← Head:fs-redis-in-memory-tools📝 Commits (4)
8820ac1feat: cache tool servers in redis when available3ec77bfaccept initialised values from env variable91707efdo not return in-memory stuff if it is not current spec1442424fix circularity📊 Changes
6 files changed (+222 additions, -111 deletions)
View changed files
📝
backend/open_webui/functions.py(+1 -1)📝
backend/open_webui/routers/configs.py(+81 -36)📝
backend/open_webui/routers/tools.py(+25 -34)📝
backend/open_webui/utils/middleware.py(+1 -1)📝
backend/open_webui/utils/tools.py(+22 -39)➕
backend/open_webui/utils/tools_cache.py(+92 -0)📄 Description
Pull Request Checklist
Note to first-time contributors: Please open a discussion post in Discussions and describe your changes before submitting a pull request.
Before submitting, make sure you've checked the following:
devbranch.Changelog Entry
Description
Implements Redis-based caching for Tool Servers to support multi-pod backend deployments. Previously, tool servers were cached in memory per pod, causing inconsistencies when multiple backend pods were running. This enhancement stores tool servers in Redis for shared access across all pods, with fallback to memory cache when Redis is unavailable.
We don't really need to keep the TOOL_SERVERS_CONNECTION in memory. We can derive the information from
tool_servers.Added
open_webui/utils/tools_cache.py)get_cached_tool_servers(): Retrieves tool servers from Redis → memory cacheset_and_cache_tool_servers(): Immediately fetches and caches tool servers when configuration is setinvalidate_tool_servers_cache(): Clears both Redis and memory cacheaccess_controland__rawpropertiesChanged
access_controlproperty and__raworiginal connection data__rawpropertyRemoved
request.app.state.config.TOOL_SERVER_CONNECTIONSfor runtime tool server accessTOOL_SERVER_CONNECTIONSfrom main.pyFixed
Breaking Changesn/a
Additional Information
Technical Implementation:
open-webui:tool_serverswith 5-minute TTLaccess_controland__raw(original connection) propertiesMigration Path:
Files Modified:
open_webui/utils/tools_cache.py- New Redis caching moduleopen_webui/utils/tools.py- Enhanced tool server data structureopen_webui/routers/configs.py- Updated configuration endpointsopen_webui/routers/tools.py- Simplified cache-only access + refresh endpointopen_webui/main.py- Removed TOOL_SERVER_CONNECTIONS dependencyTesting Recommendations:
Contributor License Agreement
By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.