[PR #21944] feat: Implement Redis migration lock for coordinated database migrations #26356

Open
opened 2026-04-20 06:27:12 -05:00 by GiteaMirror · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/21944
Author: @jmleksan
Created: 2/27/2026
Status: 🔄 Open

Base: devHead: feat/migration-redis-lock


📝 Commits (1)

  • c03c37b feat: coordinate DB migrations with Redis lock

📊 Changes

4 files changed (+222 additions, -35 deletions)

View changed files

📝 backend/open_webui/config.py (+0 -22)
📝 backend/open_webui/env.py (+29 -5)
📝 backend/open_webui/internal/db.py (+189 -2)
📝 backend/open_webui/migrations/env.py (+4 -6)

📄 Description

Pull Request Checklist

Before submitting, make sure you've checked the following:

  • Target branch: Verify that the pull request targets the dev branch. PRs targeting main will be immediately closed.
  • Description: Provide a concise description of the changes made in this pull request down below.
  • Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
  • Documentation: Add docs in Open WebUI Docs Repository. Document user-facing behavior, environment variables, public APIs/interfaces, or deployment steps.
  • Dependencies: Are there any new or upgraded dependencies? If so, explain why, update the changelog/docs, and include any compatibility notes. Actually run the code/function that uses updated library to ensure it doesn't crash.
  • Testing: Perform manual tests to verify the implemented fix/feature works as intended AND does not break any other functionality. Include reproducible steps to demonstrate the issue before the fix. Test edge cases (URL encoding, HTML entities, types). Take this as an opportunity to make screenshots of the feature/fix and include them in the PR description.
  • Agentic AI Code: Confirm this Pull Request is not written by any AI Agent or has at least gone through additional human review AND manual testing. If any AI Agent is the co-author of this PR, it may lead to immediate closure of the PR.
  • Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
  • Design & Architecture: Prefer smart defaults over adding new settings; use local state for ephemeral UI logic. Open a Discussion for major architectural or UX changes.
  • Git Hygiene: Keep PRs atomic (one logical change). Clean up commits and rebase on dev to ensure no unrelated commits (e.g. from main) are included. Push updates to the existing PR branch instead of closing and reopening.
  • Title Prefix: To clearly categorize this pull request, prefix the pull request title using one of the following:
    • feat: Introduces a new feature or enhancement to the codebase

Description

The goal of this PR is to simplify database migrations for users with multiple pods/workers without needing manual intervention or increased overhead to handle race condition.
When multiple pods start with ENABLE_DB_MIGRATIONS=True, each pod runs peewee and Alembic migrations. Migrations are idempotent, but running them concurrently can cause race conditions. This PR adds optional coordination via Redis: when REDIS_URL is set and reachable, only one pod at a time holds a migration lock; others block until the lock is free, then run migrations sequentially. If Redis is not configured or unavailable, behavior is unchanged (every pod runs migrations, relying on idempotency).

Changes:

  • env.py: New optional env vars MIGRATION_LOCK_TIMEOUT_SECS, MIGRATION_LOCK_RETRY_SLEEP_SECS, MIGRATION_LOCK_MAX_WAIT_SECS with defaults (600, 5, 900). Used only when Redis is available for the lock.
  • internal/db.py: _try_acquire_migration_lock() acquires a Redis lock (with retries and timeout) before peewee migrations; release_migration_lock_if_held() is called from config after Alembic. If Redis is unavailable, lock is skipped and all pods run migrations as before.
  • config.py: run_migrations() calls release_migration_lock_if_held() in a finally block so the lock is always released after Alembic.

No new dependencies; uses existing Redis client and RedisLock from the codebase. Design uses smart defaults; the new env vars are optional tuning knobs for large or slow DBs.


Changelog Entry

  • 🔒 Redis migration lock. When REDIS_URL is set and reachable, only one pod at a time runs DB migrations (peewee + Alembic); other pods wait for the lock then run migrations sequentially, avoiding concurrent migration races in multi-pod deployments. Optional env vars MIGRATION_LOCK_TIMEOUT_SECS, MIGRATION_LOCK_RETRY_SLEEP_SECS, and MIGRATION_LOCK_MAX_WAIT_SECS (defaults 600, 5, 900) allow tuning for large or slow databases. If Redis is not configured or unavailable, behavior is unchanged and all pods run migrations as before.

Description

  • Add optional Redis-based lock so only one pod runs DB migrations at a time when Redis is configured. Prevents concurrent peewee/Alembic runs in multi-pod deployments. If Redis is not set or unreachable, behavior is unchanged (all pods run migrations; idempotency unchanged).

Added

  • Optional env vars for migration lock tuning: MIGRATION_LOCK_TIMEOUT_SECS (default 600), MIGRATION_LOCK_RETRY_SLEEP_SECS (default 5), MIGRATION_LOCK_MAX_WAIT_SECS (default 900).
  • Redis migration lock coordination in internal/db.py: acquire before peewee migrations, release after Alembic in config.run_migrations().

Changed

  • Startup flow when ENABLE_DB_MIGRATIONS is true: attempt to acquire Redis migration lock first (when Redis is available); then run peewee and Alembic as before; release lock in run_migrations() finally block.

Breaking Changes

  • None. Existing deployments without Redis or with Redis unchanged behave as before.

Contributor License Agreement

By submitting this pull request, I confirm that I have read and fully agree to the Contributor License Agreement (CLA), and I am providing my contributions under its terms.

Note

Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in.


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/21944 **Author:** [@jmleksan](https://github.com/jmleksan) **Created:** 2/27/2026 **Status:** 🔄 Open **Base:** `dev` ← **Head:** `feat/migration-redis-lock` --- ### 📝 Commits (1) - [`c03c37b`](https://github.com/open-webui/open-webui/commit/c03c37b214abc774cf1827d354d2d6f9c9d98f86) feat: coordinate DB migrations with Redis lock ### 📊 Changes **4 files changed** (+222 additions, -35 deletions) <details> <summary>View changed files</summary> 📝 `backend/open_webui/config.py` (+0 -22) 📝 `backend/open_webui/env.py` (+29 -5) 📝 `backend/open_webui/internal/db.py` (+189 -2) 📝 `backend/open_webui/migrations/env.py` (+4 -6) </details> ### 📄 Description # Pull Request Checklist **Before submitting, make sure you've checked the following:** - [x] **Target branch:** Verify that the pull request targets the `dev` branch. **PRs targeting `main` will be immediately closed.** - [x] **Description:** Provide a concise description of the changes made in this pull request down below. - [x] **Changelog:** Ensure a changelog entry following the format of [Keep a Changelog](https://keepachangelog.com/) is added at the bottom of the PR description. - [x] **Documentation:** Add docs in [Open WebUI Docs Repository](https://github.com/open-webui/docs). Document user-facing behavior, environment variables, public APIs/interfaces, or deployment steps. - [x] **Dependencies:** Are there any new or upgraded dependencies? If so, explain why, update the changelog/docs, and include any compatibility notes. Actually run the code/function that uses updated library to ensure it doesn't crash. - [x] **Testing:** Perform manual tests to **verify the implemented fix/feature works as intended AND does not break any other functionality**. Include reproducible steps to demonstrate the issue before the fix. Test edge cases (URL encoding, HTML entities, types). Take this as an opportunity to **make screenshots of the feature/fix and include them in the PR description**. - [x] **Agentic AI Code:** Confirm this Pull Request is **not written by any AI Agent** or has at least **gone through additional human review AND manual testing**. If any AI Agent is the co-author of this PR, it may lead to immediate closure of the PR. - [x] **Code review:** Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards? - [x] **Design & Architecture:** Prefer smart defaults over adding new settings; use local state for ephemeral UI logic. Open a Discussion for major architectural or UX changes. - [x] **Git Hygiene:** Keep PRs atomic (one logical change). Clean up commits and rebase on `dev` to ensure no unrelated commits (e.g. from `main`) are included. Push updates to the existing PR branch instead of closing and reopening. - [x] **Title Prefix:** To clearly categorize this pull request, prefix the pull request title using one of the following: - **feat**: Introduces a new feature or enhancement to the codebase --- # Description The goal of this PR is to simplify database migrations for users with multiple pods/workers without needing manual intervention or increased overhead to handle race condition. When multiple pods start with `ENABLE_DB_MIGRATIONS=True`, each pod runs peewee and Alembic migrations. Migrations are idempotent, but running them concurrently can cause race conditions. This PR adds optional coordination via Redis: when `REDIS_URL` is set and reachable, only one pod at a time holds a migration lock; others block until the lock is free, then run migrations sequentially. If Redis is not configured or unavailable, behavior is unchanged (every pod runs migrations, relying on idempotency). **Changes:** - **env.py:** New optional env vars `MIGRATION_LOCK_TIMEOUT_SECS`, `MIGRATION_LOCK_RETRY_SLEEP_SECS`, `MIGRATION_LOCK_MAX_WAIT_SECS` with defaults (600, 5, 900). Used only when Redis is available for the lock. - **internal/db.py:** `_try_acquire_migration_lock()` acquires a Redis lock (with retries and timeout) before peewee migrations; `release_migration_lock_if_held()` is called from config after Alembic. If Redis is unavailable, lock is skipped and all pods run migrations as before. - **config.py:** `run_migrations()` calls `release_migration_lock_if_held()` in a `finally` block so the lock is always released after Alembic. No new dependencies; uses existing Redis client and `RedisLock` from the codebase. Design uses smart defaults; the new env vars are optional tuning knobs for large or slow DBs. --- # Changelog Entry - 🔒 **Redis migration lock.** When `REDIS_URL` is set and reachable, only one pod at a time runs DB migrations (peewee + Alembic); other pods wait for the lock then run migrations sequentially, avoiding concurrent migration races in multi-pod deployments. Optional env vars `MIGRATION_LOCK_TIMEOUT_SECS`, `MIGRATION_LOCK_RETRY_SLEEP_SECS`, and `MIGRATION_LOCK_MAX_WAIT_SECS` (defaults 600, 5, 900) allow tuning for large or slow databases. If Redis is not configured or unavailable, behavior is unchanged and all pods run migrations as before. ### Description - Add optional Redis-based lock so only one pod runs DB migrations at a time when Redis is configured. Prevents concurrent peewee/Alembic runs in multi-pod deployments. If Redis is not set or unreachable, behavior is unchanged (all pods run migrations; idempotency unchanged). ### Added - Optional env vars for migration lock tuning: `MIGRATION_LOCK_TIMEOUT_SECS` (default 600), `MIGRATION_LOCK_RETRY_SLEEP_SECS` (default 5), `MIGRATION_LOCK_MAX_WAIT_SECS` (default 900). - Redis migration lock coordination in `internal/db.py`: acquire before peewee migrations, release after Alembic in `config.run_migrations()`. ### Changed - Startup flow when `ENABLE_DB_MIGRATIONS` is true: attempt to acquire Redis migration lock first (when Redis is available); then run peewee and Alembic as before; release lock in `run_migrations()` `finally` block. ### Breaking Changes - None. Existing deployments without Redis or with Redis unchanged behave as before. --- ### Contributor License Agreement By submitting this pull request, I confirm that I have read and fully agree to the [Contributor License Agreement (CLA)](https://github.com/open-webui/open-webui/blob/main/CONTRIBUTOR_LICENSE_AGREEMENT), and I am providing my contributions under its terms. > [!NOTE] > Deleting the CLA section will lead to immediate closure of your PR and it will not be merged in. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-20 06:27:12 -05:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#26356