[PR #16552] [CLOSED] debug: add comprehensive startup instrumentation for air-gapped deployments #39797

Closed
opened 2026-04-25 12:14:40 -05:00 by GiteaMirror · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/open-webui/open-webui/pull/16552
Author: @RCastillo-Clave
Created: 8/13/2025
Status: Closed

Base: mainHead: feature/fix-owui-startup-hanging


📝 Commits (10+)

  • 9c8e401 feat(owui): Update package dependencies and add READMEs. Fixes issue #1.
  • 95f05ab fix(ci): Increase Node.js memory for frontend build
  • cad1298 Silent test commit
  • 79caf9d build: Ignore docker build artifact
  • 3a371e7 Feature: modify Dockerfile to generate an authn token to the DB
  • 714a762 Merge branch 'open-webui:main' into feature/owui-db-support
  • 6825620 Merge pull request #1 from clavesec/feature/owui-db-support
  • 8c9e50b Enable VSCode debugging for Open WebUI in Docker
  • b1e0609 fix: dockerfile and entrypoint for debugging in VSCode
  • 3b5bab4 fix: merge feat/vscode-docker-debug for local Docker debugging rig

📊 Changes

35 files changed (+3253 additions, -133 deletions)

View changed files

📝 .gitignore (+6 -0)
.vscode/launch.json (+24 -0)
AWS_INTEGRATION_MEMO_FOR_TOM.md (+454 -0)
DATABASE_VERIFICATION_QUERIES.sql (+263 -0)
DEBUG_OUTPUT_ANALYSIS_GUIDE.md (+296 -0)
DOCKER_BUILD_TEST_INSTRUCTIONS.md (+233 -0)
📝 Dockerfile (+44 -4)
📝 README.md (+4 -0)
backend/docker-entrypoint.sh (+119 -0)
backend/open_webui/README_IAM_DB.md (+16 -0)
📝 backend/open_webui/env.py (+14 -0)
📝 backend/open_webui/internal/db.py (+103 -6)
📝 backend/open_webui/internal/wrappers.py (+77 -4)
📝 backend/open_webui/main.py (+57 -3)
📝 backend/open_webui/migrations/env.py (+41 -1)
backend/open_webui/migrations/versions/c4a3b2d1e0f_add_encryption_fields_to_user.py (+42 -0)
📝 backend/open_webui/models/auths.py (+149 -30)
backend/open_webui/models/db_encryption_shim.py (+250 -0)
📝 backend/open_webui/models/users.py (+108 -44)
📝 backend/open_webui/retrieval/vector/dbs/pgvector.py (+37 -3)

...and 15 more files

📄 Description

Summary

Add detailed logging and instrumentation to diagnose startup issues in air-gapped environments.

Changes

FastAPI Lifespan Instrumentation

  • Add step-by-step logging for lifespan initialization
  • Track Redis connection setup, thread pool configuration
  • Monitor periodic cleanup task creation

Database Migration Debugging

  • Add detailed logging to Peewee migration process
  • Track each migration file processing
  • Monitor database connection establishment

Database Connection Instrumentation

  • Add comprehensive logging to register_connection function
  • Track URL augmentation, SSL/IAM setup
  • Monitor actual database connection attempts

Import-Level Debugging

  • Add tracking for critical module imports
  • Monitor model imports (Functions, Models, Users, Chats)
  • Track config imports that might access external resources

Technical Details

  • Use both print() and log.info() for visibility even when logger not initialized
  • Emoji prefixes for easy log filtering:
    • 🚀 LIFESPAN: FastAPI startup steps
    • 🗄️ DB_MIGRATION: Database migration progress
    • 🔌 DB_CONNECT: Database connection details
    • 🚨 MAIN_IMPORT: Module import tracking

Impact

  • Enables precise identification of startup hanging points
  • Critical for troubleshooting air-gapped deployment issues
  • No functional changes - debugging instrumentation only

Notes

This instrumentation was essential for identifying that startup failures were caused by external network calls during module imports, not database or Redis issues.

🤖 Generated with Claude Code


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/open-webui/open-webui/pull/16552 **Author:** [@RCastillo-Clave](https://github.com/RCastillo-Clave) **Created:** 8/13/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `feature/fix-owui-startup-hanging` --- ### 📝 Commits (10+) - [`9c8e401`](https://github.com/open-webui/open-webui/commit/9c8e401583713c5206f81cb64eb24ab740006b4a) feat(owui): Update package dependencies and add READMEs. Fixes issue #1. - [`95f05ab`](https://github.com/open-webui/open-webui/commit/95f05ab98cccf15094de9a5278c12e993fc97d1b) fix(ci): Increase Node.js memory for frontend build - [`cad1298`](https://github.com/open-webui/open-webui/commit/cad1298a13429a0e500f6c7d83a3828d27fbf968) Silent test commit - [`79caf9d`](https://github.com/open-webui/open-webui/commit/79caf9d6e4140c04fc88fda5290992eb74e275c8) build: Ignore docker build artifact - [`3a371e7`](https://github.com/open-webui/open-webui/commit/3a371e7382d52e8629a85dbc47cc3daaa68dd9d9) Feature: modify Dockerfile to generate an authn token to the DB - [`714a762`](https://github.com/open-webui/open-webui/commit/714a7627623ca2ddb47dfdf0328c336315e745df) Merge branch 'open-webui:main' into feature/owui-db-support - [`6825620`](https://github.com/open-webui/open-webui/commit/68256204a95c62b527cb6604cd34b6409a62a034) Merge pull request #1 from clavesec/feature/owui-db-support - [`8c9e50b`](https://github.com/open-webui/open-webui/commit/8c9e50b0858f2ca6af0051094e49db45fa26d612) Enable VSCode debugging for Open WebUI in Docker - [`b1e0609`](https://github.com/open-webui/open-webui/commit/b1e0609abcf5dcf50078fe377911fbf9e9ee94af) fix: dockerfile and entrypoint for debugging in VSCode - [`3b5bab4`](https://github.com/open-webui/open-webui/commit/3b5bab4921af30cbc338669f3eeccbecf92cc640) fix: merge feat/vscode-docker-debug for local Docker debugging rig ### 📊 Changes **35 files changed** (+3253 additions, -133 deletions) <details> <summary>View changed files</summary> 📝 `.gitignore` (+6 -0) ➕ `.vscode/launch.json` (+24 -0) ➕ `AWS_INTEGRATION_MEMO_FOR_TOM.md` (+454 -0) ➕ `DATABASE_VERIFICATION_QUERIES.sql` (+263 -0) ➕ `DEBUG_OUTPUT_ANALYSIS_GUIDE.md` (+296 -0) ➕ `DOCKER_BUILD_TEST_INSTRUCTIONS.md` (+233 -0) 📝 `Dockerfile` (+44 -4) 📝 `README.md` (+4 -0) ➕ `backend/docker-entrypoint.sh` (+119 -0) ➕ `backend/open_webui/README_IAM_DB.md` (+16 -0) 📝 `backend/open_webui/env.py` (+14 -0) 📝 `backend/open_webui/internal/db.py` (+103 -6) 📝 `backend/open_webui/internal/wrappers.py` (+77 -4) 📝 `backend/open_webui/main.py` (+57 -3) 📝 `backend/open_webui/migrations/env.py` (+41 -1) ➕ `backend/open_webui/migrations/versions/c4a3b2d1e0f_add_encryption_fields_to_user.py` (+42 -0) 📝 `backend/open_webui/models/auths.py` (+149 -30) ➕ `backend/open_webui/models/db_encryption_shim.py` (+250 -0) 📝 `backend/open_webui/models/users.py` (+108 -44) 📝 `backend/open_webui/retrieval/vector/dbs/pgvector.py` (+37 -3) _...and 15 more files_ </details> ### 📄 Description ## Summary Add detailed logging and instrumentation to diagnose startup issues in air-gapped environments. ## Changes ### FastAPI Lifespan Instrumentation - Add step-by-step logging for lifespan initialization - Track Redis connection setup, thread pool configuration - Monitor periodic cleanup task creation ### Database Migration Debugging - Add detailed logging to Peewee migration process - Track each migration file processing - Monitor database connection establishment ### Database Connection Instrumentation - Add comprehensive logging to `register_connection` function - Track URL augmentation, SSL/IAM setup - Monitor actual database connection attempts ### Import-Level Debugging - Add tracking for critical module imports - Monitor model imports (Functions, Models, Users, Chats) - Track config imports that might access external resources ## Technical Details - Use both `print()` and `log.info()` for visibility even when logger not initialized - Emoji prefixes for easy log filtering: - 🚀 LIFESPAN: FastAPI startup steps - 🗄️ DB_MIGRATION: Database migration progress - 🔌 DB_CONNECT: Database connection details - 🚨 MAIN_IMPORT: Module import tracking ## Impact - Enables precise identification of startup hanging points - Critical for troubleshooting air-gapped deployment issues - No functional changes - debugging instrumentation only ## Notes This instrumentation was essential for identifying that startup failures were caused by external network calls during module imports, not database or Redis issues. 🤖 Generated with [Claude Code](https://claude.ai/code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-04-25 12:14:40 -05:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/open-webui#39797