[GH-ISSUE #20445] Improve dev container cold start time #34718

New Issue

GiteaMirror · 2026-04-25T08:49:21-05:00

GiteaMirror commented

2026-04-25 08:49:21 -05:00

Originally created by @vlthr on GitHub (Jan 7, 2026).
Original GitHub issue: https://github.com/open-webui/open-webui/issues/20445

Problem

The Docker dev container (docker-compose.dev.yaml) has a slow cold start. The "Waiting for backend..." phase takes significant time even with optimizations applied.

Current Optimizations (Limited Effect)

Already tried in docker-compose.dev.yaml:

- BYPASS_EMBEDDING_AND_RETRIEVAL=true  # Skip SentenceTransformer loading
- ENABLE_BASE_MODELS_CACHE=false       # Skip model pre-caching

These had limited effect on startup time.

Known Bottlenecks (from investigation)

Priority	Cause	Location
CRITICAL	Database migrations at import time	`config.py:70` - `run_migrations()`
CRITICAL	Heavy imports (torch, transformers, langchain)	Various
HIGH	Vector DB client init at import	`vector/factory.py:78`
HIGH	Plugin dependency installation	`main.py:585`
MEDIUM	PyTorch/CUDA checks	`env.py`
MEDIUM	Config loading from DB	`config.py:83-125`

Root Cause Analysis

Many heavy operations happen at module import time, not at request time:

run_migrations() runs when config.py is imported
VECTOR_DB_CLIENT is instantiated when vector/factory.py is imported
Large transitive dependencies (transformers, langchain, chromadb) are imported eagerly

Suggested Investigations

1. Backend Startup

Profile import times with python -X importtime
Defer database migrations to explicit startup phase
Lazy-load vector DB client
Investigate if torch/transformers can be lazy-imported
Consider a "lite mode" that skips all ML dependencies

2. Frontend Startup (Pyodide)

Investigate precaching pyodide dependencies in Dockerfile.dev
npm run pyodide:fetch downloads ~100MB on first run
Could be baked into the dev image to avoid repeated downloads

3. Other Ideas

Use --reload-exclude to reduce uvicorn file watching overhead
Investigate if ENV=dev triggers any extra initialization
Profile the full startup sequence end-to-end

Parent issue: #20444 (Docker hot-reload development setup)

Context

The dev container setup works but takes too long to become ready. This impacts developer iteration speed when restarting the container.

Originally created by @vlthr on GitHub (Jan 7, 2026). Original GitHub issue: https://github.com/open-webui/open-webui/issues/20445 ## Problem The Docker dev container (`docker-compose.dev.yaml`) has a slow cold start. The "Waiting for backend..." phase takes significant time even with optimizations applied. ## Current Optimizations (Limited Effect) Already tried in `docker-compose.dev.yaml`: ```yaml - BYPASS_EMBEDDING_AND_RETRIEVAL=true # Skip SentenceTransformer loading - ENABLE_BASE_MODELS_CACHE=false # Skip model pre-caching ``` These had **limited effect** on startup time. ## Known Bottlenecks (from investigation) | Priority | Cause | Location | |----------|-------|----------| | CRITICAL | Database migrations at import time | `config.py:70` - `run_migrations()` | | CRITICAL | Heavy imports (torch, transformers, langchain) | Various | | HIGH | Vector DB client init at import | `vector/factory.py:78` | | HIGH | Plugin dependency installation | `main.py:585` | | MEDIUM | PyTorch/CUDA checks | `env.py` | | MEDIUM | Config loading from DB | `config.py:83-125` | ### Root Cause Analysis Many heavy operations happen at **module import time**, not at request time: - `run_migrations()` runs when `config.py` is imported - `VECTOR_DB_CLIENT` is instantiated when `vector/factory.py` is imported - Large transitive dependencies (transformers, langchain, chromadb) are imported eagerly ## Suggested Investigations ### 1. Backend Startup - [ ] Profile import times with `python -X importtime` - [ ] Defer database migrations to explicit startup phase - [ ] Lazy-load vector DB client - [ ] Investigate if torch/transformers can be lazy-imported - [ ] Consider a "lite mode" that skips all ML dependencies ### 2. Frontend Startup (Pyodide) - [ ] Investigate precaching pyodide dependencies in `Dockerfile.dev` - [ ] `npm run pyodide:fetch` downloads ~100MB on first run - [ ] Could be baked into the dev image to avoid repeated downloads ### 3. Other Ideas - [ ] Use `--reload-exclude` to reduce uvicorn file watching overhead - [ ] Investigate if `ENV=dev` triggers any extra initialization - [ ] Profile the full startup sequence end-to-end ## Related - Parent issue: #20444 (Docker hot-reload development setup) ## Context The dev container setup works but takes too long to become ready. This impacts developer iteration speed when restarting the container.

GiteaMirror closed this issue

2026-04-25 08:49:22 -05:00

Sign in to join this conversation.

Branches Tags

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/open-webui#34718