# Local Development This setup runs the KohakuHub backend locally in your Python virtualenv, while Docker provides the supporting services: - PostgreSQL for application metadata - MinIO for local S3-compatible storage - LakeFS for repository versioning - Vite dev servers for the main UI and admin UI It does not require `docker compose`. The scripts below use plain `docker`. ## Prerequisites - Docker Engine - Python 3.10+ - Node.js 18+ - An existing virtualenv for backend work ## One-Time Setup If you prefer a single command surface, run `make help` from the repo root to see the shortcuts below. ### 1. Backend dependencies ```bash ./venv/bin/pip install -e ".[dev]" ``` If you already activated the virtualenv: ```bash pip install -e ".[dev]" ``` ### 2. Frontend dependencies ```bash pnpm install ``` ### 3. Create your local env file ```bash cp .env.dev.example .env.dev ``` Or: ```bash make init-env ``` The defaults are already wired to the local Docker services and Vite dev servers. ## Start Local Infra ```bash ./scripts/dev/up_infra.sh ``` Or: ```bash make infra-up ``` This starts: - Postgres on `127.0.0.1:25432` - MinIO API on `127.0.0.1:29001` - MinIO console on `127.0.0.1:29000` - LakeFS on `127.0.0.1:28000` Persistent dev data is stored under `hub-meta/dev/`. ### MinIO CORS (required for in-browser preview) The pure-client safetensors / parquet preview (issue #27) issues cross-origin HTTP `Range` reads against the presigned S3 URL that `/resolve/` 302s to. Browsers will block those reads unless MinIO advertises CORS. `up_infra.sh` already passes `MINIO_API_CORS_ALLOW_ORIGIN=*` to the MinIO container by default, which is what both the Vite dev origin (`http://127.0.0.1:28300`) and production deploys need. Override it with the `DEV_MINIO_CORS_ALLOW_ORIGIN` variable in `.env.dev` if you want to restrict it to a specific origin (the value is forwarded verbatim as `Access-Control-Allow-Origin`; comma-separated origins also work). If you already have a MinIO container from before this change, recreate it so the env var lands: ```bash docker rm -f kohakuhub-dev-minio ./scripts/dev/up_infra.sh ``` Smoke-test the CORS response (should include `Access-Control-Allow-Origin: *` and `Access-Control-Allow-Methods: GET`): ```bash curl -i -X OPTIONS http://127.0.0.1:29001/hub-storage \ -H 'Origin: http://127.0.0.1:28300' \ -H 'Access-Control-Request-Method: GET' ``` Without this, the preview modal opens, shows its spinner, then surfaces a browser-level CORS error instead of the parsed metadata. Downloads via `hf_hub_download` / direct `/resolve/` hits are unaffected — only the cross-origin Range probe the SPA does breaks. ## Start The Backend ```bash ./scripts/dev/run_backend.sh ``` Or: ```bash make backend ``` What this script does: - loads `.env.dev` - initializes LakeFS on first run - writes LakeFS credentials to `hub-meta/dev/lakefs/credentials.env` - runs database migrations - auto-seeds fixed demo users/orgs/repos on a fresh local environment when `KOHAKU_HUB_DEV_AUTO_SEED=true` - starts `uvicorn` with `--reload` on `127.0.0.1:48888` Swagger docs will be available at `http://127.0.0.1:48888/docs`. If you want the migrations + demo seed without holding the terminal open for `uvicorn`, run: ```bash make seed-demo ``` This writes a local manifest to `hub-meta/dev/demo-seed-manifest.json`. ## Start The Frontends Main UI: ```bash pnpm run dev:ui ``` Or: ```bash make ui ``` Admin UI: ```bash pnpm run dev:admin ``` Or: ```bash make admin ``` Access: - Main UI: `http://127.0.0.1:5173` - Admin UI: `http://127.0.0.1:5174` The Vite configs already proxy API traffic to the backend at `127.0.0.1:48888`. ## Why `KOHAKU_HUB_INTERNAL_BASE_URL` Exists For local development, the backend should generate public links that point to the main UI dev server (`5173`), but its own internal follow-up requests should still hit the backend directly (`48888`). Set in `.env.dev`: ```bash KOHAKU_HUB_BASE_URL=http://127.0.0.1:5173 KOHAKU_HUB_INTERNAL_BASE_URL=http://127.0.0.1:48888 ``` This keeps: - browser-facing links on the frontend dev server - backend self-calls off the Vite proxy path ## First Login / Admin Main UI seeded account: - Username: `mai_lin` - Password: `KohakuDev123!` Additional seeded users use the same password: - `leo_park` - `sara_chen` - `noah_kim` - `ivy_ops` The seeded data also includes fixed organizations and repositories, including public/private repos, model/dataset/space types, branches, tags, likes, LFS files, and dataset preview files. Admin UI login does not use a username/password. Open `http://127.0.0.1:5174` and use the token from `.env.dev`. Default local token: ```bash KOHAKU_HUB_ADMIN_SECRET_TOKEN=dev-admin-token-change-me ``` ## Common Commands Restart infra: ```bash ./scripts/dev/down_infra.sh ./scripts/dev/up_infra.sh ``` Or: ```bash make infra-down make infra-up ``` Stop infra only: ```bash ./scripts/dev/down_infra.sh ``` Or: ```bash make infra-down ``` Tail a container log: ```bash docker logs -f kohakuhub-dev-lakefs docker logs -f kohakuhub-dev-minio docker logs -f kohakuhub-dev-postgres ``` ## Backend Tests Backend tests run against the real Postgres, MinIO, and LakeFS services. The same `make test` entrypoint is used locally and in GitHub Actions. Start the local infra first: ```bash make infra-up ``` Run the full backend suite with coverage: ```bash make test ``` Run only one backend submodule by passing a path relative to both `test/kohakuhub/` and `src/kohakuhub/`: ```bash make test RANGE_DIR=api make test RANGE_DIR=api/repo/routers ``` When `RANGE_DIR` is set, pytest runs `test/kohakuhub/${RANGE_DIR}` and coverage focuses on `src/kohakuhub/${RANGE_DIR}`. If you keep local test overrides in a repo-root `.env`, load them into your shell before running tests: ```bash source .env make test ``` The test code reads environment variables only. It does not load `.env` directly. ## Reset Local Data `make reset-local-data` is intentionally destructive. The script prints a bold red warning, explains the consequences, and asks for a single `y/N` confirmation before it clears the local app state through the in-process local reset helper. The reset flow no longer deletes `hub-meta/dev/` directly. Instead, it: - deletes all LakeFS repositories through the local LakeFS API - clears the configured S3 bucket through the storage client - rebuilds the KohakuHub application schema - removes the local demo seed manifest This avoids Docker bind-mount ownership issues and keeps the local infra containers running so you can re-seed immediately. If you want a clean local reset followed by fresh demo data bootstrapping: ```bash make reset-and-seed ``` That command still goes through the same single `y/N` confirmation before anything is deleted. ## Troubleshooting ### Docker service ports are already taken Adjust the port mappings inside [`scripts/dev/up_infra.sh`](/home/zhangshaoang/wtf-projects/KohakuHub/scripts/dev/up_infra.sh) and keep `.env.dev` in sync. ### LakeFS says it is already initialized but credentials are missing The bootstrap credentials are only returned once. If `hub-meta/dev/lakefs-data/` still exists but `hub-meta/dev/lakefs/credentials.env` was removed, either: - restore the credentials file, or - delete `hub-meta/dev/lakefs-data/` and initialize again ### Backend cannot connect to Postgres Check: ```bash docker logs kohakuhub-dev-postgres cat .env.dev ``` Make sure `KOHAKU_HUB_DATABASE_URL` matches `DEV_POSTGRES_*`.