KohakuHub/docs/development/local-dev.md

Local Development

This setup runs the KohakuHub backend locally in your Python virtualenv, while Docker provides the supporting services:

• PostgreSQL for application metadata
• MinIO for local S3-compatible storage
• LakeFS for repository versioning
• Vite dev servers for the main UI and admin UI

It does not require docker compose. The scripts below use plain docker.

Prerequisites

• Docker Engine
• Python 3.10+
• Node.js 18+
• An existing virtualenv for backend work

One-Time Setup

If you prefer a single command surface, run make help from the repo root to see the shortcuts below.

1. Backend dependencies

./venv/bin/pip install -e ".[dev]"

If you already activated the virtualenv:

pip install -e ".[dev]"

2. Frontend dependencies

pnpm install

3. Create your local env file

cp .env.dev.example .env.dev

Or:

make init-env

The defaults are already wired to the local Docker services and Vite dev servers.

Start Local Infra

./scripts/dev/up_infra.sh

Or:

make infra-up

This starts:

• Postgres on 127.0.0.1:25432
• MinIO API on 127.0.0.1:29001
• MinIO console on 127.0.0.1:29000
• LakeFS on 127.0.0.1:28000

Persistent dev data is stored under hub-meta/dev/.

MinIO CORS (required for in-browser preview)

The pure-client safetensors / parquet preview (issue #27) issues cross-origin HTTP Range reads against the presigned S3 URL that /resolve/ 302s to. Browsers will block those reads unless MinIO advertises CORS.

up_infra.sh already passes MINIO_API_CORS_ALLOW_ORIGIN=* to the MinIO container by default, which is what both the Vite dev origin (http://127.0.0.1:28300) and production deploys need. Override it with the DEV_MINIO_CORS_ALLOW_ORIGIN variable in .env.dev if you want to restrict it to a specific origin (the value is forwarded verbatim as Access-Control-Allow-Origin; comma-separated origins also work).

If you already have a MinIO container from before this change, recreate it so the env var lands:

docker rm -f kohakuhub-dev-minio
./scripts/dev/up_infra.sh

Smoke-test the CORS response (should include Access-Control-Allow-Origin: * and Access-Control-Allow-Methods: GET):

curl -i -X OPTIONS http://127.0.0.1:29001/hub-storage \
     -H 'Origin: http://127.0.0.1:28300' \
     -H 'Access-Control-Request-Method: GET'

Without this, the preview modal opens, shows its spinner, then surfaces a browser-level CORS error instead of the parsed metadata. Downloads via hf_hub_download / direct /resolve/ hits are unaffected — only the cross-origin Range probe the SPA does breaks.

Start The Backend

./scripts/dev/run_backend.sh

Or:

make backend

What this script does:

• loads .env.dev
• initializes LakeFS on first run
• writes LakeFS credentials to hub-meta/dev/lakefs/credentials.env
• runs database migrations
• auto-seeds fixed demo users/orgs/repos on a fresh local environment when KOHAKU_HUB_DEV_AUTO_SEED=true
• starts uvicorn with --reload on 127.0.0.1:48888

Swagger docs will be available at http://127.0.0.1:48888/docs.

If you want the migrations + demo seed without holding the terminal open for uvicorn, run:

make seed-demo

This writes a local manifest to hub-meta/dev/demo-seed-manifest.json.

Start The Frontends

Main UI:

pnpm run dev:ui

Or:

make ui

Admin UI:

pnpm run dev:admin

Or:

make admin

Access:

• Main UI: http://127.0.0.1:5173
• Admin UI: http://127.0.0.1:5174

The Vite configs already proxy API traffic to the backend at 127.0.0.1:48888.

Why KOHAKU_HUB_INTERNAL_BASE_URL Exists

For local development, the backend should generate public links that point to the main UI dev server (5173), but its own internal follow-up requests should still hit the backend directly (48888).

Set in .env.dev:

KOHAKU_HUB_BASE_URL=http://127.0.0.1:5173
KOHAKU_HUB_INTERNAL_BASE_URL=http://127.0.0.1:48888

This keeps:

• browser-facing links on the frontend dev server
• backend self-calls off the Vite proxy path

First Login / Admin

Main UI seeded account:

• Username: mai_lin
• Password: KohakuDev123!

Additional seeded users use the same password:

• leo_park
• sara_chen
• noah_kim
• ivy_ops

The seeded data also includes fixed organizations and repositories, including public/private repos, model/dataset/space types, branches, tags, likes, LFS files, and dataset preview files.

Admin UI login does not use a username/password. Open http://127.0.0.1:5174 and use the token from .env.dev.

Default local token:

KOHAKU_HUB_ADMIN_SECRET_TOKEN=dev-admin-token-change-me

Common Commands

Restart infra:

./scripts/dev/down_infra.sh
./scripts/dev/up_infra.sh

Or:

make infra-down
make infra-up

Stop infra only:

./scripts/dev/down_infra.sh

Or:

make infra-down

Tail a container log:

docker logs -f kohakuhub-dev-lakefs
docker logs -f kohakuhub-dev-minio
docker logs -f kohakuhub-dev-postgres

Backend Tests

Backend tests run against the real Postgres, MinIO, and LakeFS services. The same make test entrypoint is used locally and in GitHub Actions.

Start the local infra first:

make infra-up

Run the full backend suite with coverage:

make test

Run only one backend submodule by passing a path relative to both test/kohakuhub/ and src/kohakuhub/:

make test RANGE_DIR=api
make test RANGE_DIR=api/repo/routers

When RANGE_DIR is set, pytest runs test/kohakuhub/${RANGE_DIR} and coverage focuses on src/kohakuhub/${RANGE_DIR}.

If you keep local test overrides in a repo-root .env, load them into your shell before running tests:

source .env
make test

The test code reads environment variables only. It does not load .env directly.

Reset Local Data

make reset-local-data is intentionally destructive. The script prints a bold red warning, explains the consequences, and asks for a single y/N confirmation before it clears the local app state through the in-process local reset helper.

The reset flow no longer deletes hub-meta/dev/ directly. Instead, it:

• deletes all LakeFS repositories through the local LakeFS API
• clears the configured S3 bucket through the storage client
• rebuilds the KohakuHub application schema
• removes the local demo seed manifest

This avoids Docker bind-mount ownership issues and keeps the local infra containers running so you can re-seed immediately.

If you want a clean local reset followed by fresh demo data bootstrapping:

make reset-and-seed

That command still goes through the same single y/N confirmation before anything is deleted.

Troubleshooting

Docker service ports are already taken

Adjust the port mappings inside scripts/dev/up_infra.sh and keep .env.dev in sync.

LakeFS says it is already initialized but credentials are missing

The bootstrap credentials are only returned once. If hub-meta/dev/lakefs-data/ still exists but hub-meta/dev/lakefs/credentials.env was removed, either:

• restore the credentials file, or
• delete hub-meta/dev/lakefs-data/ and initialize again

Backend cannot connect to Postgres

Check:

docker logs kohakuhub-dev-postgres
cat .env.dev

Make sure KOHAKU_HUB_DATABASE_URL matches DEV_POSTGRES_*.