[PR #16] [MERGED] Sync dev/narugo1992: HF compat hardening, client-side preview, dev+test infra, classified-error UX #16

New Issue

GiteaMirror · 2026-04-30T09:29:03-05:00

GiteaMirror commented

2026-04-30 09:29:03 -05:00

📋 Pull Request Information

Original PR: https://github.com/KohakuBlueleaf/KohakuHub/pull/16
Author: @narugo1992
Created: 4/26/2026
Status: ✅ Merged
Merged: 4/26/2026
Merged by: @KohakuBlueleaf

Base: main ← Head: dev/narugo1992

📝 Commits (10+)

704cd67 add action: delete ghost repo before create new repo
51a544e fix delete function in crud
4c6dba6 Add local development bootstrap and deterministic demo fixtures
b5a6ab5 Add new-tab support to repo card titles
77786df Add new-tab support to commit links and breadcrumbs
e53f48b Merge pull request #2 from deepghs/dev/click
98c0e02 Add session-scoped repo sort preferences and timezone-aware timestamps
7b29dbf Merge pull request #3 from deepghs/dev/order
42b7f3f Raise LFS keep versions UI limit to 9999
82ea9d9 Merge pull request #4 from deepghs/chore/lfs-keep-versions-max-9999

📊 Changes

292 files changed (+51635 additions, -2204 deletions)

View changed files

➕ .coveragerc (+21 -0)
➕ .env.dev.example (+51 -0)
➕ .github/workflows/fullstack-tests.yml (+205 -0)
📝 .gitignore (+12 -0)
➕ Makefile (+150 -0)
➕ codecov.yml (+11 -0)
📝 config-example.toml (+2 -1)
📝 docker-compose.example.yml (+4 -0)
➕ docs/development/local-dev.md (+329 -0)
📝 pyproject.toml (+4 -1)
📝 pytest.ini (+11 -2)
➕ scripts/dev/down_infra.sh (+26 -0)
➕ scripts/dev/generate_preview_test_fixtures.py (+79 -0)
➕ scripts/dev/init_lakefs.py (+152 -0)
➕ scripts/dev/reset.py (+175 -0)
➕ scripts/dev/reset_local_data.sh (+68 -0)
➕ scripts/dev/reset_local_data_direct.py (+39 -0)
➕ scripts/dev/run_backend.sh (+76 -0)
➕ scripts/dev/seed_demo_data.py (+3291 -0)
➕ scripts/dev/seed_shared.py (+3 -0)

...and 80 more files

📄 Description

Overview

This PR proposes upstreaming the work that has accumulated on deepghs/KohakuHub@dev/narugo1992. It supersedes the earlier exploratory PRs KohakuBlueleaf/KohakuHub#14 and KohakuBlueleaf/KohakuHub#15, which were closed as drafts while the work consolidated.

The branch has been split into focused PRs in the deepghs fork (each reviewed, tested, and merged in isolation). This PR is therefore a single sync that bundles those already-reviewed pieces rather than a giant unreviewed dump.

📊	Stat
Commits	86
Files changed	292
Insertions / Deletions	+51,635 / -2,204
Backend src files touched	25
Frontend src files touched	30
New/changed test files	164
New dev scripts	11
New CI workflow	1 (`fullstack-tests.yml`)

The work clusters into seven independently-reviewable themes, listed below from highest to lowest blast radius. Every change is gated by tests; the new CI job frontend-backend-unit-tests runs the matrix on each push.

🧭 Theme map

Theme	What it does	Deepghs PRs	Closed issues
🧪 Test infrastructure & CI	Service-backed pytest harness, fast harness, frontend Vitest suites for both UIs, coverage gate, multi-version `huggingface_hub` matrix	#6	—
🤝 HuggingFace Hub compatibility	Fixes a swathe of compat regressions surfaced by `huggingface_hub`, plus a deep coverage matrix	#18, #20, #21, #25	#7, #8, #9, #10, #11, #12, #13, #14, #15, #16, #17, #24
🔍 Pure-client metadata preview	Safetensors tree view + parquet preview, no backend reads, no precomputation	#28, #29	#27
🚦 Classified-error UX	Replace generic toasts with structured `ErrorState` (gated / not-found / unavailable / private) across file-facing surfaces	#30	#31
🛠 Repo viewer & UX polish	Open-in-new-tab on titles/breadcrumbs/commits, session-scoped sort, timezone-aware times, login-on-Enter, file-list rows, LFS keep-versions cap	#2, #3, #4, #5, #32	—
🧰 Local dev bootstrap	One-shot Postgres/MinIO/LakeFS infra, deterministic demo seed, reset/verify scripts, `docs/development/local-dev.md`	(initial commits + #6)	—
🚑 302→403 orphan-state fix	Heal stale `_lakefs/dummy` markers on create; reorder delete to `LakeFS-then-S3` to avoid orphan-storage 302→403	#33	(production incident)

🧪 Test infrastructure & CI

The biggest single contribution by file count. Before this branch the repo had no automated test suite; now there is a service-backed backend harness, a fast in-process harness, frontend Vitest suites for both UIs, and a CI workflow that runs them against a real Postgres/MinIO/LakeFS stack.

🔬 Area	What was added
Backend pytest	`test/conftest.py`, `test/kohakuhub/support/` (bootstrap, seed, fakes, live server, mock HF server, service state), per-module `test_.py` and `test__unit.py`
Frontend Vitest	`test/kohaku-hub-ui/` and `test/kohaku-hub-admin/`: stores, pages, utils, components, MSW setup, fixtures
HF compat matrix	Backend tests run against `huggingface_hub` `0.20.3 / 0.30.2 / 0.36.2 / 1.0.1 / 1.6.0 / latest`, on Python 3.10 / 3.11 / 3.12
CI	`.github/workflows/fullstack-tests.yml` (backend + frontend + admin frontend), Codecov upload made non-fatal
Coverage gates	`.coveragerc`, `codecov.yml`; backend defaults to 80% with per-range overrides through `Makefile` `make test-backend RANGE_DIR=...`
Test isolation	Per-module backend baseline restore, `backend_per_test` marker for tests that need a fresh slate, no `.env` loading inside tests

⚠️ The backend tests do hit real services (Postgres/MinIO/LakeFS). make infra-up brings them up; CI provisions them as service containers.

🤝 HuggingFace Hub compatibility

A cluster of compat bugs were reported by triaging real huggingface_hub calls against KohakuHub. Each was filed as an issue in the deepghs fork, fixed, and covered by tests in the matrix above.

🔢	Issue	Symptom	Fix
①	deepghs#7	Repo tree navigation duplicated nested paths in tree/blob/edit routes	Path normalisation in router
②	deepghs#8	Commit creation API returned non-HF-compatible commit URLs	Normalise commit URL shape
③	deepghs#9	Branch / tag / refs workflows incomplete	Implement HF-compatible branches/tags endpoints
④	deepghs#10	Repo info sliced a missing commit SHA → recoverable lookup turned into 500	Defensive SHA handling
⑤	deepghs#11	HF-compatible error responses didn't sanitise headers	Strip CR/LF before emitting
⑥	deepghs#12	Admin create-user response omitted `is_org` field	Align with admin user shape
⑦	deepghs#13	Legacy quota endpoint drifted after public/private quota split	Bring legacy endpoint back in line
⑧	deepghs#14	Config loader broke on Python 3.10 — unconditional `tomllib` import	Guarded import / `tomli` fallback
⑨	deepghs#15	Dataset viewer self-calls used the frontend base URL	Switch to `internal_base_url`
⑩	deepghs#16	Git Smart HTTP token auth helper declared `async`, used sync	Make it consistent
⑪	deepghs#17	Trending endpoint raised `NameError` on inaccessible repos	Fix variable scope in skip path
⑫	deepghs#24	`hf_hub_download` `FileMetadataError`: `/resolve` 302 lacked `X-Repo-Commit` / `Cache-Control`	Carry HF metadata headers on the 302

Beyond bug fixes, the branch adds:

🧬 What	Where
HF-compatible tree-expand metadata (last commit per entry)	`src/kohakuhub/api/repo/routers/tree.py`
Fallback transparency to `huggingface_hub`: HEAD semantics, real-client tests, HF seeded as a global fallback in `make seed-demo`	`src/kohakuhub/api/fallback/`, `scripts/dev/seed_demo_data.py`
`not_implemented` API surface (telemetry, etc.) returning the right HF shape rather than 500	`src/kohakuhub/api/not_implemented.py`
409-conflict surfacing in the UI on create-repo (deepghs#20)	`src/kohaku-hub-ui/src/pages/[type]s/...`
Fallback error classification: `gated` / `not-found` / `unavailable` rather than collapsing everything to 404 (deepghs#29)	`src/kohakuhub/api/fallback/operations.py`, `utils.py`

🔍 Pure-client metadata preview (deepghs#27 → deepghs#28)

Adds a file-preview dialog that runs entirely in the browser: no backend reads, no precomputation, no metadata cache. Targets parity with HF's preview UX for safetensors and parquet specifically.

📦 Format	What it shows	Implementation
`.safetensors`	Tree view of tensor names with chain-collapse + toggle, dtype/shape per leaf, total parameter count	`utils/safetensors.{js,test}` + `FilePreviewDialog.vue`
`.parquet`	Schema + first N rows, page-aware row count	`utils/parquet.{js,test}` + `FilePreviewDialog.vue`
Range fetches	Both formats only fetch the byte ranges they need (header → metadata → optional sample), so the dialog opens fast on multi-GB files	`utils/file_preview.{js,test}`

Tests live in test/kohaku-hub-ui/utils/test_safetensors.test.js (708 lines) and test_parquet.test.js — including binary fixtures under test/kohaku-hub-ui/fixtures/previews/.

🚦 Classified-error UX (deepghs#30, closes deepghs#31)

Standardises a single <ErrorState> component used wherever a file or repo could be unreachable. Replaces ad-hoc toasts that masked real causes (gated repo, 404, upstream down, private without auth).

📍 Surface	Before	After
Blob view	"Failed to load file"	Categorised: gated / private / not-found / network / fallback-upstream-down
Edit view	Toast	Inline `ErrorState` with retry / sign-in / request-access affordance
Upload view	Toast	Inline `ErrorState`
Settings	Toast	Inline `ErrorState`
File preview dialog	Generic	Same classification, including upstream fallback errors
Right-click on file list rows	Forced router push	Plain `<a>` anchors so middle-click / Cmd-click open in new tab (deepghs#32)

utils/http_errors.{js,test} (390 lines of tests) carries the classifier so every surface uses the same rules.

🛠 Repo viewer & UX polish

A sequence of small, low-risk UX improvements, each landed as its own deepghs PR:

🪛 PR	Change
deepghs#2	New-tab support on repo card titles, commit links, breadcrumbs
deepghs#3	Session-scoped repo sort preferences + timezone-aware timestamps
deepghs#4	LFS "keep versions" UI input cap raised to 9999
deepghs#5	HF-compatible tree expand metadata (last commit per entry, file size restore)
deepghs#32	File-list rows are plain anchors so right-click / middle-click / Cmd-click work
(small commits)	Login form submits on Enter; tone down file-list commit link styling; tighten repo branch selector width

🧰 Local development bootstrap

The repo previously assumed a hand-rolled Docker stack. The branch adds a turnkey dev workflow:

🚀 Command	What it does
`make init-env`	Copy `.env.dev.example` → `.env.dev` if missing
`make install`	Backend `pip install` + frontend `npm install` for both UIs
`make infra-up`	Start Postgres / MinIO / LakeFS as named containers with persisted volumes
`make seed-demo`	Run migrations + deterministic demo seed (users, orgs, repos, files, LFS objects, fallback sources)
`make reset-and-seed`	Wipe persisted local data and reseed
`make backend`	Run FastAPI in reload mode
`make ui` / `make admin`	Run the two Vite frontends
`make test-backend` / `make test-ui` / `make test-ui-admin`	Run the suites with coverage

📂 Key new files
`Makefile`, `.env.dev.example`, `docker-compose.example.yml`
`scripts/dev/up_infra.sh`, `down_infra.sh`, `init_lakefs.py`, `seed_demo_data.py` (3,291 lines), `verify_seed_data.py`, `reset.py`, `reset_local_data.sh`
`docs/development/local-dev.md` (329-line guide)

🚑 302→403 orphan-state fix (deepghs#33)

The most recent piece. A pair of related production failures were producing a 302 redirect that resolved to a 403 on subsequent reads:

🔥 Failure mode	Pre-fix	Post-fix
Create after aborted create	LakeFS refused with `storage namespace already in use` pointing at our own `_lakefs/dummy`; user could never recreate	Heal narrowly: verify LakeFS truly has no repo, sample S3 prefix, only clean up if every key is an internal marker, then retry once
Partial delete	S3 cleanup ran first; if LakeFS deletion then failed, the LakeFS repo survived but its storage was wiped → 302 → 403 on next read	Delete LakeFS metadata first; S3 cleanup only follows once LakeFS no longer references it

Both are covered by regression tests (test/kohakuhub/api/repo/routers/test_crud_unit.py) that fail on the prior code and pass after the fix — including guardrail tests that the heal path refuses to clean a namespace containing any user object.

✅ How to validate locally

# Bring up infra
make infra-up

# Backend tests (real Postgres/MinIO/LakeFS, ~35s for the repo router suite)
make test-backend RANGE_DIR=api/repo/routers

# Full backend suite
make test-backend

# Frontend suites
make test-ui
make test-ui-admin

CI runs the same matrix on every push.

🧷 Open issues acknowledged but not resolved here

These are deliberately deferred and tracked as open issues in the deepghs fork:

Issue	Why deferred
deepghs#19	Downstream library E2E (transformers / datasets / diffusers / timm) — needs a separate harness
deepghs#22	HF `/api/telemetry/` — protocol notes captured, decision to defer until HF stabilises the contract
deepghs#23	Organizations list page slowness — needs aggregation rework, not just a bug fix
deepghs#26	Backend audit of broad `except Exception` (168 sites) — multi-month cleanup, not a single PR

🧾 Test plan

Backend service-backed tests pass on Python 3.10/3.11/3.12 against huggingface_hub 0.20.3 / 0.30.2 / 0.36.2 / 1.0.1 / 1.6.0 / latest
Frontend Vitest suites pass for both UIs (>80% coverage)
CI green on every commit in the branch
302→403 regression suite verified to fail on pre-fix and pass on post-fix
Manual smoke: huggingface_hub create_repo → upload_file → hf_hub_download round trip against a fresh make infra-up && make seed-demo stack
Manual smoke: file preview opens correctly on a multi-GB safetensors and a real parquet from a seeded repo

🤖 Generated with Claude Code

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/KohakuBlueleaf/KohakuHub/pull/16 **Author:** [@narugo1992](https://github.com/narugo1992) **Created:** 4/26/2026 **Status:** ✅ Merged **Merged:** 4/26/2026 **Merged by:** [@KohakuBlueleaf](https://github.com/KohakuBlueleaf) **Base:** `main` ← **Head:** `dev/narugo1992` --- ### 📝 Commits (10+) - [`704cd67`](https://github.com/KohakuBlueleaf/KohakuHub/commit/704cd675624cb65945c16b8c03edffc999b270a5) add action: delete ghost repo before create new repo - [`51a544e`](https://github.com/KohakuBlueleaf/KohakuHub/commit/51a544eb2c4b52f68f22e4f31b65270b6d04b402) fix delete function in crud - [`4c6dba6`](https://github.com/KohakuBlueleaf/KohakuHub/commit/4c6dba64584e1a2ed3e4fda4515d6f32519e6ea9) Add local development bootstrap and deterministic demo fixtures - [`b5a6ab5`](https://github.com/KohakuBlueleaf/KohakuHub/commit/b5a6ab5134b6f7dd9dae0213255d0b0e6d045e8e) Add new-tab support to repo card titles - [`77786df`](https://github.com/KohakuBlueleaf/KohakuHub/commit/77786df1d940038000de1932baa18a02644dd044) Add new-tab support to commit links and breadcrumbs - [`e53f48b`](https://github.com/KohakuBlueleaf/KohakuHub/commit/e53f48b3f4d6a23e0997ed9ac7107416e1736036) Merge pull request #2 from deepghs/dev/click - [`98c0e02`](https://github.com/KohakuBlueleaf/KohakuHub/commit/98c0e020e914e6cb17615013286139175af8f0ba) Add session-scoped repo sort preferences and timezone-aware timestamps - [`7b29dbf`](https://github.com/KohakuBlueleaf/KohakuHub/commit/7b29dbf677f68e0010a8b3464f02a95eec716dcd) Merge pull request #3 from deepghs/dev/order - [`42b7f3f`](https://github.com/KohakuBlueleaf/KohakuHub/commit/42b7f3f1958e9c65cf10d4f15c544982bb02704f) Raise LFS keep versions UI limit to 9999 - [`82ea9d9`](https://github.com/KohakuBlueleaf/KohakuHub/commit/82ea9d927264e58fcbe7d101135749fb99d1dc21) Merge pull request #4 from deepghs/chore/lfs-keep-versions-max-9999 ### 📊 Changes **292 files changed** (+51635 additions, -2204 deletions) <details> <summary>View changed files</summary> ➕ `.coveragerc` (+21 -0) ➕ `.env.dev.example` (+51 -0) ➕ `.github/workflows/fullstack-tests.yml` (+205 -0) 📝 `.gitignore` (+12 -0) ➕ `Makefile` (+150 -0) ➕ `codecov.yml` (+11 -0) 📝 `config-example.toml` (+2 -1) 📝 `docker-compose.example.yml` (+4 -0) ➕ `docs/development/local-dev.md` (+329 -0) 📝 `pyproject.toml` (+4 -1) 📝 `pytest.ini` (+11 -2) ➕ `scripts/dev/down_infra.sh` (+26 -0) ➕ `scripts/dev/generate_preview_test_fixtures.py` (+79 -0) ➕ `scripts/dev/init_lakefs.py` (+152 -0) ➕ `scripts/dev/reset.py` (+175 -0) ➕ `scripts/dev/reset_local_data.sh` (+68 -0) ➕ `scripts/dev/reset_local_data_direct.py` (+39 -0) ➕ `scripts/dev/run_backend.sh` (+76 -0) ➕ `scripts/dev/seed_demo_data.py` (+3291 -0) ➕ `scripts/dev/seed_shared.py` (+3 -0) _...and 80 more files_ </details> ### 📄 Description ## Overview This PR proposes upstreaming the work that has accumulated on [`deepghs/KohakuHub@dev/narugo1992`](https://github.com/deepghs/KohakuHub/tree/dev/narugo1992). It supersedes the earlier exploratory PRs [KohakuBlueleaf/KohakuHub#14](https://github.com/KohakuBlueleaf/KohakuHub/pull/14) and [KohakuBlueleaf/KohakuHub#15](https://github.com/KohakuBlueleaf/KohakuHub/pull/15), which were closed as drafts while the work consolidated. The branch has been split into focused PRs in the deepghs fork (each reviewed, tested, and merged in isolation). This PR is therefore a **single sync that bundles those already-reviewed pieces** rather than a giant unreviewed dump. | 📊 | Stat | |---|---| | Commits | 86 | | Files changed | 292 | | Insertions / Deletions | +51,635 / -2,204 | | Backend src files touched | 25 | | Frontend src files touched | 30 | | New/changed test files | 164 | | New dev scripts | 11 | | New CI workflow | 1 (`fullstack-tests.yml`) | The work clusters into seven independently-reviewable themes, listed below from highest to lowest blast radius. Every change is gated by tests; the new CI job `frontend-backend-unit-tests` runs the matrix on each push. --- ## 🧭 Theme map | Theme | What it does | Deepghs PRs | Closed issues | |---|---|---|---| | 🧪 Test infrastructure & CI | Service-backed pytest harness, fast harness, frontend Vitest suites for both UIs, coverage gate, multi-version `huggingface_hub` matrix | [#6](https://github.com/deepghs/KohakuHub/pull/6) | — | | 🤝 HuggingFace Hub compatibility | Fixes a swathe of compat regressions surfaced by `huggingface_hub`, plus a deep coverage matrix | [#18](https://github.com/deepghs/KohakuHub/pull/18), [#20](https://github.com/deepghs/KohakuHub/pull/20), [#21](https://github.com/deepghs/KohakuHub/pull/21), [#25](https://github.com/deepghs/KohakuHub/pull/25) | [#7](https://github.com/deepghs/KohakuHub/issues/7), [#8](https://github.com/deepghs/KohakuHub/issues/8), [#9](https://github.com/deepghs/KohakuHub/issues/9), [#10](https://github.com/deepghs/KohakuHub/issues/10), [#11](https://github.com/deepghs/KohakuHub/issues/11), [#12](https://github.com/deepghs/KohakuHub/issues/12), [#13](https://github.com/deepghs/KohakuHub/issues/13), [#14](https://github.com/deepghs/KohakuHub/issues/14), [#15](https://github.com/deepghs/KohakuHub/issues/15), [#16](https://github.com/deepghs/KohakuHub/issues/16), [#17](https://github.com/deepghs/KohakuHub/issues/17), [#24](https://github.com/deepghs/KohakuHub/issues/24) | | 🔍 Pure-client metadata preview | Safetensors tree view + parquet preview, no backend reads, no precomputation | [#28](https://github.com/deepghs/KohakuHub/pull/28), [#29](https://github.com/deepghs/KohakuHub/pull/29) | [#27](https://github.com/deepghs/KohakuHub/issues/27) | | 🚦 Classified-error UX | Replace generic toasts with structured `ErrorState` (gated / not-found / unavailable / private) across file-facing surfaces | [#30](https://github.com/deepghs/KohakuHub/pull/30) | [#31](https://github.com/deepghs/KohakuHub/issues/31) | | 🛠 Repo viewer & UX polish | Open-in-new-tab on titles/breadcrumbs/commits, session-scoped sort, timezone-aware times, login-on-Enter, file-list rows, LFS keep-versions cap | [#2](https://github.com/deepghs/KohakuHub/pull/2), [#3](https://github.com/deepghs/KohakuHub/pull/3), [#4](https://github.com/deepghs/KohakuHub/pull/4), [#5](https://github.com/deepghs/KohakuHub/pull/5), [#32](https://github.com/deepghs/KohakuHub/pull/32) | — | | 🧰 Local dev bootstrap | One-shot Postgres/MinIO/LakeFS infra, deterministic demo seed, reset/verify scripts, `docs/development/local-dev.md` | (initial commits + [#6](https://github.com/deepghs/KohakuHub/pull/6)) | — | | 🚑 302→403 orphan-state fix | Heal stale `_lakefs/dummy` markers on create; reorder delete to `LakeFS-then-S3` to avoid orphan-storage 302→403 | [#33](https://github.com/deepghs/KohakuHub/pull/33) | (production incident) | --- ## 🧪 Test infrastructure & CI The biggest single contribution by file count. Before this branch the repo had no automated test suite; now there is a service-backed backend harness, a fast in-process harness, frontend Vitest suites for both UIs, and a CI workflow that runs them against a real Postgres/MinIO/LakeFS stack. | 🔬 Area | What was added | |---|---| | Backend pytest | `test/conftest.py`, `test/kohakuhub/support/` (bootstrap, seed, fakes, live server, mock HF server, service state), per-module `test_*.py` and `test_*_unit.py` | | Frontend Vitest | `test/kohaku-hub-ui/` and `test/kohaku-hub-admin/`: stores, pages, utils, components, MSW setup, fixtures | | HF compat matrix | Backend tests run against `huggingface_hub` `0.20.3 / 0.30.2 / 0.36.2 / 1.0.1 / 1.6.0 / latest`, on Python 3.10 / 3.11 / 3.12 | | CI | `.github/workflows/fullstack-tests.yml` (backend + frontend + admin frontend), Codecov upload made non-fatal | | Coverage gates | `.coveragerc`, `codecov.yml`; backend defaults to 80% with per-range overrides through `Makefile` `make test-backend RANGE_DIR=...` | | Test isolation | Per-module backend baseline restore, `backend_per_test` marker for tests that need a fresh slate, no `.env` loading inside tests | > ⚠️ The backend tests **do hit real services** (Postgres/MinIO/LakeFS). `make infra-up` brings them up; CI provisions them as service containers. --- ## 🤝 HuggingFace Hub compatibility A cluster of compat bugs were reported by triaging real `huggingface_hub` calls against KohakuHub. Each was filed as an issue in the deepghs fork, fixed, and covered by tests in the matrix above. | 🔢 | Issue | Symptom | Fix | |---|---|---|---| | ① | [deepghs#7](https://github.com/deepghs/KohakuHub/issues/7) | Repo tree navigation duplicated nested paths in tree/blob/edit routes | Path normalisation in router | | ② | [deepghs#8](https://github.com/deepghs/KohakuHub/issues/8) | Commit creation API returned non-HF-compatible commit URLs | Normalise commit URL shape | | ③ | [deepghs#9](https://github.com/deepghs/KohakuHub/issues/9) | Branch / tag / refs workflows incomplete | Implement HF-compatible branches/tags endpoints | | ④ | [deepghs#10](https://github.com/deepghs/KohakuHub/issues/10) | Repo info sliced a missing commit SHA → recoverable lookup turned into 500 | Defensive SHA handling | | ⑤ | [deepghs#11](https://github.com/deepghs/KohakuHub/issues/11) | HF-compatible error responses didn't sanitise headers | Strip CR/LF before emitting | | ⑥ | [deepghs#12](https://github.com/deepghs/KohakuHub/issues/12) | Admin create-user response omitted `is_org` field | Align with admin user shape | | ⑦ | [deepghs#13](https://github.com/deepghs/KohakuHub/issues/13) | Legacy quota endpoint drifted after public/private quota split | Bring legacy endpoint back in line | | ⑧ | [deepghs#14](https://github.com/deepghs/KohakuHub/issues/14) | Config loader broke on Python 3.10 — unconditional `tomllib` import | Guarded import / `tomli` fallback | | ⑨ | [deepghs#15](https://github.com/deepghs/KohakuHub/issues/15) | Dataset viewer self-calls used the frontend base URL | Switch to `internal_base_url` | | ⑩ | [deepghs#16](https://github.com/deepghs/KohakuHub/issues/16) | Git Smart HTTP token auth helper declared `async`, used sync | Make it consistent | | ⑪ | [deepghs#17](https://github.com/deepghs/KohakuHub/issues/17) | Trending endpoint raised `NameError` on inaccessible repos | Fix variable scope in skip path | | ⑫ | [deepghs#24](https://github.com/deepghs/KohakuHub/issues/24) | `hf_hub_download` `FileMetadataError`: `/resolve` 302 lacked `X-Repo-Commit` / `Cache-Control` | Carry HF metadata headers on the 302 | Beyond bug fixes, the branch adds: | 🧬 What | Where | |---|---| | HF-compatible tree-expand metadata (last commit per entry) | `src/kohakuhub/api/repo/routers/tree.py` | | Fallback transparency to `huggingface_hub`: HEAD semantics, real-client tests, HF seeded as a global fallback in `make seed-demo` | `src/kohakuhub/api/fallback/`, `scripts/dev/seed_demo_data.py` | | `not_implemented` API surface (telemetry, etc.) returning the right HF shape rather than 500 | `src/kohakuhub/api/not_implemented.py` | | 409-conflict surfacing in the UI on create-repo ([deepghs#20](https://github.com/deepghs/KohakuHub/pull/20)) | `src/kohaku-hub-ui/src/pages/[type]s/...` | | Fallback error classification: `gated` / `not-found` / `unavailable` rather than collapsing everything to 404 ([deepghs#29](https://github.com/deepghs/KohakuHub/pull/29)) | `src/kohakuhub/api/fallback/operations.py`, `utils.py` | --- ## 🔍 Pure-client metadata preview ([deepghs#27](https://github.com/deepghs/KohakuHub/issues/27) → [deepghs#28](https://github.com/deepghs/KohakuHub/pull/28)) Adds a file-preview dialog that **runs entirely in the browser**: no backend reads, no precomputation, no metadata cache. Targets parity with HF's preview UX for safetensors and parquet specifically. | 📦 Format | What it shows | Implementation | |---|---|---| | `.safetensors` | Tree view of tensor names with chain-collapse + toggle, dtype/shape per leaf, total parameter count | `utils/safetensors.{js,test}` + `FilePreviewDialog.vue` | | `.parquet` | Schema + first N rows, page-aware row count | `utils/parquet.{js,test}` + `FilePreviewDialog.vue` | | Range fetches | Both formats only fetch the byte ranges they need (header → metadata → optional sample), so the dialog opens fast on multi-GB files | `utils/file_preview.{js,test}` | Tests live in `test/kohaku-hub-ui/utils/test_safetensors.test.js` (708 lines) and `test_parquet.test.js` — including binary fixtures under `test/kohaku-hub-ui/fixtures/previews/`. --- ## 🚦 Classified-error UX ([deepghs#30](https://github.com/deepghs/KohakuHub/pull/30), closes [deepghs#31](https://github.com/deepghs/KohakuHub/issues/31)) Standardises a single `<ErrorState>` component used wherever a file or repo could be unreachable. Replaces ad-hoc toasts that masked real causes (gated repo, 404, upstream down, private without auth). | 📍 Surface | Before | After | |---|---|---| | Blob view | "Failed to load file" | Categorised: gated / private / not-found / network / fallback-upstream-down | | Edit view | Toast | Inline `ErrorState` with retry / sign-in / request-access affordance | | Upload view | Toast | Inline `ErrorState` | | Settings | Toast | Inline `ErrorState` | | File preview dialog | Generic | Same classification, including upstream fallback errors | | Right-click on file list rows | Forced router push | Plain `<a>` anchors so middle-click / Cmd-click open in new tab ([deepghs#32](https://github.com/deepghs/KohakuHub/pull/32)) | `utils/http_errors.{js,test}` (390 lines of tests) carries the classifier so every surface uses the same rules. --- ## 🛠 Repo viewer & UX polish A sequence of small, low-risk UX improvements, each landed as its own deepghs PR: | 🪛 PR | Change | |---|---| | [deepghs#2](https://github.com/deepghs/KohakuHub/pull/2) | New-tab support on repo card titles, commit links, breadcrumbs | | [deepghs#3](https://github.com/deepghs/KohakuHub/pull/3) | Session-scoped repo sort preferences + timezone-aware timestamps | | [deepghs#4](https://github.com/deepghs/KohakuHub/pull/4) | LFS "keep versions" UI input cap raised to 9999 | | [deepghs#5](https://github.com/deepghs/KohakuHub/pull/5) | HF-compatible tree expand metadata (last commit per entry, file size restore) | | [deepghs#32](https://github.com/deepghs/KohakuHub/pull/32) | File-list rows are plain anchors so right-click / middle-click / Cmd-click work | | (small commits) | Login form submits on Enter; tone down file-list commit link styling; tighten repo branch selector width | --- ## 🧰 Local development bootstrap The repo previously assumed a hand-rolled Docker stack. The branch adds a turnkey dev workflow: | 🚀 Command | What it does | |---|---| | `make init-env` | Copy `.env.dev.example` → `.env.dev` if missing | | `make install` | Backend `pip install` + frontend `npm install` for both UIs | | `make infra-up` | Start Postgres / MinIO / LakeFS as named containers with persisted volumes | | `make seed-demo` | Run migrations + deterministic demo seed (users, orgs, repos, files, LFS objects, fallback sources) | | `make reset-and-seed` | Wipe persisted local data and reseed | | `make backend` | Run FastAPI in reload mode | | `make ui` / `make admin` | Run the two Vite frontends | | `make test-backend` / `make test-ui` / `make test-ui-admin` | Run the suites with coverage | | 📂 Key new files | |---| | `Makefile`, `.env.dev.example`, `docker-compose.example.yml` | | `scripts/dev/up_infra.sh`, `down_infra.sh`, `init_lakefs.py`, `seed_demo_data.py` (3,291 lines), `verify_seed_data.py`, `reset.py`, `reset_local_data.sh` | | `docs/development/local-dev.md` (329-line guide) | --- ## 🚑 302→403 orphan-state fix ([deepghs#33](https://github.com/deepghs/KohakuHub/pull/33)) The most recent piece. A pair of related production failures were producing a 302 redirect that resolved to a 403 on subsequent reads: | 🔥 Failure mode | Pre-fix | Post-fix | |---|---|---| | **Create after aborted create** | LakeFS refused with `storage namespace already in use` pointing at our own `_lakefs/dummy`; user could never recreate | Heal narrowly: verify LakeFS truly has no repo, sample S3 prefix, only clean up if every key is an internal marker, then retry once | | **Partial delete** | S3 cleanup ran first; if LakeFS deletion then failed, the LakeFS repo survived but its storage was wiped → 302 → 403 on next read | Delete LakeFS metadata first; S3 cleanup only follows once LakeFS no longer references it | Both are covered by regression tests (`test/kohakuhub/api/repo/routers/test_crud_unit.py`) that fail on the prior code and pass after the fix — including guardrail tests that the heal path refuses to clean a namespace containing any user object. --- ## ✅ How to validate locally ```bash # Bring up infra make infra-up # Backend tests (real Postgres/MinIO/LakeFS, ~35s for the repo router suite) make test-backend RANGE_DIR=api/repo/routers # Full backend suite make test-backend # Frontend suites make test-ui make test-ui-admin ``` CI runs the same matrix on every push. --- ## 🧷 Open issues acknowledged but not resolved here These are deliberately deferred and tracked as open issues in the deepghs fork: | Issue | Why deferred | |---|---| | [deepghs#19](https://github.com/deepghs/KohakuHub/issues/19) | Downstream library E2E (transformers / datasets / diffusers / timm) — needs a separate harness | | [deepghs#22](https://github.com/deepghs/KohakuHub/issues/22) | HF `/api/telemetry/` — protocol notes captured, decision to defer until HF stabilises the contract | | [deepghs#23](https://github.com/deepghs/KohakuHub/issues/23) | Organizations list page slowness — needs aggregation rework, not just a bug fix | | [deepghs#26](https://github.com/deepghs/KohakuHub/issues/26) | Backend audit of broad `except Exception` (168 sites) — multi-month cleanup, not a single PR | --- ## 🧾 Test plan - [x] Backend service-backed tests pass on Python 3.10/3.11/3.12 against `huggingface_hub` 0.20.3 / 0.30.2 / 0.36.2 / 1.0.1 / 1.6.0 / latest - [x] Frontend Vitest suites pass for both UIs (>80% coverage) - [x] CI green on every commit in the branch - [x] 302→403 regression suite verified to fail on pre-fix and pass on post-fix - [ ] Manual smoke: `huggingface_hub` `create_repo` → `upload_file` → `hf_hub_download` round trip against a fresh `make infra-up && make seed-demo` stack - [ ] Manual smoke: file preview opens correctly on a multi-GB safetensors and a real parquet from a seeded repo --- 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-04-30 09:29:03 -05:00

GiteaMirror closed this issue

2026-04-30 09:29:07 -05:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/KohakuHub#16