Visual feedback after wiring up the lazy thumbnails:
- The container used `object-cover`, which center-cropped the
preview. Switched to `object-contain` so the full image fits
inside the square area (letterboxed when not square). Drops
the "interesting bit got cut off" surprise on tall illustrations.
- The grid-card thumbnail used a fixed 56-px wrapper, so the image
was always smaller than the card. Now the wrapper auto-stretches
to the card's inner width via `aspect-square w-full`, and the
image inside uses `w-full h-full object-contain` so the bitmap
scales up to the card edges with proper letterboxing.
- Bumped THUMB_MAX_DIM from 128 → 256 px (and quality 0.7 → 0.78).
At 128 px the rendered <img> still showed an empty halo inside
the larger grid card (a 128-px source can't fill a 200+ px slot
without being upscaled). 256 px keeps caches small (~25 KB per
thumbnail) while feeding the larger grid layout.
- List-view fixed-size mode also bumped from 28 → 48 px so the
thumbnail is actually identifiable at a glance.
- Component now exposes a no-`size`-prop "auto" mode that the
panel uses for grid view (parent picks the width); the explicit
pixel size mode is retained for list rows.
Seed:
- New `archives/gallery/flat-images.tar` + sidecar carrying all 24
Danbooru showcase images flat at the tar root. Lives next to
`bundle.tar` so a single folder visit produces a long
scroll-driven listing — the right shape for hand-checking the
IntersectionObserver lazy-load + concurrency-pool drain
behaviour. SEED_VERSION → v7.
Visual verification: drove Chromium via Playwright through the
flat-images.tar listing (list mode + grid mode + toggle off);
24/24 thumbnails load, fill the squares without cropping, and the
toggle correctly reverts every row to the placeholder icon.
UI suite remains 366/40 green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Image rows inside an indexed-tar listing now lazily decode a small
preview in place of the static placeholder icon. The work is gated
on visibility (IntersectionObserver), bandwidth-aware (per-format
strategy chain), and globally togglable (localStorage-persisted).
Strategy chain — first non-null wins:
1. jpeg-exif Range-read first 64 KB; parse JPEG markers;
extract APP1 / IFD1 thumbnail JPEG. Saves
up to 99% of bandwidth on EXIF-bearing JPEGs.
2. small-image ≤ 256 KB images: full read + canvas resize.
3. medium-image 256 KB – 5 MB images: full read + canvas
resize (head bytes from #1 are reused via
a shared ExtractionContext).
> 5 MB no strategy matches → placeholder stays.
The chain is registered in `_STRATEGIES`, so future formats
(video first-frame, etc.) plug in by appending another entry.
Pure-client end to end: no backend thumbnail service, no extra API.
Concurrency capped at 4 in flight; an LRU of 100 thumbnail blob URLs
caches across folder navigation and revokes the oldest when evicted.
A toolbar switch in the panel persists the user's choice via
localStorage (key: kohaku-tar-thumbnail-enabled). Default ON; OFF
short-circuits everything — no IO subscription, no Range read, no
cache lookup.
Failure semantics: every error path (network 5xx, decode failure,
abort) silently falls back to the placeholder. Never a thrown
render, never a toast. The user just sees the same icon they would
have seen pre-feature.
Test fixtures generated via scripts/dev/generate_preview_test_fixtures.py:
- with_exif_thumb.jpg — JPEG carrying an embedded thumbnail in
APP1 / IFD1, exercises the EXIF probe success path.
- no_exif_thumb.jpg — same visual, EXIF stripped, exercises
the parser's null-return + caller's strategy fall-through.
Coverage on the new modules:
TarMemberThumbnail.vue 100% / 100%
TarBrowserPanel.vue 99.42% / 84.72% (was 95.98%)
tar-thumbnail.js 78.98% / 75.75% (canvas-decode path
is jsdom-untestable)
Total UI suite: 363 passing across 40 files.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three real-world bugs surfaced once the modal was actually used:
1. Member download returned the entire tar instead of the slice.
The memberView wrapper used by the Download button copied
{path, name, size, sha256} but dropped .offset; downloadMember
then re-called extractMemberBytes with offset === undefined,
the resulting "bytes=undefined-..." Range header was silently
ignored by MinIO, and the response was the full archive. Fix:
include offset in memberView, cache the already-extracted
bytes so the download reuses them with no second round-trip,
and have extractMemberBytes throw a clear TypeError on shape
misuse instead of delegating to the network.
2. Parquet / safetensors preview inside an archive failed with a
CORS-shaped "Browser blocked the request" error. The cause was
handing FilePreviewDialog a `blob:` URL — hyparquet's
asyncBufferFromUrl issues HEAD + Range against the source, and
`blob:` URLs do not honour those reliably. The safetensors
path showed the same failure as a "header too large" garbage
value because the intercepted response carried wrong bytes at
offset 0. Fix: add parseSafetensorsMetadataFromBuffer +
parseParquetMetadataFromBuffer that work on the in-memory
bytes the modal already has; FilePreviewDialog now accepts a
`bytes` prop and routes to the from-buffer parsers.
3. Inner FilePreviewDialog auto-opened on prop change and stacked
over the "Open metadata preview" button, intercepting clicks.
Fix: only open it on explicit button click; clear it on member
close.
README files inside the archive (with or without an extension) now
classify as markdown / text by basename instead of falling through
to "binary". Fixes the missing icon report.
Real-browser verification with Playwright drove the actual flow:
parquet + safetensors metadata dialogs render with the seeded
columns, tensors, parameter counts and __metadata__ — covered by
new from-buffer unit tests plus the screenshots collected during
the verification run.
Danbooru picks regenerated with order:random per the user request:
24 posts spanning post-id 3.6M – 11.2M (no longer adjacent IDs),
same 16-arknights-by-rating + 8-mixed-IP layout.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Plants 24 real Danbooru posts on top of the existing remote-asset
fetch path (hashes verified at seed time, content-addressed CDN URLs):
- 16 Arknights posts, four per Danbooru rating (g/s/q/e per the
howto:rate wiki), grouped under images/arknights/<rating>/<id>.<ext>
- 8 mixed-IP posts at the same rating spread (Genshin Impact / Blue
Archive / Hololive / original) under images/misc/<ip>/<rating>/...
The stale and no-hash demo archives now also reuse a real image so
the showcase no longer carries any synthetic colour rectangles.
The make_solid_color_image_bytes helper is removed accordingly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a third pure-client preview kind alongside safetensors and
parquet. A `.tar` file lights up an archive icon only when the same
listing also carries a `<basename>.json` sidecar — bare tars stay
untreated so the icon is not noise.
Clicking the icon opens TarBrowserDialog, which fetches the sidecar
JSON via /resolve/ and walks the in-tar directory tree client-side.
Member preview is routed to the same renderer the standalone blob
page uses (image / video / audio / pdf / text / markdown), and a
member is materialized via a single Range read against the .tar URL,
mirroring the algorithm in hfutils.index.local_fetch.
Hash banner:
- mismatch (sha256 in sidecar disagrees with the .tar): warning
- empty hash + hash_lfs: info notice (no verification possible)
- otherwise: silent
Member download builds an object URL anchor with a `download`
attribute equal to the member basename so the browser's native
save flow shows the in-tar filename, not the .tar filename.
Seed adds open-media-lab/indexed-tar-showcase with five tar+sidecar
pairs covering nested navigation, ~600-entry pagination, hash
mismatch, missing-hash, and inner safetensors / parquet metadata
preview. SEED_VERSION is bumped to v6 so existing local envs are
told to `make reset-and-seed`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The dev seed (``make reset-and-seed``) and the test baseline now plant a
known set of API tokens and SSH keys per-user. Three real ed25519 keypairs
ship in source, private half included, so local SSH smoke tests against a
freshly seeded backend can sign with the matching key without generating
one.
Dev seed (``scripts/dev/seed_demo_data.py``)
- 6 named tokens spread across mai_lin / leo_park / sara_chen / ivy_ops,
with last_used set to a mix of "recent", "180+ days stale" and
"never used" so admin Credentials filters have something to bite on.
- 3 SSH keys: mai_lin holds two distinct keypairs (Workstation +
Archived MBP), leo_park holds the secondary keypair. The schema's
global UNIQUE on ``fingerprint`` is what forces three keypairs for two
users, hence the third generated keypair.
- Helpers are idempotent: re-running the seed without a reset is a
no-op for already-planted tokens / keys.
- ``SEED_VERSION`` bumped from ``v4`` → ``v5`` so existing dev databases
are flagged as ``incomplete`` and the operator is told to run
``make reset-local-data``.
- Manifest at ``hub-meta/dev/demo-seed-manifest.json`` now lists every
planted token plaintext and SSH private key — those are explicitly
test fixtures, never production credentials.
Test baseline (``test/kohakuhub/support/seed.py``)
- Mirrors the same plants for the test users (owner, member, outsider)
via the same keypair PEMs, so anything written against
``SEED_TOKENS`` / ``SEED_SSH_KEYS`` carries over to dev manually.
- Tokens go in via direct DB insert (the public API never re-emits a
fixed plaintext); SSH keys go through ``POST /api/user/keys`` so the
fingerprint computed by the production code is the canonical one.
Tests (``test/kohakuhub/api/admin/routers/test_credentials.py``)
- 8 new cases assert against the seeded baseline:
- every seed plant surfaces in the admin list,
- planted plaintexts authenticate as real Bearer tokens,
- SSH fingerprints match the constants in ``seed_credentials``,
- per-user isolation holds,
- ``unused_for_days`` filters actually pick stale rows,
- revoking a seeded token immediately kills its Bearer auth.
- ``test_ssh_keys.py``'s "no keys yet" assumption is replaced with a
baseline-aware count check so it coexists with the planted keys.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
verify_seed_data.py hardcoded EXPECTED_SEED_VERSION = "local-dev-demo-v3"
but seed_demo_data.py was bumped to v4 in the preview PR, so the
post-seed verifier would falsely fail with a version mismatch. Extract
the constant to scripts/dev/seed_shared.py and import it from both
sides so the two scripts always agree.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements issue #27 v4: file-level HF-compatible metadata preview
computed entirely in the browser via HTTP Range reads against the
existing /resolve/ 302 → presigned S3/MinIO URL. Zero new backend
preview code, zero LRU, zero precomputation, zero new DB state.
Backend (minimal CORS plumbing only):
- main.py CORSMiddleware: add `expose_headers` so browsers can read
Content-Range / X-Linked-* / X-Repo-Commit / ETag / Location off
the final 206 response that follows the /resolve/ 302.
- docker-compose.example.yml + scripts/dev/up_infra.sh: wire
`MINIO_API_CORS_ALLOW_ORIGIN` so the SPA can cross-origin Range-read
presigned targets. Configurable via `DEV_MINIO_CORS_ALLOW_ORIGIN`.
- docs/development/local-dev.md: MinIO CORS section explaining the
hard prerequisite + smoke-test probe + how to recreate the container.
Frontend:
- utils/safetensors.js (~190 LOC): pure-JS parser mirroring
huggingface_hub.parse_safetensors_file_metadata byte-for-byte
(speculative 100 KB first read, two-read fallback for fat headers,
SAFETENSORS_MAX_HEADER_LENGTH guard). Exposes parseSafetensorsMetadata
+ summarizeSafetensors.
- utils/parquet.js: thin wrapper over hyparquet's asyncBufferFromUrl +
parquetMetadataAsync with mode:"cors" + credentials:"omit" so cookies
never leak onto presigned URLs. Normalizes BigInt row counts.
- components/repo/preview/FilePreviewDialog.vue: ElDialog with
per-phase spinner text (range-head → parsing → done for safetensors,
head → footer → parsing → done for parquet), dtype/row-group tables,
and an explicit "CORS likely misconfigured" placeholder on failure.
- RepoViewer.vue: HF-style chart-line-data icon next to .safetensors
and .parquet rows; click opens the modal with the resolved /resolve/
URL for the current branch.
Tests + fixtures:
- test_files.py::test_resolve_get_302_exposes_cors_headers_for_browser_preview
pins the `Access-Control-Expose-Headers` list against regressions.
- test/kohaku-hub-ui/utils/test_safetensors.test.js: 6 cases covering
the real-HF-format fixture, dtype summary, progress phases, fat-header
fallback, oversized-header guard, and non-206 error paths.
- test/kohaku-hub-ui/utils/test_parquet.test.js: footer parse +
progress phase assertions.
- test/kohaku-hub-ui/fixtures/previews/{tiny.safetensors,tiny.parquet}:
byte-identical-to-HF fixtures produced by the real safetensors /
pyarrow libs via scripts/dev/generate_preview_test_fixtures.py
(committed so tests stay offline per AGENTS.md §5.2).
Seed:
- seed_demo_data.py: add two RemoteAsset entries for real HF-hosted
small fixtures pinned by sha256, and wire them into visible paths
(open-media-lab/vision-language-assistant-3b/fixtures/hf-tiny-random-bert.safetensors,
open-media-lab/multimodal-benchmark-suite/fixtures/hf-no-robots-test.parquet)
so the preview can be exercised against files that actually came off
huggingface.co rather than purely local pyarrow/safetensors output.
SEED_VERSION bumped to local-dev-demo-v4.
Verified end-to-end against the dev stack: safetensors parser output
on the seeded fixtures matches huggingface_hub.parse_safetensors_file_metadata
byte-for-byte on the same file (100 tensors, 126,851 params, I64=512
/ F32=126,339, metadata `{format: pt, ...}`). Browser preview modal
renders both file kinds correctly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The local demo seed now installs https://huggingface.co as a low-priority
(priority=1000) global fallback source via the admin API, so a fresh
`make seed-demo` can resolve public HF repos out-of-the-box. Bumps the
seed version to local-dev-demo-v3 and updates verify_seed_data.py to
assert the seeded source is advertised via /api/fallback-sources/available.
The creation step is idempotent: it lists global sources first and skips
the insert when a matching URL already exists.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
narugo1992andClaude Opus 4.7
narugo1992andClaude Opus 4.7