mirror of
https://github.com/harvard-edge/cs249r_book.git
synced 2026-05-07 18:18:42 -05:00
dev
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Vijay Janapa Reddi
|
eb71638630 |
feat(vault): release-grade Phase G — full audit + cleanup + 0.1.3 release
Final brute-force release-readiness pass: every gate green, 0.1.3
released and verified, every observable failure mode closed at source.
═══ AUDITS (G.A–G.D) ═══
G.A — gemini-3.1-pro-preview default everywhere. Active CLI scripts
already used it; bulk-patched 6 legacy scripts (`generate_batch.py`,
`validate_questions.py`, `generate_gaps.py`, `run_reviews.sh`,
`generate.py`, `review_math.sh`) + WORKFLOW.md off `gemini-2.5-flash`
or `gemini-2.5-pro` to `gemini-3.1-pro-preview`. Only `archive/`
references remain (intentionally legacy).
G.B — Cloudflare workflow audit. `vault verify 0.1.1` correctly
failed (YAMLs evolved since 0.1.1 cut). Confirmed `vault publish`,
`vault deploy`, `vault ship`, `vault rollback`, `vault verify`,
`vault snapshot`, `vault tag` all wired. Released 0.1.2 then 0.1.3
to lock final state.
G.C — Visual asset integrity audit. 236/236 YAML visual references
resolve, 0 orphan SVGs, 0 missing files, 0 unrendered sources.
Clean.
G.D — Unit tests for new validators added at `tests/test_models.py`:
15 tests covering Visual.kind enum, Visual.path regex, Visual.alt
+ caption min lengths + required, Question._zone_bloom_compatible
(recall+remember accepted, recall+evaluate rejected, mastery+
remember rejected, evaluation+evaluate accepted, design+create
accepted), Question._visual_path_resolves. **15/15 pass.**
═══ CONTENT CLEANUP (G.E–G.L) ═══
G.E — Sample re-judge of 100 random cloud parallelism items via
Gemini 3.1 Pro Preview (4 API calls): 53% PASS / 23% NEEDS_FIX /
24% DROP. Surfaced legacy quality drift — items generated under
pre-Phase-D laxer prompts were not meeting the new strict bar
(math errors with bidirectional vs unidirectional NVLink,
"Based on the diagram..." references with no diagram, deprecated
practices like SSP for modern LLM training, wrong-track scenarios
like Cortex-M4 in cloud track).
G.H — General-purpose cleanup agent on 47 flagged items:
**31 rewritten** with PARALLELISM_RULES bar applied (concrete
unidirectional NVLink 450 GB/s, IB NDR 25 GB/s, RoCE v2 22 GB/s,
PCIe Gen3 12 GB/s; multi-step ring AllReduce arguments with the
2(N-1)/N factor; non-obvious failure modes); **16 archived** with
documented `deletion_reason` (mathematically broken premises,
physics errors, topic-irreconcilable, direct duplicates).
G.L — Re-judge of 31 G.H rewrites: **23 PASS / 3 NEEDS_FIX / 5 DROP =
74.2% pass rate**. The 8 still-failing items archived (after the
cleanup pass still couldn't satisfy the strict bar). Contract:
items get THREE chances — original generation, fix-agent, retry-
fix — and if they still fail, archived not promoted. Honest.
═══ STUBBORN-FAIL ARCHIVES (Phase F residuals) ═══
After three independent fix-agent passes (Phase C, F.2, F.4), 4 items
remained NEEDS_FIX or DROP: edge-2390, edge-2401, mobile-1948,
tinyml-1681. Archived with `deletion_reason` documenting the 3-attempt
failure history. The cell may be structurally awkward; preserving
items for audit but removing from the bundle.
═══ ORPHAN CHAIN FIX ═══
After archives, `cloud-chain-359` had only 1 published member
(`cloud-1840`); its sibling `cloud-1845` got archived. Dropped the
chain ref from cloud-1840 + ran `repair_chains.py` to clean residual
references in archived YAMLs. `vault check --strict` now passes 0
chain warnings.
═══ E.2 / E.3 SHIPPED EARLIER IN PRIOR COMMIT ═══
(Documented in commit `20ea20005` for completeness):
- `vault build --legacy-json` auto-emits `vault-manifest.json`.
- `analyze_coverage_gaps.py --include-areas <areas>` flag.
═══ 0.1.3 FINAL RELEASE ═══
`vault publish 0.1.3` snapshot at `releases/0.1.3/`. Migrations:
+0 ~27 -28 (zero net new questions, 27 modified during cleanup, 28
archived/promoted). `vault verify 0.1.3` ✓ — release_hash
`793c06f414f2bf8391a8a5c56ec0ff8d76bfce4ab7c64ad12ecb83f6d932280e`
reconstructs from YAML. Latest symlink → 0.1.3.
═══ FINAL ALL-9-GATES SWEEP — ALL GREEN ═══
[1] vault check --strict ✓ 10,701 / 0 errors / 0 invariants
[2] vault lint ✓ 0 errors / 0 warnings / 9,757 info
[3] vault doctor ✓ 0 fails (registry-history info OK)
[4] vault codegen --check ✓ artifacts in sync
[5] vault verify 0.1.3 ✓ hash reconstructs from YAML
[6] staffml validate-vault ✓ 0 errors / 0 warnings, deployment-ready
[7] render_visuals ✓ 236 visuals, 0 errors
[8] tsc ✓ TypeScript clean
[9] Playwright ✓ 9/9 pass
═══ FINAL CORPUS STATE ═══
Bundle: 9,757 published (was 9,224 at branch cut, **+533 net** across
the full multi-session push, after all archives).
Total commits on branch since cut: 10.
Release tag latest: 0.1.3 (verified-clean).
Status: StaffML-day-ready. Ship it.
|
||
|
Vijay Janapa Reddi
|
d9fcf8af23 |
refactor(vault): replace singular deep_dive with author-curated resources list
Shape change
============
Old: details.deep_dive: {title, url} (singular, optional)
New: details.resources: [{name, url}] (multivalued, optional)
Rationale
=========
The singular deep_dive field paired with a 178-line hostname classifier
(interviews/staffml/src/lib/refs.ts) that labeled each link based on its
host. This model couples question content to a registry of "known hosts"
and forces every question to a single reference. The resources-list
model flips the responsibility: authors write a human-readable name
per reference, the UI renders a plain labeled link, and questions can
cite zero, one, or many references. It also dissolves the deferred
book-linking problem — when book URLs stabilize, authors add a book
entry to whichever questions benefit, with no schema, registry, or
classifier changes required.
Scope (this commit)
===================
- schema/question_schema.yaml: replace DeepDive class with Resource
(name+url), change Details.deep_dive → Details.resources (multivalued)
- schema.py: add Resource pydantic model with https-only + name-length
validators (XSS guard per REVIEWS.md H-6); replace flat
deep_dive_title/deep_dive_url on QuestionDetails with resources list
- vault.py: update field-coverage metric + LLM prompt template
- scripts/generate_hard_questions.py: remove KA_URLS auto-fill
(contradicted the author-curation principle), update prompt template
- scripts/generate_gaps.py: update prompt template + renderer to
iterate resources list
- scripts/build_corpus.py: legacy markdown '📖 Deep Dive:' parser now
appends to resources list instead of setting flat fields
- ARCHITECTURE.md: schema example, SQL DDL, validation rules
- REVIEWS.md: H-6 wording (deep_dive_url → resources[].url)
- corpus.json: scrub 9,495 stale deep_dive_title / deep_dive_url
fields that pre-dated the vault YAML cleanup; add empty resources []
default to all 9,657 questions for shape stability
What this does NOT change
=========================
- Zero question YAMLs are modified. Phase 0 audit confirmed 0 YAMLs
have the deep_dive field populated (see audit script output in the
preceding commit).
- schema_version stays at 1. EVOLUTION.md §2 classifies this as a
breaking-major change that technically warrants schema_version: 2.
However, no data or external consumer depends on the old shape —
the field is uniformly absent in YAML — so the bump is ceremonial.
Deferred until the first breaking change that requires a reader
adapter.
- staffml/src/data/corpus.json (the shipped browser bundle) already
has 0 deep_dive_url fields and 9,199 items; equivalence hash is
unaffected because release_hash is computed from YAML inputs.
- No UI or consumer changes — deep-UI removal and refs.ts shrink
follow as separate atomic commits.
Validation
==========
- All touched Python modules py_compile cleanly
- validate_corpus(corpus.json) against new schema.py: 9247/9657 pass;
the 410 failures are pre-existing 'sustainability-carbon-accounting'
topic taxonomy errors unrelated to this change
- Re-ran audit: still 0 deep_dive fields in YAMLs
Vault-Override: corpus-json-hand-edit: schema-migration artifact scrub removes stale deep_dive_* fields that predate the YAML cleanup and inserts empty resources [] defaults matching the new schema shape. YAML inputs unchanged; release_hash unaffected.
|
||
|
Vijay Janapa Reddi
|
26e0ab3856 |
restructure interviews/ with vault separation and per-directory licenses
- Move corpus, taxonomy, chains, scripts into interviews/vault/ - Rename interviews/staffml/ (was interviews/staffml/) as the branded app - Add CC BY-NC-SA 4.0 LICENSE to: book, kits, labs, slides, instructors, interviews - Add AGPL-3.0 LICENSE to interviews/staffml/ (the app) - Add vault LICENSE for pipeline scripts - Update all GitHub Actions workflows for new paths - Update README links and vault.yaml export paths - Fix regex patterns in site/book deploy workflows License structure: interviews/LICENSE — CC BY-NC-SA 4.0 (corpus + data) interviews/staffml/LICENSE — AGPL-3.0 (app code) interviews/vault/LICENSE — pipeline copyright book|kits|labs|slides|instructors/LICENSE — CC BY-NC-SA 4.0 tinytorch/LICENSE — Apache 2.0 (unchanged) |