20 Commits
Author SHA1 Message Date
Vijay Janapa Reddi 713fa75c91 Report duplicate-command usage errors without a Rich exception 2026-11-05 01:24:40 +01:00
Vijay Janapa Reddi 769241d0c0 Fix vault authoring commands to use current question and review fields 2026-11-05 01:23:00 +01:00
Vijay Janapa Reddi 0b6c2e6086 Refresh vault model checksum after taxonomy path comment correction 2026-11-05 01:07:44 +01:00
Vijay Janapa Reddi 649d2954bb Refine taxonomy editorial workflow, presentation, and documentation 2026-11-04 22:03:03 +01:00
Vijay Janapa Reddi f713ec7b5c taxonomy: 0.4-draft, the second periodic review with an audit prune
878 concepts (+71 net), 2,020 relationships (+326, -25, 25 reworded), 27
redefinitions, 67 alias sets, 33 background tags, 6 moves; guides 16 roles,
24 symptoms, 7 paths; gaps: attention-FFN disaggregation closed, verifiable
ML computation and reasoning-trace monitoring declared. Coverage judgments
updated for the new concepts and a blind held-out checklist added
(OSDI/NSDI/EuroSys/ASPLOS 2026, 202 papers, 22.3% covered, 96.5% homed).

Symptoms now declare the quantity they are measured by (`quantities`), so a
symptom's own metric no longer poses as a cause; the compiler, the StaffML
loader, and the explorer accept a check or lever that touches a cause or the
quantity and reject a quantity listed in another bucket.

README and CHANGELOG record the method, the numbers, and the open author
decisions.
2026-11-02 12:23:35 +02:00
Vijay Janapa Reddi cec2230543 Add taxonomy usage dropdown and deduplicate taxonomy compiler and frontend 2026-11-01 10:15:51 +02:00
Vijay Janapa Reddi 532e793fb2 fix(taxonomy): resolve red-team and anti-reward-hack audit findings across schema, CI, and consumers 2026-10-31 23:16:49 +02:00
Vijay Janapa Reddi ae25ccccbd feat(taxonomy): enforce end-to-end LinkML single source of truth across Binder, StaffML, and site 2026-10-30 19:05:39 +02:00
Vijay Janapa Reddi 2288cf53bb feat(taxonomy): unify authored taxonomy into single canonical ml-systems-taxonomy.yml 2026-10-30 14:42:49 +02:00
Vijay Janapa Reddi 1f6c0ed095 Update vault-cli codegen hash for models.py taxonomy comment 2026-10-28 08:17:04 +01:00
Vijay Janapa Reddi 983e7649e2 Consolidate ML Systems Taxonomy into root taxonomy/ directory 2026-10-28 08:02:35 +01:00
Vijay Janapa Reddi 3a158cacbc site: serve the ML Systems Taxonomy at /taxonomy/ and retire /explore/
The page moves from site/explore/ to site/taxonomy/, matching its title,
its nav label, and the identifier namespace the taxonomy already used.
/explore/ keeps unlisted noindex redirect stubs, with relative targets
so they resolve on both the live site and the dev preview subpath.

The binder now holds the folder and the public address in one place
(concept_graph.SITE_DIR and PAGE_URL), and the generated page data carries
the address so the on-page citation reads it instead of repeating it.
The dev preview rewrites links inside taxonomy/ like the other subsites.
2026-10-27 05:45:54 +01:00
Vijay Janapa Reddi e23f79a02d StaffML: ship taxonomy areas with the labels and rebuild the site bundle
The labels file now lists the 20 taxonomy areas in order (code, group,
guiding question, subjects) and each subject's definition. The committed
corpus summary is rebuilt, so every question carries its taxonomy tags;
its textbook links follow dev's renamed chapter folders.
2026-10-22 17:22:11 +02:00
Vijay Janapa Reddi 9606ef9201 StaffML: point the topic checks at the live topic registry
Checks #11 (topic exists) and #14 (prerequisite DAG) and the release
hash read vault/taxonomy.yaml, which no longer exists, so both checks
passed without checking anything. They now read
schema/taxonomy_data.yaml. Turning #11 on found one question filed under
an unregistered topic (sequence-parallelism); it moves to
model-tensor-parallelism, and its taxonomy tags already name sequence
parallelism. Tests pin that both checks fail when they should.
2026-10-22 15:12:41 +02:00
Vijay Janapa Reddi 4ea354b235 StaffML: add ML Systems Taxonomy tags to the question schema
Questions gain optional taxonomy_subject and taxonomy_concepts (identities
from shared/concepts). The model checks their format and vault check
--strict checks they exist, naming the replacement for a retired concept.
They join the content hash (CANON_VERSION 3), SQLite and D1
(questions.taxonomy_subject, question_concepts), and the site corpus, and
vault build writes the labels the site needs.
2026-10-22 14:26:54 +02:00
Vijay Janapa Reddi a70aebd249 Move authoring tooling out of the repository
Removes content-authoring scripts, figure and data generators, and their working notes, which are not needed to build, check, or release the books and sites. The vault CLI drops the matching subcommands and doctor check; docs, comments, ignore rules, and the codegen hash are updated to match what remains.
2026-09-12 11:55:22 +02:00
Vijay Janapa Reddi ade4273086 Fix suite checks and links across StaffML, labs, and companion sites
- StaffML book references use numbered chapter URLs; the topic map and
  tests follow. The draft validator writes its scorecard to the pipeline
  directory, docs point at vault/README.md, and the architecture doc
  drops dated session notes.
- Instructor course map and scale syllabus link to labs that exist, and
  model compression links use the numbered chapter URL.
- Labs: remove per-lab planning files and the test that required them.
- MLPerf EDU: formatting and workflow fixes, dataset URLs, portable
  evidence report paths, ignored run reports, and removed working notes.
- Kits social image points at an existing cover; slide deck header
  trimmed; unused images, wheels, scripts, and duplicate modules removed.
- TinyTorch installer cleanup list and gitignore trimmed; retired build
  scripts removed.
2026-09-12 09:19:31 +02:00
Vijay Janapa Reddi 762e98ad69 Replace the remaining symlinks with real files
- kits, labs, and the MLSys·im docs: _quarto.yml was a symlink to
  config/_quarto-html.yml. The website config now lives at _quarto.yml.
  The kits PDF build copies config/_quarto-pdf.yml over it for the render
  and restores it afterwards, and the CI PDF job copies instead of linking.
- Book builds: the binder CLI wrote books/_quarto.yml and books/index.qmd
  as symlinks. ConfigManager.activate_config (formerly setup_symlink) now
  writes copies, and the first line of _quarto.yml names its source, which
  status, doctor, clean, and save_latex_log read. The mapped chapter
  harness snapshots and restores both files by content. The Linux container
  job copies index.qmd, as the Windows job already did.
- StaffML: releases/latest was a symlink to the current release directory.
  releases/latest.txt now holds the version and is replaced atomically on
  publish; vault doctor, release publish, and the paper macro script read
  it.

A worktree that still holds the old books/_quarto.yml or index.qmd symlink
is safe: the next build removes the link before writing, so the source
config and volume index are never overwritten through it.
2026-09-10 16:03:37 +02:00
Vijay Janapa Reddi 57d7440671 Repair paths the book/ and publishing/ layouts left behind
Moving the sources under books/ and the toolchain under bindery/ rewrote
slash-style paths but missed paths built with Path and os.path.join, test
fixtures, and a few regexes. Several checks and scripts have been pointing
at directories that no longer exist, most of them silently.

- binder check index scanned book/ and found nothing; it now scans the 151
  book sources. The callout reference check read its prefix table from a
  toolchain quarto/config folder that does not exist and reported itself
  inert; it now reads books/config and scans 140 files. Both pass.
- The registry checks, the audit command, bib update, the EPUB baseline,
  the table layout audit, release, and doctor looked for book/tools,
  book/tests, book/binder, or book/.layout.
- Five call sites computed the repository root as books/../.., one level
  above the repository.
- The StaffML vault looked for chapter sources under book/quarto/contents,
  so its book cross-references silently fell back to slug titles and skipped
  link-checking.
- The VS Code extension located the repository by book/binder and read
  chapter lists from book/quarto/contents.
- Tests loaded Lua filters, build scripts, the LaTeX header, and a patched
  module from paths that do not exist. The binder suite went from 12
  collection errors, with no test running, to 342 passed; the 8 that still
  fail are the mlsysim import path and the URL flattening script, both
  fixed separately.
- test_content_structure and test_volume_bibliography matched config
  entries by a contents/ prefix, so "every config path exists" checked
  nothing. They now match the flat books/ paths. The structure generator
  wrote books/STRUCTURE.md rather than books/shared/STRUCTURE.md, which is
  regenerated here; the committed copy was missing the Volume III and IV
  front matter the PDF configs list.
- extract_figures.py fell back to a hard-coded personal path.
- Five documents linked to ../book/README.md or ../quarto/config/, and the
  SocratiQ bundle README described a symlink that does not exist.

A legacy quizzes script under _legacy/ still names a cwd-relative
quarto/config and is left as it is.
2026-09-10 15:13:28 +02:00
Vijay Janapa Reddi f1ffe5b6fa Rename interviews/ to staffml/ and drop the repeated prefix inside it
interviews/ held StaffML and nothing else, so the directory now carries the
product's name. Three names inside it repeated that prefix, and now line up
with vault/, vault-cli/, and paper/:

  interviews/staffml/               -> staffml/app/
  interviews/staffml-vault-worker/  -> staffml/vault-worker/
  interviews/staffml-vault-types/   -> staffml/vault-types/

Every repository path moved with it: workflow paths: triggers and variable
fallbacks, pre-commit hooks, .gitignore, contributor tooling, vault-cli path
joins, the app's tsconfig mapping to the shared types, and the docs.

Left alone on purpose:
- the published /interviews/ redirect to /staffml/ and the gh-pages subsite
  lists that protect it; those are URLs, not repository paths
- package and worker names (staffml-vault-worker, @staffml/vault-types) and
  the vitest job name, which branch protection may key on
- question YAMLs and generated track JSON, which are content-addressed.
  Three questions carry an older corruption, "recall @interviews/vault/
  questions/...", that predates this change
- the applied D1 migration 0001 and the vault CHANGELOG, which are history

codegen-hashes.txt is re-recorded; the three pinned artifacts changed only in
path strings. vault-cli: 97 passed, codegen --check clean.

interviewer.py's __main__ loader was one parent short and pointed at a
corpus.json that never existed; it now resolves staffml/app/src/data/
corpus.json. tools/validate_playbook.py and generate_interview_flashcard.py
still expect per-track directories that no longer exist; they point at the
new root but remain dead.

The GitHub repository variables STAFFML_ROOT, VAULT_DIR, and VAULT_CLI_DIR
still name the old paths and must change when this reaches origin.
2026-09-10 14:48:04 +02:00