The linter silently missed *_unit_str = fmt(...) assignments; replace the broken
space-strip check with L014_CLOSED_FMT and allowlist 81 known cases until 8½-B.
Pre-commit lego-units now lints only staged QMDs with per-file baseline so
parallel WIP does not block atomic migration commits; full-repo baseline remains for CI.
Visual QA of all 207 margin figures surfaced figures that contradicted
their own captions/labels. Fixes:
- 6 ladder inversions where bar length taught the opposite of the data
(P99 vs mean, Image vs Text bandwidth, FHE vs AES latency, HBM vs
register energy, training vs inference memory, radio vs arithmetic).
Regenerated via margin_devices with consistent-unit / mlsysim SSOT
values.
- 5 trend/threshold figures whose shape did not match the relationship
(cloud-edge crossover, FPS collapse, burst-vs-sustained levels) plus
two caption reframes (retraining knee, training power).
- 7 segmented-bar figures whose labels were truncated to 4 chars and two
whose highlight contradicted the caption (egress, optimizer state);
regenerated readable with full labels.
- 10 passive/meta captions rewritten as declarative takeaways.
Root cause hardened in generate_margin_figures.py: _parse_number now
unit-aware (ms vs s, MB/s vs GB/s, pJ vs nJ) instead of mis-reading the
first letter of a unit as a k/M/B/T multiplier; ironbar label cap raised
from 5 chars so words are no longer chopped.
Replace pointers to the private project rules/docs tree (relative .claude/rules
and .claude/docs paths) in code comments and docstrings with neutral phrasing
("the project prose style guide", etc.). Load-bearing runtime paths that the
tooling reads or writes are left intact.
Drop absolute references to the author's private config repo (home-dir paths
into an external AIConfigs tree) from audit/newsletter tooling, and resolve the
prose-rule file in scan.py via the repo root instead of a hardcoded $HOME path
(also makes it portable across machines). No behavior change: the SHA lookup
keeps its graceful 'unknown' fallback when the file is absent.
With crossref prefixes set to lowercase, sentence-start refs need
the Quarto capitalized syntax (@Fig-, @Tbl-, @Sec-, @Eq-, @Lst-)
to render uppercase at sentence starts while staying lowercase
mid-sentence.
Applied 515 changes in vol1 (22 files) and 506 in vol2 (24 files).
Verified by rendering vol1/conclusion, vol1/introduction, and
vol2/introduction: capitalization correct in all contexts.
Also updated book/cli/commands/validate.py to recognize the
capitalized crossref prefixes in label/ref/citation checkers
(LABEL_REF_PATTERN, EXCLUDED_CITATION_PREFIXES, narrative_cite,
no_space_before_bracket).
Classification script added at book/tools/scripts/maintenance/
for future use and verification.
Codex chapter-by-chapter audit findings applied across vol1 and vol2:
prose rewrites where the cited source supported only a narrower or
adjacent claim, plus new audit tooling
(book/tools/scripts/build_citation_reference_packets.py and the
companion workflow doc).
Author-form cleanup pass on top of the audit:
- 19 narrative @key conversions where prose already named the author
(e.g., "Sambasivan et al. describe ... [@sambasivan2021]" -> narrative
"@sambasivan2021 describe ...", rendering "Sambasivan et al. (2021)
describe ...") - removes the citeproc duplication that the
manual-bracket regex did not catch.
- 4 [-@key] suppressed-author conversions for possessive eponyms
(Han et al., Vaswani et al., Patarasuk and Yuan, Linnainmaa).
- 2 [-@key] conversions where the prose names an author/entity that
matches the cited author (Patterson and Hennessy's iron law,
Google's data centers + Google whitepaper).
- 1 Horowitz footnote rewritten to narrative so the inline year
replaces a duplicated "(ISSCC 2014, ...) (Horowitz 2014)" pair.
- 1 narrative @russell2021 anchoring "As Russell argues" that was
previously a bare attribution.
- 1 pre-existing narrative fix for "Graham et al. report ..." in
collective_communication that had no cite on the line.
Pre-commit cleanups landing with this commit:
- subramanya2019diskann: add publisher = Curran Associates (NeurIPS
proceedings, per bib-check rule 3).
- Remove orphaned @misc{Wu2016} GNMT bib entry in vol2; it had no
citations and collided case-insensitively with @inproceedings{wu2016}
(the cited Quantized CNNs paper) under bibtex-tidy's normalization.
- vol1/training/training.qmd: drop the dead UtilizationGap LEGO cell
(gpu_real_tflops_*, cluster_*_tflops_*) and the dead
TrainingModels.{gpt3_gpu_years,gpt3_compute_cost}_str exports plus
their unused upstream values.
- vol2/distributed_training/distributed_training.qmd: anchor the
orphaned [^fn-parameter-server] footnote on the body-prose mention
of "parameter-server systems" rather than deleting the definition.
manual-bracket hook (book-check-refs) green at this tip.
Builds per-chapter review packets that compare concept-map entries
against the existing glossary headwords, classifying each missing
candidate by category and priority so editorial decisions
(add / already covered / too composite / methodology-formula-application
/ rename / needs review) can be made deliberately.
Outputs CSV + per-chapter Markdown packets + per-volume summary JSON
under .mlsysbook/glossary-pass/latest/. Pure stdlib + PyYAML; no LLM
calls. Pairs with .claude/rules/glossary.md.
Sync of 34 additional dev commits since the previous merge
(beb9e80ec). Conflict surface this round: 2 files / 6 blocks
(nn_computation, inference) — much smaller than round 1.
Resolution highlights:
- nn_computation: kept HEAD's canonical `_str` suffix exports and
MarkdownStr-wrapped mnist_arch_str; absorbed dev's space-around-arrow
prose style; fixed the missing `MnistFlopsCalc.inference_kb_display`
reference (which was removed as dead code earlier) to use the
existing `MNISTMemory.inference_kb_str` instead.
- inference: absorbed dev's callout-tip wrapper for "Debugging high
P99 latency"; took dev's deletion of the redundant callout-example
block (the dev callout-audit refactor extracted that material);
preserved HEAD's `_str` suffix discipline on all attribute refs
(e.g., HardwareSetupSharding.h100_mem_gb_str).
The bulk of dev's incoming changes auto-merged cleanly: 30+ commits of
callout refactors (table promotions, LEGO extraction, ID renames),
layout-check tooling, pre-commit autoformat sweep, and various polish
passes.
audit_math_canonical at the synced tip: 0 violations.
Every other vol1 chapter folder name matches its qmd slug; this aligns the
last outlier so folder == qmd stem across the volume. Also matches the slides
folder name (slides/vol1/10_model_compression).
- git mv contents/vol1/optimizations → contents/vol1/model_compression
- Update path refs in 4 quarto configs (html, pdf, pdf-copyedit, epub)
- Update path refs in index_prune_candidates.yml, build_locator_bins.py,
format_tables.py, learning_objectives_bolding_parallel.sh
- Update chapter-id refs (vol1/optimizations → vol1/model_compression) in
vol2 quiz integration strings (ops_scale, robust_ai, sustainable_ai)
- Update CHAPTER_DIRS, READING_ORDER, and stale "outlier" docstrings in
fix_abbreviations.py, build_prior_vocab.py, build_audit_context.py,
generate_quizzes.py (fallback logic kept as defensive code)
- Rename _audit/optimizations_audit.json → model_compression_audit.json
and fix its stale source_file/chapter fields
- Update vol1/README.md chapter table
These two patterns lived in both book/cli/commands/validate.py and the
legacy book/tools/scripts/maintenance/validate_inline_refs.py. They were
created to flag inline `{python}` substitutions inside `$..$` math mode
that would silently corrupt commas/decimals via Quarto's auto-escape.
Both checks carried an explicit `_str`-variable exemption (negative
lookahead `(?!\w+(?:\.\w+)?_str)`) in the regex — the convention assumed
pre-formatted `_str` substitutions were safe in math contexts. That
assumption was wrong: Quarto escapes the substituted text regardless of
the variable's Python type, and the exemption let 639 latent corruption
sites slip through (catalog in .claude/_reviews/math_audit_2026-05-19/).
The previous commit (1022ffc2c0) eliminated the underlying bug class by
making fmt() return a MarkdownStr that renders verbatim through Quarto's
markdown-detection path. With the bug class gone, these checks generate
no signal and add noise to the validator. Removing them.
71/71 binder checks continue to pass.
Pre-commit's check-internal-links hook flagged book/tools/scripts/README.md:27
referencing glossary/README.md, which dev removed in 5706f2c9ba
("Retire glossary JSON pipeline"). Drops the orphan row from the
subfolder-docs table.
Three new native binder scopes (`tables.caption-required`,
`figures.label-required`, `listings.caption-required`) catch the
silent gap PR #1744 surfaced: pipe tables in body prose with no
`: caption {#tbl-X}` line, markdown-image figures with no
`{#fig-X}` label, and `::: {#lst-X ...}` listing divs without
`lst-cap`. All three feed into the existing pre-commit hooks
(`book-check-tables`, `book-check-figures`, new `book-check-listings`)
and skip tables nested inside `::: {.callout-*}` divs, frontmatter,
and backmatter, matching the surrounding editorial conventions.
Pre-existing corpus violations (48 tables, 207 figures, 0 listings)
are grandfathered via book/tools/audit/baselines/captions_baseline.json
following the bib_lint_baseline.json pattern: per-file count budgets
that block only NEW violations beyond the recorded budget. Regenerate
with book/tools/scripts/maintenance/regen_captions_baseline.py after
shrinking the debt.
Removed orchestration code, prompt templates, and one-off LLM scripts
that were either zero-referenced, self-referencing, or already retired:
book/tools/audit/
- subagent_lane.py + subagent_prompts/sentence_case_h3.md
(audit-driver scaffolding, no external callers)
book/tools/scripts/
- gemini_review.py, figure_audit_gemini.py
- testing/gemini_math_check.py
(zero external references; one-off audit utilities)
scripts/cross-references/
- agent-playbook.md (prompt template; only referenced by one
print line in audit_crossrefs.py, scrubbed)
periodic-table/scripts/archive/
- entire directory (14 files): iteration-loop scripts and
panel-debate runners used to produce v0.2 of the periodic
table. README labeled it research provenance only; not part
of any active build.
No CI workflows, Makefiles, or active scripts touch any of the
removed files. Net: -2,523 lines.
Branch added bib_mitpress_openai.py at the repo root, but the convention
for these standalone bibliography utilities is book/tools/scripts/ —
that's where the predecessor it replaces (bib_verify_gemini.py) lived,
and where its sibling tools (check_bib_boundaries.py,
check_bib_qmd_integrity.py, figure_audit_gemini.py, etc.) live.
Moved the file and updated its REPO_ROOT computation from
'Path(__file__).resolve().parent' (which only worked at repo root) to
'Path(__file__).resolve().parents[3]' (correct for the new location).
Smoke-tested with --help; runs cleanly. Nothing else in the repo
references this script by path.
The bib/qmd integrity check now enforces a static scope mapping (vol1
qmds resolve only against vol1's bib, vol2 against vol2's, companion
papers against their own) so cross-volume citation leaks become a hard
failure. The pooled "any bib counts" mode is preserved behind
--no-scope-check for legacy callers.
The cite-extraction regex was also masking real problems: the buggy
fence-stripping pattern (matching any pair of triple-backticks) was
swallowing prose between unrelated fences when an HTML-commented figure
contained a stray ```. Anchored fences to line-start and added HTML
comment stripping. The fix surfaces 14 keys that were falsely flagged
as orphan, 5 unresolved citations, and 1 scope violation that were
hidden by the regex.
The previous scope was .qmd only, which made every paper-bib entry not
cited from a chapter look orphan even when paper.tex cited it. Adds
\cite/\citep/\citet/\citeauthor/\citeyear pattern matching for the four
paper.tex files (interviews, tinytorch, periodic-table, mlsysim).
Also fixes 6 broken links in tinytorch/paper/paper.tex caused by earlier
bib-audit deletions of keys that had year/surname mismatches. Restored
these entries with corrected keys (appel2002tiger, reddi2021widening,
pfaff2009pintos, rush2020minitorch, zhang2023dive) and updated the .tex
file to match. Replaced kannan2022astrasim with won2023astrasim2.
New script: book/tools/scripts/check_bib_qmd_integrity.py
Closes a real gap in the existing `binder check refs --scope citations`
check, which only validates `vol1/**/*.qmd` against vol1 bib and
`vol2/**/*.qmd` against vol2 bib — and skips `interviews/`,
`tinytorch/`, `mlsysim/`, `periodic-table/` entirely.
This script does the comprehensive sweep:
- Walks every `.qmd` under book/quarto/contents, interviews,
tinytorch, mlsysim, periodic-table (237 files in current tree).
- Checks every `[@key]` and narrative `@key` against the union of
all 7 references.bib files (1,378 unique keys, 1,604 entries
including duplicates).
- Reports two failure modes:
1. Unresolved citations: `[@key]` in a .qmd with no matching
bib entry anywhere. Hard error — broken link.
2. Orphan bib entries: keys defined in a bib but cited nowhere.
Soft warning — candidate for cleanup.
Filters false positives:
- Quarto cross-refs (`@sec-`, `@fig-`, `@tbl-`, etc.)
- CSS at-rules (`@media`, `@keyframes`, `@import`, ...)
- Python decorators (`@grad`, `@staticmethod`, ...)
- Math notation (`A@B` matmul, single-letter "keys" like `@B.T`)
- Negative lookbehind on `=@` / `(@` / `,@` to skip
decorator/expression contexts entirely
Current state on this branch:
qmd files scanned: 237
bib files scanned: 7
unique cited keys: 781
unique bib keys: 1,378
unresolved (broken): 0 ← every cite resolves
orphans (uncited): 597 ← 38% of the bib is unused
The 597 orphans are not blocking but are a significant maintenance
surface — every audit pass re-checks them and many are exactly the
corrupted CrossRef-mining entries the recent fix passes have been
cleaning up. A separate `--prune-orphans` pass is the natural
follow-up.
Usage:
python3 book/tools/scripts/check_bib_qmd_integrity.py
python3 book/tools/scripts/check_bib_qmd_integrity.py --json
python3 book/tools/scripts/check_bib_qmd_integrity.py --orphans-only
python3 book/tools/scripts/check_bib_qmd_integrity.py --strict
# exit 1 on orphans too
Exit codes:
0 — every cite resolves (default; orphans don't fail)
1 — at least one [@key] is missing from every bib
2 — --strict mode and orphan bib entries exist
Empirical findings from running the script against all 7 bibs
(1,646 entries) on 2026-05-03:
1. **Empty-response failures with -p PROMPT.** The CLI's yargs parser
re-tokenizes long prompts containing dash patterns and chokes,
returning empty stdout in 3-4s. Switched to `--prompt=VALUE`
form which binds tightly and keeps the parser out of the value.
2. **240s timeout was too short.** Gemini + Google Search round-trips
regularly took 5-12 minutes for 50-entry batches. Bumped to 900s
(15 min) and dropped retries to 1 — if a call times out at 15 min,
retrying is throwing good money after bad.
3. **Burst rate-limit on parallel launch.** The original "stagger every
max_parallel-th launch" pattern still fired N workers simultaneously,
triggering Gemini's per-minute rate limit. Replaced with a true
pipeline stagger: every batch waits STAGGER_SECONDS before launch,
regardless of pool size. Default 25s × 4 workers ≈ one in-flight
batch per 6s on the launch side.
4. **Batch size 25 was wasteful.** Gemini handles much larger contexts;
bumped default to 50 to halve the call count without quality loss.
5. **Added --stagger CLI flag** so users can tune stagger independently
of max_parallel.
These changes were validated end-to-end during the round-1 audit pass
(168 batches → 37 batches at the new defaults) before the script was
sidelined in favor of parallel Claude Agent invocations for the
production audit.
The genai/ folder advertised an implementation detail (AI-driven) and
held only one tool. The siblings under book/tools/scripts/ are all
named by purpose (audit/, glossary/, mit_press/, publish/, ...) — this
brings the quiz pipeline into line with that convention.
Changes:
- git mv book/tools/scripts/genai/quiz_refresh -> book/tools/scripts/quizzes
- empty genai/ parent directory removed
- 4 path-string references rewritten (filters.yml comment, three
internal docstrings/CLI examples in README/generate_quizzes/
build_audit_context)
- 2 BUILD.md + DEVELOPMENT.md tree-listing entries updated to point at
the new path
- README.md title and tagline updated (no longer "Quiz Refresh — runner";
the "quiz-refresh pattern" label is replaced with "spec-plus-runner
pattern" since the directory name now self-documents)
- generate_quizzes.py: meta["generated_by"] now writes
"quizzes/generate_quizzes.py" (was "quiz-refresh/generate_quizzes.py");
--skip-existing matcher loosened from "quiz-refresh" substring to
"generate_quizzes" so it accepts both old and new descriptors
- 33 chapter quiz JSONs and 30 _audit/*_audit.json files: the
generated_by metadata string updated to match new descriptor
- extract_anchors.py docstring: "quiz-refresh pipeline" -> "quizzes pipeline"
Verified post-rename:
- All scripts parse (ast.parse on all three .py files passes)
- validate_quiz_json.py works on sample chapters from each volume
- --skip-existing dry-run matches the new descriptor correctly
- git grep finds no remaining "genai/" or "quiz_refresh" references;
the one remaining "quiz-refresh" reference is an intentional
backwards-compat note in the --skip-existing comment block
The chapter quiz corpus and the spec at .claude/rules/quiz-generation.md
are unchanged in content — this is purely a directory + descriptor rename.
The quiz-refresh pipeline accumulated 68 .md files of session-generated
documentation (handoff plans, per-chapter audit memos, master reports,
the legacy agent brief) that have all served their purpose now that the
A-grade corpus is committed at 9913a78d7.
Removed:
- PATH_TO_A_PLAN.md (handoff plan; plan is complete)
- _audit/MASTER_REPORT.md (gpt-5.4 audit; corpus has been replaced)
- _audit/opus_improved/MASTER_REPORT.md (Opus pass summary; commit body
of 9913a78d7 already records the same facts)
- _audit/opus_improved/vol*_changes.md (33 per-chapter change logs from
the Opus improvement pass; rationale captured in 9913a78d7's commit
body, and the actual rewrites are visible via git diff against that
commit's parent)
- _reviews/*_memo.md (31 generate_quizzes.py validator memos from older
passes; superseded)
- prompts/agent_brief.md (legacy sub-agent brief; no code references it,
the canonical spec at .claude/rules/quiz-generation.md replaced it)
Kept:
- README.md (human-authored runner documentation, still the entry point
for anyone touching this directory)
- _audit/*_audit.json (31 gpt-5.4 audit JSONs; out of scope for this
cleanup, which targeted .md leftovers only)
Replace the 33 gpt-5.4-improved quiz JSONs with claude-opus-4-7-improved
versions produced by 33 parallel sub-agents calibrated against §16 of
quiz-generation.md (25 gold-standard worked examples).
Corpus state after promotion:
- 33/33 chapters pass validate_quiz_json.py (schema + anchor validation,
zero errors, zero warnings).
- 1,684 questions total across the two volumes (was 1,685; one net
deletion from a §Mathematical Foundations trivia FILL on "arithmetic
intensity" that was absorbed into a strengthened roofline MCQ).
- Self-reported grades: 2 chapters A, 31 chapters A-; every sub-agent
identified and rewrote the audit-flagged issue patterns (trivia_fill,
easy_tf, recall_only, throwaway_distractor, vague_lo,
tautological_lo, build_up_violation) in its chapter.
- Zero Option/Choice/Answer [A-D] or ([A-D]) patterns anywhere in the
corpus (§10 anti-shuffle-bug rule satisfied).
- Every MCQ distractor refuted by content, not by letter.
- Prior-vocabulary build-up enforced: questions that previously
redefined prior-chapter terms (MLP inductive bias, attention
mechanism, speculative decoding, fault vs error model, co-packaged
optics) rewritten to test application in each chapter's specific
context.
Artifacts committed alongside the quiz JSONs for traceability:
- book/tools/scripts/genai/quiz_refresh/build_audit_context.py — the
new helper that packaged each chapter's identity, prior chapters,
prior vocab, and per-section prose+questions+audit-issues bundle
into a single Markdown context file read by each sub-agent.
- _audit/opus_improved/MASTER_REPORT.md — aggregated grade distribution
and per-chapter roll-up.
- _audit/opus_improved/{vol}_{chap}_changes.md — per-chapter change log
(sections rewritten vs kept, issue patterns fixed, substantial-rework
sections, validator status) for every one of the 33 chapters.
Process note: the initial attempt to parallelize 32 agents in one wave
tripped server-side concurrency throttling; converged via retry batches
of 4 agents. The single outlier naming convention at vol1/optimizations
(stem model_compression) was handled during promotion.
Per Phase 2/3 of book/tools/scripts/genai/quiz_refresh/PATH_TO_A_PLAN.md,
with the substitution of sub-agent Opus calls for the direct Anthropic
API path (no ANTHROPIC_API_KEY was available in this session).
vol1/conclusion and vol2/conclusion both passed chapter="conclusion" to
memo_path() and the audit-output builder, so the second write clobbered
the first. The same collision hit vol1/introduction vs vol2/introduction.
Verified by reading _audit/conclusion_audit.json and
_audit/introduction_audit.json: both currently contain vol2 content
(metadata.chapter reports "vol2/conclusion" / "vol2/introduction"),
confirming that the vol1 runs were silently overwritten during the
2026-04-24 audit pass.
Fix prefixes both filenames with {vol}_:
- memo_path(chapter) -> memo_path(vol, chapter), writes
"_reviews/{vol}_{chapter}_memo.md"
- audit output writes "_audit/{vol}_{chapter}_audit.json"
Existing audit/memo files under pre-fix names remain on disk; Phase 2
will regenerate under the new names, at which point the orphaned
pre-fix files can be pruned.
Per PATH_TO_A_PLAN.md Phase 2 "also fix during this phase" bullet.
Vijay Janapa Reddi