A bold lead-in paragraph (e.g. **Step 2**: ...) immediately followed by a
list item with no blank line between them is parsed by Pandoc as one
paragraph, so the list marker can render inline. Add a markup check
(check markup --scope list-spacing) that flags this, extend the
format lists fixer to insert the blank line for bold lead-ins (not just
colon-terminated lead-ins), and cover both with tests.
Some servers and doi.org (DataCite DOIs, e.g. NIST) answer HEAD with 404 but
GET with 200. Retry GET before declaring a locator dead. Cuts false positives
from 33 to 17 true dead 404s; timeouts/DNS now warn (could-not-confirm) rather
than error.
binder check bib --scope links resolves every doi/url in the shared bib
(MIT Press: confirm all URLs). HTTP 404/410 = error (dead); timeout/DNS/5xx
= warning (could-not-confirm); 403/401 = warning (bot-block). Network-bound,
default=False so it is not a pre-commit gate. Found 21 dead 404 locators.
Two new default-on, errors-only bib check scopes (native in Binder):
- key-year: key year vs entry year (>1yr gap = silent citation corruption).
Errors-only subset of key-content, so the intentional surname WARNINGs stay
opt-in and do not block commits. Skips standards-numbered keys.
- duplicate-keys: every citekey must be unique (bibtex-tidy only warned;
the openai2023gpt4 collision slipped through). Both auto-join the curated
'binder check bib' set -> wired to the book-check-bib pre-commit hook.
The key-content regex ^([a-z]+)(\d{4}) read the first 4 digits of a standard
number as the entry year (iso14040 -> 1404), firing 5 false-positive ERRORs on
ISO/IEC/IEEE/NIST entries. Add a STANDARDS_KEY_PREFIXES skip. Verified it skips
exactly 7 genuine standards entries and no real author.
Add native Binder checks for LEGO formatting/unit/prose contracts, extend mlsysim formatter helpers, and normalize Vol1/Vol2 LEGO output strings through typed formatters.
Validate precision-sensitive prose rendering across both volumes and promote direct math/string assembly to fmt_math, fmt_display_math, fmt_text, and domain-specific helpers.
Five war-stories (PaLM loss spikes, AlexNet two-GPU split, Google TPU,
Discord Rust rewrite, Facebook inference tax) have a third beat that is
the engineering *response* to the constraint, not the damage from a
disaster. Folding that fix into Failure mode buried the most useful beat
for a student (what the engineers actually did) and the book's core move
(a constraint forcing an architecture). Restore it as a distinct
**Resolution** beat. Disaster war-stories, whose third beat is damage,
stay folded to Context / Failure mode / Systems lesson.
Teach the callout-schema check to accept both war-story archetypes:
Context / Failure mode / Systems lesson, or with an optional Resolution.
New 'markup --scope callout-schema' enforces a uniform bold-label
structure per callout type: every .callout-war-story must be
Context / Failure mode / Systems lesson and every .callout-definition a
single triple-bold term head. Runs in pre-commit and *-validate-dev CI
like the sibling 'callouts' scope. Registered default=False until vol2
narrative callouts are normalized to the same schema (17 still flagged
there); vol1 passes clean.
Adds 'binder check cli --scope binder-canonical' (default=yes): parses
.pre-commit-config.yaml and fails if any local hook scoped to book/quarto/
contents/ calls a raw script instead of ./book/binder. Repo-wide guards, CI
hygiene, third-party hooks, and other subprojects are not book content and
are not inspected; an explicit ALLOWLIST covers any justified exception.
This makes Binder-as-single-front-door a mechanically enforced invariant
instead of a convention — the day someone wires an off-Binder book hook (as
book-check-lego-units once was), the commit fails with an actionable fix.
Runs via the existing always-run book-check-cli-contract hook; no new hook.
Also hardens the cli-contract 'check cli help' case to assert on stable
tokens (group header + scope names) rather than the note phrase, which Rich
wraps once a group gains a scope. Extends it to cover the new scope.
Body prose forbids contractions as a deliberate register override of the copy
editor's permissive default (mit-press-editorial.md §10.11). Two improvements:
- LLM-actionable: emit suggestion '<contraction> → <full form>' from an
expansion map (can't → cannot, it's → it is, …), case-preserving at sentence
start.
- §10.11 exception: mask inline code and double-quoted spans before matching so
contractions in quoted speech / direct dialogue are not flagged.
Automated fixers consume 'binder check <group> --json', so every issue must
carry the exact fix. Populate ValidationIssue.suggestion as '<wrong> → <right>'
for both percent checks:
- percent_in_table: '50 percent → 50%'
- percent_in_prose: '98.5% → 98.5 percent'; range hits get a judgment note
('30–50% → (range) X to Y percent') instead of a misleading per-value fix.
percent_prose.Hit now carries an explicit replacement + is_range flag.
Documented the contract (file/line/code/suggestion/context) in the
ValidateCommand 'adding a check' docstring so future checks inherit it.
The _run_mitpress_* prefix already marks checks that trace to the MIT Press
copyedit (em-dash, slash, vs., e.g./i.e., percent-in-captions, etc.). Extend
it to six more that derive from the copy editor's style sheet / AU query
categories but were not yet labeled, so provenance is visible in the check
registry:
_run_abbreviation_first_use -> _run_mitpress_abbreviation_first_use (Cat F + abbrev list)
_run_concept_term_capitalization -> _run_mitpress_concept_term_capitalization (Cat G + glossary cap)
_run_percent_in_prose -> _run_mitpress_percent_in_prose (style sheet: spell out %)
_run_percent_in_tables -> _run_mitpress_percent_in_tables (style sheet: % kept in tables)
_run_latin_running_text -> _run_mitpress_latin_running_text (spell out e.g./i.e./viz.)
_run_compound_prefix -> _run_mitpress_compound_prefix (close up non-/re-/pre-)
Scope names (CLI surface) are unchanged, so pre-commit, CI, and the CLI
contract are unaffected; only the internal method identifiers move.
New 'percent-in-prose' scope (numbers group, default=True) flags the %
symbol where body prose should spell out 'percent', the symmetric inverse of
percent-in-tables. Per MIT Press AU response Category H / Chicago §9.18,
running prose (body, caption sentences, alt text) uses the word; % is for
tables, equations, code, and in-figure data.
Detection (cli/checks/percent_prose.py) reuses the pipe-table walk to exclude
tables and exempts every legitimate non-prose context: code, math, Quarto
attributes ({width=100%}), link/image targets and URLs, <style> CSS, HTML
comments, caption/alt lines, and quoted material (a % inside double quotes is
a verbatim quotation). No auto-fixer: prose ranges need judgment.
Tables invert the prose convention and use the % symbol, not the word
'percent'. This adds enforcement on both sides, sharing one implementation
in cli/checks/percent_tables.py (matching the cli/checks/currency_style.py
pattern) so the fixer fixes exactly what the check flags:
- check: 'percent-in-tables' scope in the numbers group (validate.py),
default=True → runs in book-check-numbers pre-commit + validate-dev
CI automatically.
- fix: native 'percent-tables' target in the format command
(./book/binder format percent-tables), wired as the
book-format-percent-tables pre-commit hook ahead of prettify so
columns realign after the substitution shortens cells.
Detection reuses the header+separator pipe-table walk shared by the other
table checks, gated by a number-token before 'percent' so header labels and
'percentage points / pp' are never touched.
Introduce 13 LaTeX-typeset algorithm blocks across 8 chapters (Adam update,
gradient checkpointing, FlashAttention, backpropagation, mini-batch SGD,
reverse-mode AD, PTQ calibration, tiled GEMM, ring all-reduce, 1F1B pipeline,
continuous-batching scheduler, speculative decoding, power-of-two routing),
each integrated into existing prose with a motivating lead-in and a
systems-consequence follow-up.
Extension (book/quarto/_extensions/mlsysbook-ext/pseudocode):
- Vendored under the owned mlsysbook-ext/ namespace (moved from leovan/,
version -mlsysbook-custom) and wired into the html, pdf, and epub filters.
- MIT Press cross-ref casing: @algo- -> 'algorithm N' mid-sentence,
@Algo- -> 'Algorithm N' at sentence start.
- Right-aligned italic triangle comments (CLRS column convention).
- Static EPUB renderer (upstream supports only HTML via pseudocode.js and PDF
via algpseudocodex; EPUB previously emitted raw \begin{algorithm}).
- Uses the algo- prefix: native @alg- collides with Quarto's algorithm theorem
environment and FATALs the build.
Ref tooling (book/cli/commands): teach validate.py, bib.py, and info.py the
algo-/Algo- crossref prefix (LABEL_REF_PATTERN, EXCLUDED_CITATION_PREFIXES,
Algorithm label-def patterns, and a chunk-option label harvest that runs inside
the pseudocode fence) so check refs and check labels resolve @algo-/@Algo-
instead of misreading them as missing citations.
Verified rendering in HTML, PDF, and EPUB; check refs and check labels pass
book-wide.
A plain-prose colon introduces a single clause, which MIT Press lowercases
(Chicago 17th 6.61 + 8.180), so the cross-ref prefix is mid-sentence: @sec-.
A colon after a bold run-in label (**Setup**:) opens a sentence and keeps the
capital. Previously every colon forced a capital, matching the engineering-
textbook habit the copyedit rejected.
- check refs --scope xref-case: split the colon branch (bold-label lead-in =
sentence start; plain-prose colon = mid-sentence)
- lowercase the 7 prose-colon cross-refs the corrected check now flags
The refs --scope xref-case check previously only caught the forward
direction (lowercase @sec-/@fig-/... at a sentence start should be
capitalized). Extend it to the reverse direction documented in
cross-references.md -> Prefix Casing: a capitalized @Sec-/@Tbl-/...
after a comma, semicolon, open paren, or a lowercase word renders a
stray capital mid-sentence and must be lowercased.
Both directions now share one is_sentence_start / is_mid_sentence
computation so they stay consistent. Also adds @alg- to the prefix
set (forward-compat; no algorithm refs in the corpus yet) and fixes
a forward-direction blind spot where a standalone \index{} line
before a line-start ref was misread as ongoing prose. Mid-sentence
classification is deliberately conservative (em-dash / heading /
preceding-acronym boundaries are left unflagged) to avoid false
positives.
Introduce semantic value-kind formatters (fmt_percent with 0-1 ratio
guard, fmt_pp, fmt_multiple, fmt_count, fmt_ratio) plus a universal
finite guard, moving away from generic fmt(prefix=/suffix=). Add the
fmt_semantic_suffix checker (forbids %/percent/x/scale glyphs in
suffix=) wired as an opt-in math scope, update math_canonical messaging
to the new style= API, and land the AST audit tooling.
Foundation for the corpus-wide formatting migration; no .qmd changes yet.
Currency was the only fmt_* value-kind without its own helper, so every
dollar amount carried a hand-escaped prefix="\$" -- the exact pattern that
silently breaks Pandoc/LaTeX when the escape is forgotten. Introduce
fmt_usd() (the currency sibling of fmt_percent): it owns the escaped \$,
the ~ approximation marker, and whole-dollar rounding, and never emits a
literal "USD".
- mlsysim.fmt: add fmt_usd; fmt_qty now refuses currency units (was a
dormant " USD" leak); export fmt_usd from mlsysim.
- Migrate all 271 currency call sites across both volumes to fmt_usd.
- currency_style check: forbid $ in any prefix=/suffix= and enforce the
fmt_usd path; register fmt_usd as a canonical helper in math_canonical.
- units.py: document dollar (magnitude) vs fmt_usd (display) split.
- Restore canonical $\times$ math notation in the vol2 glossary.
Validated: numbers/currency/math/code checks clean; 22 fmt + 9 checker +
499 mlsysim tests pass; 4 currency-heavy chapters render with $ output
and no leaked \$ or visible USD.
Replace pointers to the private project rules/docs tree (relative .claude/rules
and .claude/docs paths) in code comments and docstrings with neutral phrasing
("the project prose style guide", etc.). Load-bearing runtime paths that the
tooling reads or writes are left intact.
Unnumbered .column-margin figures (Tufte marginalia) injected at concept-
introduction moments: hierarchy ladders, D·A·M locators, scale-anchor knees,
iron-law bars, roofline, blast-radius fans, sparklines, and labeled taxonomies.
Every data-bearing figure pulls its magnitudes live from mlsysim (SSOT) via a
per-chapter generator; schematic devices carry no numbers. Captions are added
only where the margin is empty and clear of footnotes.
Also exempt .column-margin images from the figure-label-required check: margin
illustrations are intentionally unnumbered and never @fig-referenced
(figure-margin.md §1/§8), so they must not require a {#fig-X} label.
vol1: ml_systems, data_engineering, training, hw_acceleration, model_serving,
ml_ops, responsible_engr, benchmarking.
vol2: compute_infrastructure, network_fabrics, distributed_training,
fault_tolerance, conclusion.
