Files
Vijay Janapa Reddi a70aebd249 Move authoring tooling out of the repository
Removes content-authoring scripts, figure and data generators, and their working notes, which are not needed to build, check, or release the books and sites. The vault CLI drops the matching subcommands and doctor check; docs, comments, ignore rules, and the codegen hash are updated to match what remains.
2026-09-12 11:55:22 +02:00

5.8 KiB

binder/tools/audit — Editorial and Registry Audits

Detection tooling that scans textbook content against the house style rules and the MLSys·im registry, plus the verification scripts that back the book checks.

Fmt / notation audit (inline {python} numbers, spurious .0): see fmt/README.md.

Registry migration build (constants → zoos)

End-to-end gate for the completed mlsysim registry migration:

python3 binder/tools/audit/check_registry_migration_build.py

Individual tools:

Script Purpose
book_check_registry_sources.py Ban legacy flat constants / alias paths in QMD LEGO cells
generate_appendix_constants.py --verify Execute appendix assumption-table cells against live registries
refresh_mlsysim_constants_yamls.py --finalize Mark audit YAML inventory should_change: false
audit_mlsysim_drift.py Detect hand-coded canonical constant drift

Style scan

binder/tools/audit/
├── README.md                      — this file
├── __init__.py
├── protected_contexts.py          — LineWalker + inline span detection
├── ledger.py                      — Issue + Ledger JSON model
├── scan.py                        — scanner CLI
├── accept_list.py                 — persistent false-positive accept-list
├── accepted_fps.json              — accepted false positives
├── verify.py                      — structural verification checks
├── math_render/                   — rendered HTML/PDF math-leak audits (see math_render/README.md)
└── checks/
    ├── __init__.py
    ├── vs_period.py               — bare 'vs' → 'vs.'
    ├── compound_prefix.py         — pre-/non- close-up (strict 6-term list)
    ├── percent_symbol.py          — '%' → 'percent' in body prose
    ├── lowercase_prose_references.py  — 'Chapter 12' → 'chapter 12'
    ├── acknowledgements_spelling.py   — British → American
    ├── binary_units.py            — GiB/TiB in prose (detection only)
    └── h3_titlecase.py            — H3+ headings in title case (detection only)

The scanner is read-only. It reports issues; an editor decides each fix in context.

Persistent accept-list

accept_list.py + accepted_fps.json together record editorial verdicts on scanner false positives (proper-noun-heavy headings, named principles, legislation, after-colon CMS 8.158 caps, D·A·M/C³ taxonomy axes). After every scan, matching issues are flipped from open to accepted and tagged with the sub-rule that justifies them. Match key is (category, repo-relative file, exact before line) — if a heading is intentionally edited, its accept-list entry stops matching and the issue correctly returns to open for re-review.

# Default: accept-list applied, summary shows matched + stale counts
python3 binder/tools/audit/scan.py --scope vol1 -v

# Report every issue, ignoring the accept-list
python3 binder/tools/audit/scan.py --scope vol1 --no-accept-list -v

# Use a different accept-list file (e.g. a draft to iterate on)
python3 binder/tools/audit/scan.py --scope vol1 --accept-list /tmp/draft.json

Scan usage

All commands are from the repo root.

python3 binder/tools/audit/scan.py --scope vol2 --verbose
python3 binder/tools/audit/scan.py --scope vol1 --output vol1-ledger.json --verbose

Produces audit-ledger.json (or the path given by --output).


Check categories

Category Rule Notes
vs-period §10.10 Bare vs in prose
compound-prefix-closeup §10.8 Strict 6-term list, no extrapolation
percent-symbol §10.2 HTML attribute filter (width=N%)
lowercase-prose-references §10.4 Hand-written "Chapter 12"
acknowledgements-spelling §10.7 British → American
binary-units-in-prose §1 Detection only
h3-titlecase §10.9 Per-heading judgment required

Validation anchors

Scan times on a cold run from the repo root:

$ python3 binder/tools/audit/scan.py --scope vol1 -v
Total: 629 issues across 34 files (0.4s)

$ python3 binder/tools/audit/scan.py --scope vol2 -v
Total: 969 issues across 39 files (0.4s)

Steady-state anchor (2026-04-08)

Category vol1 open vol1 accepted vol2 open vol2 accepted
vs-period 0 0 0 0
compound-prefix-closeup 0 0 0 0
percent-symbol 0 0 0 0
lowercase-prose-references 0 0 0 0
acknowledgements-spelling 0 0 0 0
binary-units-in-prose 0 0 0 0
h3-titlecase 61 0 29 0
concept-term-capitalization 19 0 62 0
abbreviation-first-use 163 0 111 0
TOTAL 243 0 202 0

Item D uses file-level exclusions for glossary.qmd (glossaries are definitions, not first uses) and excludes the SIFT homonym (CV meaning vs fault-tolerance meaning).

Reproduce with python3 binder/tools/audit/scan.py --scope vol1 -v. Run the detector self-tests with:

PYTHONPATH=binder/tools python3 binder/tools/audit/checks/h3_titlecase.py
PYTHONPATH=binder/tools python3 binder/tools/audit/checks/concept_term_capitalization.py
PYTHONPATH=binder/tools python3 binder/tools/audit/checks/abbreviation_first_use.py

Expect 41/41 passed, 32/32 passed, and 17/17 passed.


Adversarial test coverage

  • protected_contexts.py: 14 adversarial tests covering bold definitions, callout titles, table headers, sentence starts, index entries, @-refs, citations, footnote refs, fig-cap attributes, inline code, and inline math.
  • compound_prefix.py: 21 tests covering the strict 6-term list, domain-compound preservation, acronym/proper-noun continuation, and case preservation.

Run the adversarial tests inline with each check module during development.