- book/quarto/mlsys/__init__.py: add repo-root sys.path injection so
mlsysim is importable when scripts run from book/quarto/ context
- book/quarto/mlsys/{constants,formulas,formatting,hardware}.py: new
compatibility shims that re-export from mlsysim.core.* and mlsysim.fmt
- mlsysim/viz/__init__.py: remove try/except for dashboard import; use
explicit "import from mlsysim.viz.dashboard" pattern instead
- .codespell-ignore-words.txt: add "covert" (legitimate security term)
- book/tools/scripts/reference_check_log.txt: delete generated artifact
- Various QMD, bib, md files: auto-formatted by pre-commit hooks
(trailing whitespace, bibtex-tidy, pipe table alignment)
Introduces `--only-from-report` and `--only-keys` arguments to the `references` validation command.
These allow re-validating only specific citation keys, either from a previous validation report or a custom list.
This significantly improves the workflow for correcting references by enabling focused re-runs and reducing validation time.
Removes the standalone `README_REFERENCE_CHECK.md` documentation, as its content is now implicitly handled by the integrated CLI help and broader documentation.
Refines book abstracts, table of contents, and diagram configurations for improved clarity and structure.
This commit enhances the descriptions of both Volume I and Volume II, emphasizing their respective focuses. It also introduces a framework decision tree to guide the selection of parallel training strategies and inference frameworks, and diagrams for visualizing hardware constraints.
- Add war story callout definition in custom-numbered-blocks.yml
- Create war story icon in all three formats (SVG, PNG, PDF) matching
the 64x64 stroke-only style used by all other callout icons
- Add war story bibliography and PDF config entry
- Add first war story ("The Quadratic Wall") in nn_architectures
- Include icon conversion utility script
Checkpoint the branch-wide content/config revisions together with workbench enhancements so chapter rendering and developer workflows stay aligned. This captures the current validation-driven formatting and parallel build/debug improvements in one commit.
Convert all remaining lowercase 'x' used as multiplication (e.g.,
"1000x faster") to $\times$ across 17 vol2 chapters. These were
flagged by the new lowercase_x_multiplication validator check.
Simplifies the validator regex from a fragile word-list approach to a
broader pattern matching digit-x-lowercase (e.g., \dx\s+[a-z]) which
naturally excludes hardware counts (8x A100) and hex literals (0x61).
Includes the conversion script in _archive.
Convert all Unicode × (U+00D7) to LaTeX $\times$ in prose, tables, and
math contexts across both volumes. Unicode × is preserved only inside
fig-alt text for accessibility screen readers. One instance inside a
plain markdown backtick code span (frameworks.qmd) was reverted to
Unicode × since LaTeX doesn't render in code spans.
Updates validate.py with a new lowercase-x-as-multiplication check and
refines the latex_adjacent warning to distinguish _str variables (safe)
from raw inline Python. Updates validate_inline_refs.py comments to
reflect the new convention. Includes the conversion script in _archive.
Integrate figure rendering into the binder CLI so plots can be previewed
without a full Quarto build. Extracts Python code blocks with fig-* labels
from QMD files, renders them to PNG, and outputs a browsable gallery at
_output/plots/<chapter>/. Also fixes the package import chain so `binder`
works correctly as an installed entry point.
- Add book/cli/commands/render.py with RenderCommand class
- Wire into main.py with help table entry and command dispatch
- Add matplotlib>=3.7.0 to pyproject.toml dependencies
- Add book/quarto/_output/ to .gitignore
- Archive standalone render_figures.py to _archive/
Remove redundant ml_ prefix from ml_workflow chapter files and update all
Quarto config references. Consolidate custom scripts into native binder
subcommands and archive obsolete tooling.
Unifies Quarto metadata into shared base/format/volume fragments while carrying through chapter path, asset, and tooling updates to keep the repository consistent and easier to maintain.
- Completed full standardization of 150+ calculation headers across all 16 Volume 1 chapters.
- Replaced legacy 'Why:' blocks with the 'Goal/Show/How' documentation pattern.
- Finalized P.I.C.O. class refactors for complex cells in frameworks and serving.
- Verified header consistency across introduction, ml_systems, training, and optimizations.
- Performed minor stabilization in book/vscode-ext logic.
Treat internal spacing changes as real formatting differences and normalize separator padding so table prettification is applied consistently. Save files before running pre-commit fixers from the extension so results match editor state.
- Completed full Volume 1 refactor to Safe Class Namespace pattern.
- Fixed render errors and verified all 16 chapters.
- Updated 'binder' CLI with native validation and maintenance namespaces.
- Enhanced VS Code extension with Chapter Navigator and Run History.
- Integrated 'binder validate' into pre-commit workflows.
- Refactored to use , , and registries.
- Expanded with devices (Coral, Jetson, NUC) and .
- Expanded with (DLRM) and family.
- Fixed registry bugs (Coral bandwidth, GPT class definition).
- All unit tests passing.
Renamed the core physics engine from to to better align with
the book's title and pedagogical goals.
New features:
- Introduced and classes in and .
- Digital twins now carry their own attributes (bandwidth, FLOPS) and
methods (ridge_point, memory_size).
- Global search and replace updated all 16 chapters to use .
- Maintained backward compatibility for legacy constants during migration.
This commit refactors the underlying Python calculation cells for Chapters 1-16
to strictly enforce mathematical consistency with the narrative.
**Key Text/Numeric Updates (For Editorial Review):**
1. **Chapter 3 (Workflow) - Edge Necessity Scenario:**
- *Change:* Increased clinic patient count from **100** to **150**.
- *Reason:* With 100 patients, the calculated upload time was ~5.5 hours, which fits within the 8-hour clinic day, contradicting the chapter's conclusion that 'Edge is Mandatory.' Increasing to 150 pushes upload time to >8 hours, mathematically validating the narrative.
2. **Chapter 1 (Introduction) - Model Drift Scenario:**
- *Change:* Reduced monthly accuracy drift rate from **8.0%** to **0.8%**.
- *Reason:* An 8% monthly drop is a catastrophic failure that would be immediately noticed. A 0.8% drop correctly models the 'silent failure' (boiling frog) scenario described in the text.
3. **Chapter 3 (Workflow) - Velocity vs Quality:**
- *Change:* Reduced 'Large Model' accuracy gain per iteration from **0.5%** to **0.15%**.
- *Reason:* The original rate caused the large model to hit 99% accuracy almost instantly, invalidating the 'Velocity is a Feature' argument. The new rate correctly models diminishing returns, allowing the faster (small) model to win.
4. **Chapter 15 (Responsible Engineering) - TCO Analysis:**
- *Verification:* Verified and stabilized the 3-year Total Cost of Ownership (TCO) calculations. Confirmed that Inference TCO (.5M) dominates Training TCO (8K) by ~40x, supporting the 'Efficiency as Responsibility' thesis.
**Technical Changes (Code Only):**
- Refactored all calculation cells to use the **P.I.C.O. (Parameters, Invariants, Calculation, Outputs)** design pattern.
- Added assertion guards (Invariants) to prevent future regressions where math contradicts prose.
- Fixed variable scope issues in Chapter 10 (Model Compression) and Chapter 15.
- Disabled false-positive linter warnings for standard LaTeX spacing.
- Fix broken cross-refs in training.qmd (em-dash parsed as part of ID)
- Remove footnote from table cell in ml_systems.qmd
- Add @tbl- references for 22 unreferenced tables across 5 files
- Comment out stale SVG prevention hook in pre-commit config
- Auto-fixes from bibtex-tidy, blank-line collapse, pipe-table prettify
- Revise prose, index entries, and references across all 16 chapters
- Update quarto config (epub/html) for vol1 and vol2
- Add render_figures.py testing script
- Update inject_parts.lua filter and header-includes.tex
- Update notation, appendix_dam, and references.bib
- Remove vol1/vol2 foreword files
- Widen gap between ML systems stack layers and data bar in both
margin and full-size variants so connector lines are clearly visible
- Add D·A·M Intersection Landscape Venn diagram (SVG + PNG) to the
D·A·M Taxonomy appendix with reference table and explanatory prose
- Add forward references to the Venn from Introduction and Part III
- Add learning objective for intersection mapping
- Chapter prose refinements across multiple chapters
Add proper equation labels ({#eq-...}) and prose references (@eq-...)
to 138 equations across 15 Volume 1 chapters following the gold-standard
pattern from serving.qmd.
Key changes:
- Label all display math equations with {#eq-kebab-case-name}
- Add @eq-name references in prose before each equation
- Equations include: Iron Law, Amdahl's Law, Roofline Model,
activation functions, backpropagation, attention mechanisms,
queuing theory, quantization, and system throughput formulas
Also includes:
- PDF formatting improvements (newpage directives for Vol 2)
- LaTeX header updates for chapter styling
- Pre-commit config and validation script updates
Fix NameError build failures in ml_systems, data_engineering, and
benchmarking chapters caused by missing imports and variables referenced
before their defining code cells.
- ml_systems: add missing Kparam and Bparam imports from physx.constants
- data_engineering: compute transfer_time_10g_md preview in setup cell,
add md_math import, add deduplication-dividend-calc cell, convert
hardcoded values to physics engine units
- benchmarking: compute BERT roofline preview values in roofline-example-calc
cell before they are referenced in narrative text, convert hardcoded
values to inline Python, condense redundant footnotes
Also includes physics engine integration improvements across all Vol 1
chapters: unit-safe conversions, inline Python for previously hardcoded
values, streamlined footnotes with cross-references, and new content
validation scripts.
All 21 Vol 1 chapters pass PDF build tests.
Replaces hardcoded numerical values with symbolic Python variables derived from defined constants and formulas.
This improves code maintainability and consistency, ensuring calculations are based on accurate and up-to-date physical values.
Ensures copyedit PDFs use the same font size as regular PDFs.
Increases float capacity for copy-edit mode to accommodate double-spacing and prevent potential layout issues.
Copies the base configuration file by first removing any existing symbolic link to prevent errors.
- Fix \index{} commands breaking rendering when placed before footnote
definitions, div openers (:::), or on same line as headings
- Add check_index_placement.py script with --fix flag to automatically
detect and fix these patterns
- Update training.qmd and data_engineering.qmd with corrected index placement
- Include other pending content and visualization updates
- Fix footnotes incorrectly breaking into definition text (appendix_machine, appendix_algorithm)
- Move forbidden footnotes out of callout divs (hw_acceleration, serving, training)
- Convert inline footnote info to inline text where appropriate
- Add fig-alt text to code-cell figures (dl_primer, hw_acceleration, serving, workflow)
- Add fig-cap and fig-alt to TikZ figures (distributed_training, introduction, edge_intelligence, networking, sustainable_ai)
- Fix hook ordering: Black formatter now runs before blank line collapse
- Move footnote definition outside TikZ div in frameworks.qmd
- Fixes fig-amortization-comparison: scales Y-axis values from 12000 to 12
to avoid LaTeX dimension limit (~16383pt)
- Fixes fig-compute-optimal-frontier: replaces problematic \fill...plot
with proper \addplot[fill=...] \closedcycle for log-scale coordinates
- Updates figure reference text to use @fig-selection-inequality
- Adds ch_data_selection.py calculation module
- Updates viz.py with new plot functions
- Various chapter updates across vol1 and vol2
Key changes:
- sci() and sci_latex() now convert Pint quantities to base units
(fixes 10^2 showing instead of 10^14 for TFLOPs values)
- Add md_frac(), md_sci(), md_math() helpers for LaTeX in Markdown()
- Update ml_systems.qmd with proper LaTeX fraction rendering
- Add freeze: false to _quarto.yml to prevent caching issues
- Update CLAUDE.md with QMD inline Python conventions
- Fix LATEX_ADJACENT issues across multiple QMD files (Unicode symbols)
- Properly preserves left/center/right alignment from grid tables
- Added --check mode for pre-commit warning
- Added book-check-grid-tables hook to warn about grid tables
- Grid tables should be converted to pipe for better inline Python support
Updated fix_bullet_spacing.py to handle both cases:
1. Add blank line BEFORE lists (intro text followed by bullet)
2. Remove blank lines BETWEEN consecutive list items
Fixed 70 issues across 17 files in vol1, vol2, and frontmatter.
- Updated fix_bullet_spacing.py with --check mode for CI validation
- Added book-fix-bullet-spacing hook to auto-fix missing blank lines
before bullet lists during commits
- Script now provides clear error messages with line numbers
Fixed 19 bullet lists across vol1 and vol2 that were missing the blank
line before the list starts. This ensures proper rendering in PDF/LaTeX.
Added fix_bullet_spacing.py utility script for automated detection
and fixing of this pattern.
Use Unicode symbols to prevent decimal point stripping:
- × instead of $\times$
- ≈ instead of $\approx$
- ≪ instead of $\ll$
- μ instead of $\mu$
- 10⁹, 10⁶ for powers
LaTeX parsing was interfering with inline Python output,
causing "5.9" to render as "59". Unicode symbols avoid this.
Verified via HTML render - all computed values correct.
