Normalize book prose/style issues across touched chapters and remove remaining structural warnings so validation output is clean and reproducible in CI. Also tighten inline/times-spacing validation behavior to reduce noisy false positives while preserving strict checks.
Converts numerous inline TikZ diagrams to external SVG files across the book's content. This improves rendering performance, streamlines figure management, and ensures consistent visual presentation.
Enhances CLI validation by:
- Ignoring cross-reference IDs when checking for multiplication to prevent false positives.
- Stripping inline math spans before currency checks to avoid misinterpreting mathematical expressions as currency.
- Applying hex literal exclusions to pre-processed lines for more accurate validation.
Adds optional Matplotlib import to the plotting module for improved flexibility in environments where the library may not be available.
Moves the mlsysim package from book/quarto/mlsysim/ to the repo root
so it is importable as a proper top-level package across the codebase.
Key changes:
- mlsysim/fmt.py: new top-level module for all formatting helpers (fmt,
sci, check, md_math, fmt_full, fmt_split, etc.), moved out of viz/
- mlsysim/viz/__init__.py: now exports only plot utilities; dashboard.py
(marimo-only) is no longer wildcard-exported and must be imported
explicitly by marimo labs
- mlsysim/__init__.py: added `from . import fmt` and `from .core import
constants`; removed broken `from .viz import plots as viz` alias
- execute-env.yml: fixed PYTHONPATH from "../../.." to "../.." so
chapters resolve to repo root, not parent of repo
- 51 QMD files: updated `from mlsysim.viz import <fmt-fns>` to
`from mlsysim.fmt import <fmt-fns>`
- book/quarto/mlsys/: legacy shadow package contents cleaned up;
stub __init__.py remains for backward compat
- All Vol1 and Vol2 chapters verified to build with `binder build pdf`
Refactors chapter discovery across CLI commands to use a single, canonical source of truth: the volume's Quarto PDF configuration file.
Introduces a new `get_chapters_from_config` function in `core/discovery.py` that parses the `_quarto-pdf-{volume}.yml` to derive the ordered list of testable chapter stems. This ensures consistent chapter order for `build` and `debug` operations, reducing duplication and improving maintainability.
Updates `build.py` and `debug.py` to delegate all chapter list retrieval to this new centralized method within `ChapterDiscovery`. Also enhances chapter QMD file location to support shared content paths.
Introduces `--only-from-report` and `--only-keys` arguments to the `references` validation command.
These allow re-validating only specific citation keys, either from a previous validation report or a custom list.
This significantly improves the workflow for correcting references by enabling focused re-runs and reducing validation time.
Removes the standalone `README_REFERENCE_CHECK.md` documentation, as its content is now implicitly handled by the integrated CLI help and broader documentation.
Refactors the build process to leverage shared output file resolution logic, ensuring consistency across build and debug commands.
Improves validation by streamlining bibliography handling and adding stricter citation matching.
Updates diagram dependencies and adjusts content for clarity and accuracy.
Checkpoint the branch-wide content/config revisions together with workbench enhancements so chapter rendering and developer workflows stay aligned. This captures the current validation-driven formatting and parallel build/debug improvements in one commit.
Convert all remaining lowercase 'x' used as multiplication (e.g.,
"1000x faster") to $\times$ across 17 vol2 chapters. These were
flagged by the new lowercase_x_multiplication validator check.
Simplifies the validator regex from a fragile word-list approach to a
broader pattern matching digit-x-lowercase (e.g., \dx\s+[a-z]) which
naturally excludes hardware counts (8x A100) and hex literals (0x61).
Includes the conversion script in _archive.
Convert all Unicode × (U+00D7) to LaTeX $\times$ in prose, tables, and
math contexts across both volumes. Unicode × is preserved only inside
fig-alt text for accessibility screen readers. One instance inside a
plain markdown backtick code span (frameworks.qmd) was reverted to
Unicode × since LaTeX doesn't render in code spans.
Updates validate.py with a new lowercase-x-as-multiplication check and
refines the latex_adjacent warning to distinguish _str variables (safe)
from raw inline Python. Updates validate_inline_refs.py comments to
reflect the new convention. Includes the conversion script in _archive.
Integrate figure rendering into the binder CLI so plots can be previewed
without a full Quarto build. Extracts Python code blocks with fig-* labels
from QMD files, renders them to PNG, and outputs a browsable gallery at
_output/plots/<chapter>/. Also fixes the package import chain so `binder`
works correctly as an installed entry point.
- Add book/cli/commands/render.py with RenderCommand class
- Wire into main.py with help table entry and command dispatch
- Add matplotlib>=3.7.0 to pyproject.toml dependencies
- Add book/quarto/_output/ to .gitignore
- Archive standalone render_figures.py to _archive/
Remove redundant ml_ prefix from ml_workflow chapter files and update all
Quarto config references. Consolidate custom scripts into native binder
subcommands and archive obsolete tooling.
- Default label types now include Equation (was missing from default set)
- --check-patterns now defaults to True for inline-refs
- Removed redundant --all-types from VSCode extension command
All five label types (Figure, Table, Section, Equation, Listing) are
now always checked unless explicitly filtered with --figures/--tables/etc.
Port all custom validation and maintenance scripts into the binder CLI
as native subcommands, eliminating the need for standalone scripts.
New `binder validate` subcommands (10):
- section-ids: verify all headers have {#sec-...} IDs
- forbidden-footnotes: check footnotes in tables/captions/divs
- footnotes: validate footnote refs/defs (undefined, unused, duplicate)
- figure-completeness: check figures have captions and alt-text
- figure-placement: audit figure/table proximity to first reference
- index-placement: check LaTeX \index{} placement
- render-patterns: detect problematic rendering patterns
- dropcap: validate drop cap compatibility
- part-keys: validate \part{key:...} against summaries.yml
- image-refs: validate image references exist on disk
New `binder maintain` subcommands (2):
- section-ids (add/repair/list/remove): full section ID lifecycle
- footnotes (cleanup/reorganize/remove): footnote management
Updated 11 pre-commit hooks to use binder commands instead of scripts.
Updated VSCode extension commands to use binder CLI.
All validators verified against original script output (parity confirmed).
Unifies Quarto metadata into shared base/format/volume fragments while carrying through chapter path, asset, and tooling updates to keep the repository consistent and easier to maintain.
Standardize Quarto config/style handling for HTML/EPUB volume builds, add explicit binder reset commands by format, and align QMD reference/label highlighting so structural tokens share consistent visual semantics.
- Completed full Volume 1 refactor to Safe Class Namespace pattern.
- Fixed render errors and verified all 16 chapters.
- Updated 'binder' CLI with native validation and maintenance namespaces.
- Enhanced VS Code extension with Chapter Navigator and Run History.
- Integrated 'binder validate' into pre-commit workflows.
- Widen gap between ML systems stack layers and data bar in both
margin and full-size variants so connector lines are clearly visible
- Add D·A·M Intersection Landscape Venn diagram (SVG + PNG) to the
D·A·M Taxonomy appendix with reference table and explanatory prose
- Add forward references to the Venn from Introduction and Part III
- Add learning objective for intersection mapping
- Chapter prose refinements across multiple chapters
CLI changes:
- Fix default PDF/EPUB config paths to use vol1 configs instead of
non-existent combined configs (_quarto-pdf.yml -> _quarto-pdf-vol1.yml)
ml_systems.qmd fixes:
- Fix bottleneck_str formatting (value is already a string, not a number)
- Fix malformed imports where 'from physx.formatting import fmt' was
incorrectly placed inside parentheses of another import block
- Fix Quarto cell directives placement (must come before any code)
- Fix precision=0 rounding 0.1 to 0 for energy values
- Update notation to use proper LaTeX subscripts ($D_{vol}$, $E_{tx}$, etc.)
- Update chapter flow and narrative across conclusion, data_selection,
dnn_architectures, model_compression, training, and 10+ other chapters
- Expand references.bib with additional ML systems and foundations entries
- Add new appendix content: assumptions, DAM framework, notation guide
- Update Quarto configs for PDF, HTML, and EPUB builds
- Add audit scripts for figures, tables, listings, and narrative analysis
- Add chapter flow analysis reports for systematic content review
- Include calc modules for hw_acceleration and appendix_algorithm
- Remove deprecated foreword.qmd and vol2 appendix_a.qmd
After a successful build, the flag opens the output using the system
default application (PDF viewer, browser, EPUB reader). Works with
all build paths: full, chapter, volume, and HTML-only.
Build fixes:
- Add viz.py symlink at book/quarto/ so import viz works with execute-dir: project
- Add VIDEO_1080P constants to calc/constants.py for ml_systems bandwidth calc
- Fix ml_systems bandwidth-bottleneck cell to use proper pint unit conversions
- Fix ops YAML escape sequence in fig-cap that blocked all project builds
- Remove duplicate fig-rotting-asset-curve cell and stray code block in ops
- Fix data_engineering inline math (standalone dollar on own line)
- Move fig-tail-latency-explosion out of callout box in serving (LaTeX float fix)
- Fix debug tool _get_output_path to use correct PDF filenames per volume
- Add 16 empty .bib placeholder files for Vol II chapters referenced in configs
Edits across all Vol 1 chapters (benchmarking, serving, training,
frameworks, hw_acceleration, etc.) and Vol 2 chapters. Includes
data_efficiency rename to data_selection, build CLI improvements,
and copyedit PDF config addition.
- Update ConfigManager to correctly find quarto/ directory when running
from book/ folder (expected working directory for the CLI)
- Fix data_engineering.qmd table: remove '#' prefix from 'schema version
branches' that was being parsed as a Markdown heading in PDF builds
- Parse -v/--verbose flag from command line args
- Pass verbose flag to MLSysBookCLI and BuildCommand
- Add 'Global Options' section to help output showing usage
Add --verbose / -v flag to see Quarto build output as it happens
instead of showing only a spinner. Useful for debugging build issues.
- Add verbose parameter to BuildCommand.__init__()
- Update _run_command() to stream output when verbose=True
- Use subprocess.Popen with line-by-line output streaming
Quarto book projects require index.qmd at the root level. This adds
automatic symlink management so that when building a specific volume,
the index.qmd symlink is updated to point to that volume's index file.
- Add _setup_index_symlink() method to ConfigManager
- Show index.qmd status in show_symlink_status()
- Automatically switch index symlink during volume builds
Implements two key improvements to the binder CLI:
1. **Volume flag support for chapter builds**
- Volume flags (--vol1, --vol2) now work regardless of position in args
- New build_chapters_with_volume() method routes chapters to correct config
- Auto-prefixes chapter names with volume to disambiguate (intro -> vol2/intro)
- Outputs to volume-specific directories (_build/FORMAT-vol1/, _build/FORMAT-vol2/)
2. **Smart fuzzy chapter name matching**
- Prioritizes 'starts with' matches (ops -> ops_scale.qmd)
- Falls back to 'contains' matches if no prefix match found
- Enables shorter, more intuitive chapter references
Examples:
./binder pdf intro --vol2 # Uses vol2 config, auto-disambiguates
./binder html ops --vol2 # Fuzzy matches to ops_scale.qmd
./binder pdf serving --vol1 # Uses vol1 config, finds serving.qmd
Also fixes invalid aria-current property in Quarto YAML configs that
was causing validation errors during HTML builds.
Add --vol1 and --vol2 flags for building individual volumes:
- ./binder pdf --vol1 Build Volume I as PDF
- ./binder pdf --vol2 Build Volume II as PDF
- ./binder epub --vol1 Build Volume I as EPUB
- ./binder epub --vol2 Build Volume II as EPUB
- ./binder list --vol1 List Volume I chapters only
- ./binder list --vol2 List Volume II chapters only
Add ambiguous chapter detection for chapters that exist in both
volumes (e.g., introduction, conclusion). When ambiguous, the CLI
shows an error with guidance to use vol1/ or vol2/ prefix.
Changes:
- discovery.py: Add AmbiguousChapterError, volume-aware discovery
- build.py: Add build_volume() method
- main.py: Add volume flag handling and updated help text
- Renamed vol2/advanced_intro to vol2/introduction for consistency
- Updated all scripts and configs to use vol1/ instead of core/
- Updated pre-commit config to check all contents/ not just vol1/
- Updated path references in Lua filters, Python scripts, and configs
* Restructure: Move book content to book/ subdirectory
- Move quarto/ → book/quarto/
- Move cli/ → book/cli/
- Move docker/ → book/docker/
- Move socratiQ/ → book/socratiQ/
- Move tools/ → book/tools/
- Move scripts/ → book/scripts/
- Move config/ → book/config/
- Move docs/ → book/docs/
- Move binder → book/binder
Git history fully preserved for all moved files.
Part of repository restructuring to support MLSysBook + TinyTorch.
Pre-commit hooks bypassed for this commit as paths need updating.
* Update pre-commit hooks for book/ subdirectory
- Update all quarto/ paths to book/quarto/
- Update all tools/ paths to book/tools/
- Update config/linting to book/config/linting
- Update project structure checks
Pre-commit hooks will now work with new directory structure.
* Update .gitignore for book/ subdirectory structure
- Update quarto/ paths to book/quarto/
- Update assets/ paths to book/quarto/assets/
- Maintain all existing ignore patterns
* Update GitHub workflows for book/ subdirectory
- Update all quarto/ paths to book/quarto/
- Update cli/ paths to book/cli/
- Update tools/ paths to book/tools/
- Update docker/ paths to book/docker/
- Update config/ paths to book/config/
- Maintain all workflow functionality
* Update CLI config to support book/ subdirectory
- Check for book/quarto/ path first
- Fall back to quarto/ for backward compatibility
- Maintain full CLI functionality
* Create new root and book READMEs for dual structure
- Add comprehensive root README explaining both projects
- Create book-specific README with quick start guide
- Document repository structure and navigation
- Prepare for TinyTorch integration
