Adds the 'bib normalize' subcommand referenced by .pre-commit-config.yaml
but missing from binder until now. Runs the same three-step pipeline as
pre-commit: bib_apply_mechanical_fixes -> bibtex-tidy (loop until pass) ->
bib_lint --check. Supports full-tree or scoped (--vol1/--vol2/--path) runs.
bib_lint.py: extend §5 hygiene checks — additional journal abbreviations
(J. ACM, IEEE Trans. Comput., ACM Computing Surveys, etc.), more canonical
publishers (USENIX, IEEE Computer Society, SAGE, PLOS, Frontiers), and
heuristics for trailing-period titles, all-caps paste, Title-Case, short
booktitle without (ACRONYM), @article volume requirement, and pages prefix.
Doc reference updated from book-prose-merged.md to book-prose.md.
BINDER.md: document the bib normalize command and its relationship to the
pre-commit pipeline.
The genai/ folder advertised an implementation detail (AI-driven) and
held only one tool. The siblings under book/tools/scripts/ are all
named by purpose (audit/, glossary/, mit_press/, publish/, ...) — this
brings the quiz pipeline into line with that convention.
Changes:
- git mv book/tools/scripts/genai/quiz_refresh -> book/tools/scripts/quizzes
- empty genai/ parent directory removed
- 4 path-string references rewritten (filters.yml comment, three
internal docstrings/CLI examples in README/generate_quizzes/
build_audit_context)
- 2 BUILD.md + DEVELOPMENT.md tree-listing entries updated to point at
the new path
- README.md title and tagline updated (no longer "Quiz Refresh — runner";
the "quiz-refresh pattern" label is replaced with "spec-plus-runner
pattern" since the directory name now self-documents)
- generate_quizzes.py: meta["generated_by"] now writes
"quizzes/generate_quizzes.py" (was "quiz-refresh/generate_quizzes.py");
--skip-existing matcher loosened from "quiz-refresh" substring to
"generate_quizzes" so it accepts both old and new descriptors
- 33 chapter quiz JSONs and 30 _audit/*_audit.json files: the
generated_by metadata string updated to match new descriptor
- extract_anchors.py docstring: "quiz-refresh pipeline" -> "quizzes pipeline"
Verified post-rename:
- All scripts parse (ast.parse on all three .py files passes)
- validate_quiz_json.py works on sample chapters from each volume
- --skip-existing dry-run matches the new descriptor correctly
- git grep finds no remaining "genai/" or "quiz_refresh" references;
the one remaining "quiz-refresh" reference is an intentional
backwards-compat note in the --skip-existing comment block
The chapter quiz corpus and the spec at .claude/rules/quiz-generation.md
are unchanged in content — this is purely a directory + descriptor rename.
Aligns the repository with GitHub community-standard recommendations and
makes the project's policies discoverable from the root, where contributors
and tooling actually look for them.
- CONTRIBUTING.md (root): router to per-project guides plus universal
policies (branch from dev, ./book/binder setup, no `git add .`).
- CODE_OF_CONDUCT.md (root): canonical Contributor Covenant 2.1, lifted
from book/docs/. The book/docs version becomes a thin pointer so we
have a single source of truth.
- SECURITY.md: private vulnerability reporting via GitHub Security
Advisory + maintainer email, with explicit in-scope/out-of-scope
boundaries (textbook typos are not security issues).
- CITATION.cff: machine-readable mirror of CITATION.bib so GitHub's
"Cite this repository" button works.
- .github/dependabot.yml: weekly bumps against `dev` for every actual
ecosystem in the repo (pip, npm, github-actions), grouped where it
makes sense (Next/React together).
- .github/ISSUE_TEMPLATE/config.yml: blank_issues_enabled=false to keep
reports on-template; added security-advisory contact link.
Clean up planning, kickoff, audit, and persona-feedback documents
accumulated during prior AI-assisted work sessions. These are
session artifacts, not durable documentation — the decisions they
captured have either shipped, been retired, or are traceable via
git history.
interviews/vault/REVIEWS.md is intentionally kept: it is cited by
section ID (H-6, H-7, H-21, C-6, ...) from production code in
interviews/vault-cli/ and interviews/vault/ and published as the
pyproject.toml Review-Ledger URL, which makes it engineering
documentation rather than a session artifact.
Deletions:
- RELEASE-PREP.md, review_prompt.md (root handoff / review prompts)
- interviews/vault/KICKOFF.md, BOOK_LINKING_PLAN.md, EXPANSION_PLAN.md
- interviews/staffml/FEEDBACK_SYNTHESIS.md, V1_REDESIGN_SPEC.md,
STAFFML_UX_PLAN.md, VAULT_DESIGN_PLAN.md
- interviews/staffml/.gemini-reviews/ (2 review call logs)
- book/docs/SVG_FIGURE_AUDIT_PLAN.md, book/tools/agent_personas.md
- mlsysim/docs/WEBSITE_AUDIT.md
- periodic-table/iteration-log.md, refinement-log.md
Reference fixes for pointers into deleted files:
- interviews/vault/ARCHITECTURE.md: drop section 21 (pointed at KICKOFF.md)
- interviews/vault/schema/question_schema.yaml: drop BOOK_LINKING_PLAN.md
reference in the author-curated resource description
- interviews/staffml/src/components/Footer.tsx: drop BOOK_LINKING_PLAN.md
reference from the docstring; rationale preserved
Also removes the untracked gemini_prompts/ directory at repo root.
A previously-overlooked gap: `.pre-commit-config.yaml` has been
committed to the repo with ~60 hooks for years, but contributors who
do a fresh clone never get the framework activated unless they know
to run `pre-commit install` manually. There was no onboarding doc
pointing to this step, and the `./binder setup` command that existed
ran doctor but did not install hooks.
Symptom on my local clone: `.git/hooks/pre-commit` was a symlink to
`interviews/vault-cli/scripts/pre_commit_corpus_guard.py` rather
than the pre-commit framework dispatcher. The corpus-guard fired on
every commit (good — it protects interviews/vault/corpus.json per
ARCHITECTURE.md §11.1) but the ~60 framework hooks did not — including
the EPUB hygiene hook landed in the recent fix/epub-issues merge.
Four-part fix
-------------
1. Vault corpus-guard is wired into .pre-commit-config.yaml as a
local `always_run: true` hook pointing at the repo-tracked script.
Same protection, now delivered via the framework so a contributor
who runs `pre-commit install` (or `./binder setup`) gets BOTH the
corpus-guard AND every other hook in one shot. The bare
`.git/hooks/pre-commit` symlink is no longer needed.
2. `./binder setup` now actually installs the pre-commit hooks
(previously it just ran doctor). New `_install_pre_commit_hooks`
helper in maintenance.py:
* Locates .pre-commit-config.yaml at repo root.
* Verifies `pre-commit` is on PATH.
* Detects the core.hooksPath trap that makes `pre-commit
install` refuse, and prints the one-line unset command.
* Runs `pre-commit install` with cwd=repo_root (critical — see
below).
* Streams the framework's own confirmation lines.
3. Deleted `book/.pre-commit-config.yaml`. This was a legacy,
parallel config with broken paths (entries referenced
`tools/scripts/...` rather than `book/tools/scripts/...`).
Running `pre-commit install` from inside `book/` picked up this
orphan instead of the canonical root config, wiring a broken
hook that failed every commit with "can't open file
/Users/VJ/GitHub/MLSysBook/tools/scripts/maintenance/
cleanup_build_artifacts.py". Every hook in the orphan had a
functional equivalent under a `book-*` prefix in the top-level
config (verified by cross-check), so deletion is lossless.
4. Step 3 of book/docs/CONTRIBUTING.md is now "Set Up the Development
Environment" with a one-liner invoking `./book/binder setup`.
Downstream steps renumbered (4-8). New contributors hitting
`git commit` for the first time now see the hooks fire instead
of silently bypassing them.
Verification
------------
After `./binder setup` on a clone that had the bare corpus-guard symlink:
pre-commit installed at .git/hooks/pre-commit
(old symlink preserved at .git/hooks/pre-commit.legacy)
`pre-commit run vault-corpus-guard --all-files` → Passed
`pre-commit run book-epub-hygiene --all-files` → Passed
Both the corpus-guard and the EPUB hygiene layer now dispatch through
the same framework, and the ~60 other book-* hooks that were dormant
for anyone without manual activation are now active out of the box.
Self-contained pickup-ready plan covering all 238 SVG figures under
book/quarto/contents/. Specifies inventory + batching, render tooling
(rsvg-convert primary, inkscape fallback, optional headless Chromium
for filter/font verification), gemini-3.1-pro-preview invocation
pattern with @-image attachments and JSON schema, the iterative fix
loop (max 3 attempts per figure with auto-revert), strict guardrails
(no full-file SVG overwrites, atomic commits, forbidden paths),
resumable state file, and a final report format.
Landing page (site/index.qmd, site/landing-v3.css):
Add a small freshness indicator to the hero — pulsing green dot,
"Actively maintained · Last updated April 2026 · Release notes"
link. Communicates to first-time visitors that the textbook is a
living project, not a shipped-and-forgotten resource. Pulse uses
reduced-motion-friendly animation timing.
Subscribe modal (site/_quarto.yml):
Switch the script src from a cross-site /vol1/... URL to the local
/assets/scripts/subscribe-modal.js — the marketing site now ships
its own copy via the mirror sync (shared/scripts/sync-mirrors.sh),
so it doesn't depend on the book bundle being reachable.
Navbar breakpoint (shared/config/navbar-common.yml):
Lower navbar.collapse-below from "xl" (1200px) to "lg" (992px) so
13-15in laptops keep the full nav visible instead of collapsing
into the hamburger. Affects every subsite that includes the
shared navbar config.
Mobile QA (book/docs/MOBILE_QA.md):
Manual checklist for verifying margin sidenotes, citations,
navbar, and TOC behaviour on iOS / Android at common breakpoints.
Used as a release gate; not part of automated CI yet.
Add explicit per-check preflight logging and matrix failure instance reporting in the container build workflow, and update stale documentation links and workflow/file path references.
- Quarto 1.9.27: Linux (.deb), Windows (direct download; Scoop Extras has 1.8.27)
- R 4.5.2: Linux (CRAN jammy-cran40), Windows (Scoop main/r)
- Baremetal: quarto-actions/setup for both Linux and Windows
- Remove ggrepel version pin (R 4.5.x supports ggrepel 0.9.7)
- Update docs: BUILD.md, CONTAINER_BUILDS.md, docker READMEs
Unifies Quarto metadata into shared base/format/volume fragments while carrying through chapter path, asset, and tooling updates to keep the repository consistent and easier to maintain.
- Completed full Volume 1 refactor to Safe Class Namespace pattern.
- Fixed render errors and verified all 16 chapters.
- Updated 'binder' CLI with native validation and maintenance namespaces.
- Enhanced VS Code extension with Chapter Navigator and Run History.
- Integrated 'binder validate' into pre-commit workflows.
Replaces abbreviated 'ML Systems' with 'Machine Learning Systems' across
volume titles, part openers, frontmatter, summaries, docs, and landing page
for consistency with MIT Press submission standards.
Set up separate contributor tracking for each sub-project:
- book/.all-contributorsrc - Book content contributors
- kits/.all-contributorsrc - Hardware kit contributors
- labs/.all-contributorsrc - Lab exercise contributors
- tinytorch/.all-contributorsrc - Framework contributors
Each project now has:
- Its own .all-contributorsrc config file
- Contributors section in README with All Contributors format
- Project-specific contribution types in the recognition guide
- Cheatsheet in CONTRIBUTING.md (where applicable)
Added @AmirAlasady as first TinyTorch contributor for bug report #1122.
Usage: Comment on any issue/PR with:
@all-contributors please add @username for bug, code, doc, or ideas
Removes the learning objectives framework, part key validation system, and the publish live workflow documentation.
These files are being removed to facilitate a reorganization of the project's documentation structure and workflows.
- Remove CHANGELOG.md from repo root
- Remove changelog.qmd page from book frontmatter
- Remove changelog from navbar in HTML, PDF, and EPUB configs
- Delete changelog generation scripts (generate_release_content.py, etc.)
- Update documentation to remove changelog references
GitHub Releases now serves as the primary changelog for version updates.
The version number in the navbar already links to releases.
These files contain internal planning documents that should not be
public:
- VOLUME_SPLIT_ROADMAP.md (project timeline and MIT Press plans)
- VOLUME_SPLIT_SURGICAL_PLAN.md (section-by-section surgery)
- VOLUME_STRUCTURE_PROPOSAL.md (publisher proposal draft)
Update Volume I tagline from "Build, Optimize, Operate" to
"Build, Optimize, Deploy" across all documentation.
- "Deploy" better matches Part IV content (Serving, MLOps, Responsible Engineering)
- "Operate" implies ongoing management which is more Volume II territory
- Also fixes hw_acceleration table to use proper grid table format
This commit consolidates several structural changes:
1. Merge ondevice_learning into edge_intelligence chapter
- Add support files (bib, concepts, glossary, quizzes)
- Add images from former ondevice_learning
- Update all cross-references from @sec-ondevice-learning to @sec-edge-intelligence
2. Delete Vol II hw_acceleration chapter (merged into distributed_training)
3. Add new Serving chapter to Vol I Part IV (Deployment)
- Create placeholder serving.qmd with section structure
- Add support files for bibliography, concepts, glossary, quizzes
4. Update configs and documentation
- Update _quarto-html.yml with serving chapter in sidebar and bibliography
- Update VOLUME_STRUCTURE.md to reflect 16 chapters in Vol I
- Update About page with simplified Part names
- Fix table formatting in hw_acceleration chapter
Vol I now has 16 chapters (4 in each Part) matching Vol II structure.
Remove outdated planning documents that have been superseded by
VOLUME_STRUCTURE.md which now serves as the authoritative reference
for the two-volume textbook organization.
Removed files:
- VOLUME_SPLIT_ROADMAP.md
- VOLUME_SPLIT_SURGICAL_PLAN.md
- VOLUME_STRUCTURE_PROPOSAL.md
- reviewer-feedback-synthesis-r1.md
- volume-outline-draft.md
Change Part names from compound to single-word format:
- ML Foundations → Foundations
- System Development → Development
- Model Optimization → Optimization
- System Operations → Deployment
Creates clean pedagogical progression: Foundations → Development → Optimization → Deployment
Volume I: Introduction to ML Systems (14 chapters)
- Part I: Foundations (Ch 1-4)
- Part II: Development (Ch 5-8)
- Part III: Optimization (Ch 9-12)
- Part IV: Operations (Ch 13-14)
Volume II: Advanced ML Systems (16 chapters)
- Part I: Foundations of Scale (Ch 1-4)
- Part II: Distributed Systems (Ch 5-8)
- Part III: Production Challenges (Ch 9-12)
- Part IV: Responsible Deployment (Ch 13-16)
Structure approved for website implementation.
- Root README: Add ML-Systems bridge table, book structure, and what makes this book different
- book/README: Simplify to build instructions for contributors
- tinytorch/README: Streamline with correct clone URL and Early Access status
- Rename contribute.md to CONTRIBUTING.md for consistency
- Add MIT_PRESS_PROPOSAL.md: polished proposal for two-volume series
- Add VOLUME_SPLIT_ROADMAP.md: 6-month master project plan with progress tracking
- Add VOLUME_SPLIT_SURGICAL_PLAN.md: detailed section-by-section surgery instructions
These documents establish the complete roadmap for splitting the ML Systems
textbook into two flagship volumes for MIT Press publication.
- book/docs/VOLUME_STRUCTURE_PROPOSAL.md: Proposal for textbook volume structure
- tinytorch/docs/DISTRIBUTION_DESIGN.md: Design document for TinyTorch pip distribution
* Restructure: Move book content to book/ subdirectory
- Move quarto/ → book/quarto/
- Move cli/ → book/cli/
- Move docker/ → book/docker/
- Move socratiQ/ → book/socratiQ/
- Move tools/ → book/tools/
- Move scripts/ → book/scripts/
- Move config/ → book/config/
- Move docs/ → book/docs/
- Move binder → book/binder
Git history fully preserved for all moved files.
Part of repository restructuring to support MLSysBook + TinyTorch.
Pre-commit hooks bypassed for this commit as paths need updating.
* Update pre-commit hooks for book/ subdirectory
- Update all quarto/ paths to book/quarto/
- Update all tools/ paths to book/tools/
- Update config/linting to book/config/linting
- Update project structure checks
Pre-commit hooks will now work with new directory structure.
* Update .gitignore for book/ subdirectory structure
- Update quarto/ paths to book/quarto/
- Update assets/ paths to book/quarto/assets/
- Maintain all existing ignore patterns
* Update GitHub workflows for book/ subdirectory
- Update all quarto/ paths to book/quarto/
- Update cli/ paths to book/cli/
- Update tools/ paths to book/tools/
- Update docker/ paths to book/docker/
- Update config/ paths to book/config/
- Maintain all workflow functionality
* Update CLI config to support book/ subdirectory
- Check for book/quarto/ path first
- Fall back to quarto/ for backward compatibility
- Maintain full CLI functionality
* Create new root and book READMEs for dual structure
- Add comprehensive root README explaining both projects
- Create book-specific README with quick start guide
- Document repository structure and navigation
- Prepare for TinyTorch integration
