Commit Graph

35 Commits

Author SHA1 Message Date
Vijay Janapa Reddi
2ce322def1 LEGO updates , call out updates 2026-02-28 11:47:42 -05:00
Vijay Janapa Reddi
c8dd1782d3 Math updates 2026-02-28 08:28:51 -05:00
Vijay Janapa Reddi
acd3f59f4f Displays pre-flight build manifest in output
Introduces a detailed build manifest that appears in a dedicated output channel prior to any build or debug command execution.

The manifest provides key information about the upcoming operation, including the target volume, build format, execution mode (sequential or parallel), the Quarto configuration file in use, and a comprehensive list of all chapters slated for compilation. The chapter list is derived directly from the Quarto YML, acting as a single source of truth that reflects the full intended book structure, even for entries that are currently commented out.

Additionally, the manifest clearly displays the exact shell command that will be executed, enhancing transparency and aiding in debugging.
2026-02-27 08:09:12 -05:00
Vijay Janapa Reddi
fd21a57dd3 Update vscode-ext (debug commands, terminal) 2026-02-26 15:23:08 -05:00
Vijay Janapa Reddi
b0eb8646f5 feat(vscode-ext): colored icons in Chapter Navigator for figures, tables, equations, callouts, listings
- Use ThemeColor (charts.*) for entry icons: blue=figure, purple=table, green=listing, orange=equation, yellow=callout
- Section items use descriptionForeground for subtle hierarchy
- Add mlsysbook.navigatorVisibleEntryKinds config to toggle which content types appear in navigator
2026-02-24 11:00:32 -05:00
Vijay Janapa Reddi
bccc4d2646 Misc: quarto config, vol2 chapters, vscode-ext utils, remove site_libs clipboard 2026-02-21 19:45:46 -05:00
Vijay Janapa Reddi
b756cb7da3 fix: vscode extension activation error handling and workspace detection
- extension.ts: wrap activate() body in try/catch so activation failures
  surface in the Output channel instead of crashing silently
- workspace.ts: return undefined when no book/binder marker is found
  instead of returning the first workspace folder unconditionally
2026-02-21 14:33:43 -05:00
Vijay Janapa Reddi
09602445de chore: update book content, config, appendices, and tooling
- Vol1: chapter updates across backmatter, benchmarking, data, frameworks, etc.
- Vol2: content updates, new appendices (assumptions, communication, fleet, reliability)
- Quarto: config, styles, formulas, constants
- Add SEMINAL_PAPERS_V2.md, learning_objectives_bolding_parallel.sh
- VSCode extension: package.json, chapterNavigatorProvider
- Landing page and docs updates
2026-02-20 18:55:24 -05:00
Vijay Janapa Reddi
dcc8afb4cc Parallel builds: add HTML/EPUB format picker, clearer reporting
- debugCommands: prompt for format (PDF, HTML, EPUB) in Build All Chapters (Parallel)
- parallelDebug: clearer success/fail messages, Open Reports Folder, REPORT.md header
- README: document volume + format selection for parallel builds
2026-02-15 17:41:10 -05:00
Vijay Janapa Reddi
e6051001b1 Parallel builds: track callout PDFs, fix extension errors, remove copy step
- .gitignore: allow book/quarto/assets/images/icons/callouts/*.pdf
- Add missing callout PDFs (takeaways, pitfall, fallacy) so worktrees have them
- Extension: runtime repo-root check and clearer errors for parallel debug
- Extension: remove copyCalloutPdfsToWorktree (PDFs now in repo by default)
- README: document parallel build requirements; drop callout copy note
2026-02-15 16:50:25 -05:00
Vijay Janapa Reddi
37d26f6def Book volumes: content, VS Code ext, CLI debug, and build updates 2026-02-15 16:02:54 -05:00
Vijay Janapa Reddi
73a956a09b chore(volumes,vscode-ext): batch volume updates and tooling improvements
Checkpoint the branch-wide content/config revisions together with workbench enhancements so chapter rendering and developer workflows stay aligned. This captures the current validation-driven formatting and parallel build/debug improvements in one commit.
2026-02-15 14:03:27 -05:00
Vijay Janapa Reddi
cac84290df feat(vscode-ext): auto-open PDF in editor after build completes
Use a file watcher to detect when the PDF is created/modified during
build, then automatically open it in VS Code. Build still runs in the
visible terminal so users see progress. Also fix LaTeX comma-in-title
bug in foldbox.tex by bracing the title argument inside tcolorbox options.
2026-02-15 12:13:04 -05:00
Vijay Janapa Reddi
577b5d3cc7 feat(vscode-ext): add health status bar and validation checks
Add a persistent health indicator to the extension: a status bar item
that shows pass/warn/error at a glance, plus a health summary node at
the top of the Pre-commit tree view. Fast in-process TypeScript checks
run on file save, editor switch, and startup (<100ms per file).

Checks: duplicate labels, unclosed div fences, missing figure alt-text,
and unresolved in-file cross-references.

- Add src/validation/qmdChecks.ts with four pure check functions
- Add src/validation/healthManager.ts with central status tracker
- Wire HealthManager into extension.ts with status bar and event hooks
- Add expandable health summary node to PrecommitTreeProvider
- Register showHealthDetails command in package.json
2026-02-14 16:18:16 -05:00
Vijay Janapa Reddi
6bca8ef9b0 fix(vscode-ext): fix typed reference colors and simplify settings
Typed references (@tbl-, @fig-, @sec-, @lst-, @eq-) and label
definitions ({#tbl-...}, {#fig-...}) were all rendering in generic
blue because overlapping decorations overrode the typed colors.
Fix by collecting typed matches first and excluding them from
generic/structural buckets. Also fix label-definition regexes to
match labels with trailing attributes (e.g. {#fig-foo fig-env=...}).

Change footnote colors from invisible slate-gray to distinct pink/rose
for clear visual separation from other reference types.

Remove all 27 individual color-override settings (mlsysbook.color*)
since only the preset picker (subtle/balanced/vivid) is needed.
2026-02-13 18:21:25 -05:00
Vijay Janapa Reddi
f4391ce26f refactor(vscode-ext): remove diagnostics system and clean up highlighter
Remove QmdDiagnosticsManager and WorkspaceLabelIndex which caused
false-positive blue squiggles during workspace index loading. Pre-commit
hooks already validate cross-references and inline Python at commit time.

Also remove div fence marker highlighting (Quarto handles natively),
add !inFence guards to label line checks, remove broken reference
decoration, and change footnote colors from gold to muted slate-gray
to convey their marginal/supplementary nature.
2026-02-13 16:54:42 -05:00
Vijay Janapa Reddi
e3cc9f7af3 refactor: rename ml_ml_workflow files, consolidate CLI, and clean up scripts
Remove redundant ml_ prefix from ml_workflow chapter files and update all
Quarto config references. Consolidate custom scripts into native binder
subcommands and archive obsolete tooling.
2026-02-13 11:06:28 -05:00
Vijay Janapa Reddi
acd571095a fix(binder): include all label types by default, enable pattern checks
- Default label types now include Equation (was missing from default set)
- --check-patterns now defaults to True for inline-refs
- Removed redundant --all-types from VSCode extension command

All five label types (Figure, Table, Section, Equation, Listing) are
now always checked unless explicitly filtered with --figures/--tables/etc.
2026-02-12 23:46:58 -05:00
Vijay Janapa Reddi
a0a7f7c658 feat(binder): restructure CLI into check/fix/format hierarchy
Reorganize binder commands into a clean three-verb quality system:

  check   — grouped validation (refs, labels, headers, footnotes,
            figures, rendering) with --scope for granularity
  fix     — content management (headers, footnotes, glossary, images)
  format  — auto-formatters (blanks, python, lists, divs, tables)

Key changes:
- validate → check (with backward-compat alias)
- maintain → fix (with backward-compat alias)
- 17 flat checks grouped into 6 semantic categories
- --scope flag narrows to individual checks within a group
- New FormatCommand with native blanks/lists + script delegation
- Updated pre-commit hooks, VSCode extension, and help output
2026-02-12 23:37:56 -05:00
Vijay Janapa Reddi
8caeac9cc7 refactor(binder): rename validate/maintain subcommands for clarity
Rename verbose compound names to clean, noun-based names:
- section-ids → headers
- forbidden-footnotes → footnote-placement
- footnotes → footnote-refs
- figure-completeness → figures
- figure-placement → float-flow
- index-placement → indexes
- render-patterns → rendering
- dropcap → dropcaps
- part-keys → parts
- image-refs → images

Updated in: validate.py, maintenance.py, pre-commit hooks, VSCode extension.
2026-02-12 23:26:17 -05:00
Vijay Janapa Reddi
755e4cc6a6 feat(binder): consolidate 18 custom scripts into native binder subcommands
Port all custom validation and maintenance scripts into the binder CLI
as native subcommands, eliminating the need for standalone scripts.

New `binder validate` subcommands (10):
- section-ids: verify all headers have {#sec-...} IDs
- forbidden-footnotes: check footnotes in tables/captions/divs
- footnotes: validate footnote refs/defs (undefined, unused, duplicate)
- figure-completeness: check figures have captions and alt-text
- figure-placement: audit figure/table proximity to first reference
- index-placement: check LaTeX \index{} placement
- render-patterns: detect problematic rendering patterns
- dropcap: validate drop cap compatibility
- part-keys: validate \part{key:...} against summaries.yml
- image-refs: validate image references exist on disk

New `binder maintain` subcommands (2):
- section-ids (add/repair/list/remove): full section ID lifecycle
- footnotes (cleanup/reorganize/remove): footnote management

Updated 11 pre-commit hooks to use binder commands instead of scripts.
Updated VSCode extension commands to use binder CLI.
All validators verified against original script output (parity confirmed).
2026-02-12 23:20:54 -05:00
Vijay Janapa Reddi
6b4af17b8f feat(vscode-ext): workspace-wide cross-reference validation and section ID commands
- Add WorkspaceLabelIndex: scans all .qmd files on activation, updates
  incrementally on save, provides hasLabel() for cross-file validation
- Extend QmdDiagnosticsManager to validate references against workspace
  index (not just current file); triggered on save only, not keystrokes
- Add broken reference decoration (red wavy underline) in chunk
  highlighter for refs that don't resolve to any label in the workspace
- Add commands: Add Missing Section IDs, Verify Section IDs,
  Validate Cross-References (command palette)
- Enable diagnostics by default (save-triggered, not noisy)
- Support YAML-style label definitions (#| label:, #| fig-label:, etc.)
2026-02-12 22:53:50 -05:00
Vijay Janapa Reddi
f75bd2e490 fix(vscode-ext): force matplotlib Agg backend more aggressively
Use force=True, switch_backend(), and close("all") after execution
to prevent plot windows from appearing during Python value resolution.
2026-02-12 21:57:31 -05:00
Vijay Janapa Reddi
1995a17d6b feat(vscode-ext): add inline Python value resolution and reduce visual noise
- Add Python value resolver that executes QMD code blocks and resolves
  inline `{python} var` references to their actual values
- Display resolved values as ghost text, hover tooltips, and CodeLens
- Unwrap IPython.display.Markdown objects (.data) and display_value dicts
  for readable output instead of object reprs
- Suppress matplotlib plot windows during resolution (Agg backend)
- Split inline Python highlighting into keyword + variable decorations
- Disable callout/div body and table region background highlights by
  default (too noisy); fence markers and cross-references still colored
- Add settings: showInlinePythonValues, showInlinePythonCodeLens,
  highlightCalloutBackground, highlightTables (default: false)
2026-02-12 21:53:39 -05:00
Vijay Janapa Reddi
2390c3ab31 Refactor: consolidate Quarto config layers and content reorganization.
Unifies Quarto metadata into shared base/format/volume fragments while carrying through chapter path, asset, and tooling updates to keep the repository consistent and easier to maintain.
2026-02-12 15:38:55 -05:00
Vijay Janapa Reddi
d9cb03cf38 Refactor: Systematic Goal/Show/How header audit for Volume 1
- Completed full standardization of 150+ calculation headers across all 16 Volume 1 chapters.
- Replaced legacy 'Why:' blocks with the 'Goal/Show/How' documentation pattern.
- Finalized P.I.C.O. class refactors for complex cells in frameworks and serving.
- Verified header consistency across introduction, ml_systems, training, and optimizations.
- Performed minor stabilization in book/vscode-ext logic.
2026-02-11 21:33:27 -05:00
Vijay Janapa Reddi
c015b9d80a Refactor: stabilize non-PDF build workflows and semantic editor cues.
Standardize Quarto config/style handling for HTML/EPUB volume builds, add explicit binder reset commands by format, and align QMD reference/label highlighting so structural tokens share consistent visual semantics.
2026-02-11 20:36:16 -05:00
Vijay Janapa Reddi
ce68808185 Fix: make pipe table prettifier apply visible alignment changes
Treat internal spacing changes as real formatting differences and normalize separator padding so table prettification is applied consistently. Save files before running pre-commit fixers from the extension so results match editor state.
2026-02-11 18:46:18 -05:00
Vijay Janapa Reddi
c16333cbad Refactor: Finalize Volume 2 P.I.C.O. refactor and TikZ standardization
- Refactored ops_scale.qmd to P.I.C.O. pattern and standardized constants.
- Standardized TikZ colors in storage.qmd (fig-storage-hierarchy) and distributed_training.qmd.
- Verified elimination of magic numbers (1e9, 3600, etc.) across all Volume 2 Python cells.
- Completed full conversion of narrative guards to check() helper in Volume 2.
2026-02-11 17:34:55 -05:00
Vijay Janapa Reddi
66c4970c51 Refactor: standardize all appendix files across Volume 1 and 2
- Refactored appendix_algorithm.qmd, appendix_data.qmd, and appendix_dam.qmd to P.I.C.O. pattern.
- Integrated standardized constants (BILLION, THOUSAND, SEC_PER_HOUR) in all appendix calculations.
- Simplified narrative guards in appendices using the check() helper.
- Verified backmatter consistency across both volumes.
2026-02-11 16:19:14 -05:00
Vijay Janapa Reddi
77f478c1b5 Refactor: standardize constants, simplify invariants, and polish Vol 1 docs
- Standardized time (SEC_PER_HOUR, SEC_PER_DAY), memory (KIB_TO_BYTES), and scale (BILLION, TRILLION) constants in mlsys/constants.py.
- Refactor reordered time constants to fix NameError in constants.py.
- Refactored 90+ narrative guards across Volume 1 to use the simplified check() helper.
- Eliminated remaining magic numbers (1024, 3600, 1e9, etc.) from Python calculation cells.
- Adopted 'Goal / Show / How' documentation structure for key P.I.C.O. scenarios.
- Performed minor stabilization and feature polish in book/vscode-ext.
2026-02-11 15:57:00 -05:00
Vijay Janapa Reddi
abe634ead2 Refactor: unify binder workflows and polish authoring UX
Consolidate Binder/extension command behavior, improve chapter navigation and QMD editor ergonomics, and carry forward Volume 1 content updates so build/debug and writing workflows stay aligned.
2026-02-11 12:34:06 -05:00
Vijay Janapa Reddi
ff3797a1d8 Refactor: Finalize Volume 1 and update CLI/VSCode tooling
- Completed full Volume 1 refactor to Safe Class Namespace pattern.

- Fixed render errors and verified all 16 chapters.

- Updated 'binder' CLI with native validation and maintenance namespaces.

- Enhanced VS Code extension with Chapter Navigator and Run History.

- Integrated 'binder validate' into pre-commit workflows.
2026-02-11 09:25:50 -05:00
Vijay Janapa Reddi
275475e14a Refactor: Port Chapter 7 (ML Frameworks) to mlsys with Direct Access Pattern
- Refactored  to use , , , , , ,  classes.
- Used  and  registries.
- Implemented strict narrative invariants (e.g., verifying A100 ridge point > 100).
- Removed verbose export blocks.
- Verified render.
2026-02-10 18:06:52 -05:00
Vijay Janapa Reddi
9e257f960f Add VSCode extension for book build and debug tooling
Sidebar extension with tree views for build, debug, precommit, and
publish workflows. Integrates with the book CLI for chapter-level
builds, cross-reference checks, and MIT Press release packaging.
2026-01-31 19:46:54 -05:00