Commit Graph

117 Commits

Author SHA1 Message Date
Vijay Janapa Reddi
dbb3013fa6 chore: remove TinyTorch CLI docs validation hook
- Remove tinytorch-validate-cli-docs hook from pre-commit
- TinyTorch has its own dedicated CLI validation system
- Keep only formatting checks in pre-commit hooks
2026-01-25 09:23:48 -05:00
Vijay Janapa Reddi
cac3455cab feat(tinytorch): add bibtex-tidy pre-commit hook for paper references
Add separate bibtex-tidy hook to sort and format tinytorch/paper/*.bib
files, matching the book's reference formatting standards.
2025-12-21 09:14:20 -05:00
Vijay Janapa Reddi
565955f0af fix(pdf): restore trailing newlines in titlepage templates
The whitespace cleanup in 853eb03ee removed trailing blank lines from
_titlepage.tex and before-body.tex. This caused \end{titlepage} to be
placed on the same line as a %%% comment, making LaTeX treat it as
part of the comment and skip it entirely.

This resulted in: LaTeX Error: \begin{titlepage} on input line 380
ended by \end{document}

Also excludes titlepage .tex files from end-of-file-fixer hook since
they require trailing blank lines for proper template concatenation.
2025-12-14 12:13:40 -05:00
Vijay Janapa Reddi
166789ddc8 Updates codespell configuration
Updates the codespell pre-commit hook configuration to correctly skip JavaScript files. This prevents false positives and ensures accurate spell checking.
2025-12-14 11:01:47 -05:00
Vijay Janapa Reddi
0484c68add fix: update codespell config to ignore false positives
- Exclude minified JS files (bundle.js, *.min.js) from spelling checks
- Exclude .bib files from end-of-file-fixer (bibtex-tidy handles them)
- Add ignore words for: variable names (currentY, initialY), Python
  methods (assertIn), acronyms (SER, ALS), and valid technical terms
- Skip .tex files from codespell
2025-12-14 10:26:42 -05:00
Vijay Janapa Reddi
4701309c1d style: standardize emoji placement to "Tiny🔥Torch" consistently
Fix remaining instances where emoji was before the name
(🔥 TinyTorch → Tiny🔥Torch) for consistent branding.

Also add "curren" to codespell ignore list (contributor name).
2025-12-13 15:24:33 -05:00
Vijay Janapa Reddi
2e476722ae fix: improve PDF admonition colors and convert to native Sphinx directives
- Add sphinxsetup configuration with distinct colors for each admonition type
  (tip=green, warning=orange, note=blue, caution=red, etc.)
- Convert {admonition} with :class: attribute to native {tip}, {warning},
  {seealso} directives for proper Sphinx type detection in LaTeX output
- Remove unsupported emojis from site markdown files for LaTeX compatibility
- Update codespell ignore list for font names (Heros) and PDF options (FitH)
- Update 20 ABOUT.md files and 16 site/*.md files
2025-12-13 14:58:59 -05:00
Vijay Janapa Reddi
6d7eea5c12 fix: simplify footer using built-in Jupyter Book fields
- Add author and copyright fields to _config.yml
- Remove custom extra_footer HTML block
- Simplify CSS footer styling
- Remove redundant inline footer from intro.md
2025-12-12 10:19:22 -08:00
Vijay Janapa Reddi
377967185c feat: add dev→main sync to live publish workflow
- Live publish now merges dev into main before building
- Removed auto-trigger on push to main (manual only)
- All build jobs depend on sync completing first
- Ensures public site always reflects tested dev state
- Updated README with improved badges and formatting
2025-12-10 11:00:48 -08:00
Vijay Janapa Reddi
b6b3257f9c refactor(pre-commit): reorganize hooks by domain (Book, TinyTorch, Repo)
- Add clear section headers for Book, TinyTorch, and Repo-wide hooks
- Prefix all hook names with domain tags (e.g., [Book], [Repo])
- Fix SVG prevention hook to only apply to book/ directory
- Fix codespell and list-formatting hooks to scope to book/ only
- Add placeholder section for future TinyTorch hooks with examples
- Move JSON and YAML validation to Repo-wide section

This reorganization prepares for the merged monorepo structure where
book and tinytorch have different validation requirements.
2025-12-05 20:15:14 -08:00
Vijay Janapa Reddi
1cca4139f3 Fix pre-commit config paths after restructure
- Update format_tables.py to use workspace-relative path (quarto/contents/)
- Update validate_part_keys.py script to use book/quarto paths
- Scripts in book/tools/ that calculate workspace_root need paths relative to book/
- Other scripts need full book/quarto/contents/ paths
2025-12-05 14:16:13 -08:00
Vijay Janapa Reddi
7b92e11193 Repository Restructuring: Prepare for TinyTorch Integration (#1068)
* Restructure: Move book content to book/ subdirectory

- Move quarto/ → book/quarto/
- Move cli/ → book/cli/
- Move docker/ → book/docker/
- Move socratiQ/ → book/socratiQ/
- Move tools/ → book/tools/
- Move scripts/ → book/scripts/
- Move config/ → book/config/
- Move docs/ → book/docs/
- Move binder → book/binder

Git history fully preserved for all moved files.

Part of repository restructuring to support MLSysBook + TinyTorch.

Pre-commit hooks bypassed for this commit as paths need updating.

* Update pre-commit hooks for book/ subdirectory

- Update all quarto/ paths to book/quarto/
- Update all tools/ paths to book/tools/
- Update config/linting to book/config/linting
- Update project structure checks

Pre-commit hooks will now work with new directory structure.

* Update .gitignore for book/ subdirectory structure

- Update quarto/ paths to book/quarto/
- Update assets/ paths to book/quarto/assets/
- Maintain all existing ignore patterns

* Update GitHub workflows for book/ subdirectory

- Update all quarto/ paths to book/quarto/
- Update cli/ paths to book/cli/
- Update tools/ paths to book/tools/
- Update docker/ paths to book/docker/
- Update config/ paths to book/config/
- Maintain all workflow functionality

* Update CLI config to support book/ subdirectory

- Check for book/quarto/ path first
- Fall back to quarto/ for backward compatibility
- Maintain full CLI functionality

* Create new root and book READMEs for dual structure

- Add comprehensive root README explaining both projects
- Create book-specific README with quick start guide
- Document repository structure and navigation
- Prepare for TinyTorch integration
2025-12-05 14:04:21 -08:00
Vijay Janapa Reddi
1bb5aac313 refactor(lint): use Python built-in json module for validation
Replaced external check-json hook with custom validator using Python's
built-in json module (json.load). Created validate_json.py wrapper to
handle multiple files.

Benefits:
- No external dependencies
- Uses Python's standard library json parser
- Same validation logic as the build system
- Fast and reliable (0.16s for all JSON files)
2025-11-10 13:23:15 -05:00
Vijay Janapa Reddi
fd96039400 feat(lint): add JSON validation to pre-commit hooks
Added check-json hook to automatically validate all JSON files on commit.
Fixed JSON syntax errors in:
- sustainable_ai_quizzes.json (11 missing commas)
- cross_refs_no_explanation.json (trailing commas)
- cross_refs.json (incomplete file, reset to empty array)

All JSON files now pass validation.
2025-11-10 12:35:28 -05:00
Vijay Janapa Reddi
ab75028498 feat(lint): add Purpose section check and FO to ignore list
- Adds pre-commit hook to ensure ## Purpose sections are unnumbered
- Adds FO to codespell ignore list (used in TikZ diagrams)
2025-10-17 10:01:01 -04:00
Vijay Janapa Reddi
4fe88bf456 feat(pre-commit): add citation validation hook
Add new pre-commit hook to validate that all @key citations in .qmd
files have corresponding entries in their .bib files. This catches
missing bibliography entries before they cause Quarto build failures.

Features:
- Validates citations against bibliography files
- Filters out cross-reference labels (fig-, tbl-, sec-, etc.)
- Provides clear error messages with missing citation keys
- Only checks files being committed (not entire codebase)
- Runs in quiet mode to reduce noise

New script: tools/scripts/content/validate_citations.py
Updated: .pre-commit-config.yaml with validate-citations hook
2025-10-09 14:47:35 -04:00
Vijay Janapa Reddi
3b37726b27 refactor(tools): reorganize scripts directory structure for better maintainability
Consolidated 21 root-level scripts into logical subdirectories:

New structure:
- images/: All image management scripts (10 files consolidated from 3 locations)
- infrastructure/: CI/CD and container scripts (3 files)
- content/: Added formatting scripts (3 files moved from root)
- testing/: All test scripts (5 files consolidated)
- glossary/: Added standardize_glossaries.py
- maintenance/: Added generate_release_notes.py, preflight.py
- utilities/: Added validation scripts

Benefits:
- Reduced root-level clutter (21 → 2 files)
- Related scripts grouped logically
- Easier to find and maintain scripts
- Follows standard project organization patterns

Changes:
- Created new subdirectories: images/, infrastructure/
- Moved scripts from root to appropriate subdirectories
- Consolidated scattered scripts (images were in 3 places)
- Updated all pre-commit hook references
- Created README files for new directories
- Included backup file for rollback if needed

Tool: tools/scripts/reorganize_scripts.py (for future reference)
2025-10-09 13:36:16 -04:00
Vijay Janapa Reddi
74cf4294b8 Add table formatting check to pre-commit hooks
- Validates table alignment (text=left, numbers=right)
- Checks header and first column bolding
- Ensures proper spacing and structure
- Runs on all .qmd files in quarto/contents/
- Exits with code 1 if formatting issues found
2025-10-07 14:08:17 -04:00
Vijay Janapa Reddi
ee453794e0 Fix changelog script bugs and generate AI-powered changelog
Script fixes:
- Fix year header detection to handle both '## 2025' and '## 2025 Updates' formats
- Fix labs organization to work with AI-generated summaries
- Add AI artifact cleanup to remove 'Let me know...' phrases
- Improve lab grouping logic for AI mode

Changelog updates:
- Generate comprehensive changelog with AI summaries for all changes since Aug 6
- 61 files updated: 6 frontmatter, 29 chapters, 26 labs
- Clean, professional AI-generated descriptions without artifacts
2025-10-05 12:37:07 -04:00
Vijay Janapa Reddi
3c66295cb1 feat: add pre-commit hook to prevent footnotes in tables/captions/divs
- Created check_forbidden_footnotes.py to detect problematic footnote placements
- Checks for footnotes in: table cells, figure/table captions, div blocks (callouts)
- Added to pre-commit config as 'check-forbidden-footnotes' hook
- Fixed false positive detection by requiring table rows to start with |
- Moved XOR footnote in dl_primer.qmd outside callout block
- All 62 .qmd files now pass validation
- Prevents Quarto build failures from footnotes in unsupported locations
2025-10-01 17:48:53 -04:00
Vijay Janapa Reddi
468e26db10 Refines AI systems engineering content
Enhances clarity and accuracy of explanations throughout the chapters, focusing on definitions, relationships, and system-level trade-offs. It improves flow and highlights scaling-related bottlenecks.
2025-09-28 07:12:25 -04:00
Vijay Janapa Reddi
a814e48eee feat(content): add unified footnote cleanup tool with precommit integration
- Create comprehensive footnote_cleanup.py with all footnote operations:
  * REORGANIZE: Move footnote definitions after their references
  * VALIDATE: Check for undefined references and unused definitions
  * CATALOG: Generate comprehensive footnote reports
  * CLEAN: Automatically fix validation issues
  * REMOVE: Remove all footnotes from files

- Add standard -f/-d options following MLSysBook patterns
- Integrate footnote validation into precommit hooks
- Clean up footnote issues across codebase:
  * Remove undefined reference: fn-mlperf
  * Remove unused definitions: fn-training-challenges, fn-adversarial-examples, fn-value-alignment, fn-sdc-scale
- Replace old separate scripts with unified tool
- Fix footnote spacing: proper line breaks before definitions, no extra breaks after
- Add missing _quarto.yml symlink for project structure validation

Addresses footnote management and cleanup automation
2025-09-07 15:05:41 -04:00
Vijay Janapa Reddi
4793ca1827 refactor: improve footnote validation script with cleanup capability
- Move script to tools/scripts/content/ to match project structure
- Add colored output with emoji indicators for better readability
- Add -f/--file and -d/--directory options for flexible input
- Add --clean flag to automatically remove unused footnote definitions
- Add --dry-run to preview cleanup without making changes
- Add --quiet mode for CI/CD pipelines
- Add --strict mode to fail on any issues
- Match style of other validation scripts in the project
- Update pre-commit hook to use new location and options

The script now provides clear visual feedback and can both validate
and fix footnote issues automatically when needed.
2025-09-06 16:07:53 -04:00
Vijay Janapa Reddi
e8c2bb461c feat: add footnote validation script and pre-commit hook
- Create scripts/validate_footnotes.py to check footnote consistency
- Validates all footnote references have definitions
- Validates all footnote definitions are actually used
- Detects duplicate footnote definitions
- Add to pre-commit hooks for automatic validation
- Currently reports 28 issues to be fixed in future PRs
2025-09-06 16:02:16 -04:00
Vijay Janapa Reddi
88b88a981a feat(tools): add SVG conversion helper tool
- Create convert_svg_to_png.py helper tool for manual SVG conversion
- Tool finds QMD references and assists with conversion process
- User controls all changes - nothing happens automatically
- Update pre-commit hook to mention the helper tool
- Remove automatic documentation file as requested
2025-08-28 11:38:08 +02:00
Vijay Janapa Reddi
80082cadc1 fix(pre-commit): restore SVG sanitization and improve SVG prevention
- Restore sanitize-svgs hook to clean control chars from existing SVG files
- Modify prevent-svg-files hook to exclude generated mediabag SVG files
- Convert and remove misplaced st_water_cycle.svg from sustainable_ai
- Allow generated SVG files while preventing manual SVG additions
2025-08-28 11:36:13 +02:00
Vijay Janapa Reddi
aed344bfb9 feat(pre-commit): add SVG file prevention hook
- Add pre-commit hook to prevent SVG files from being committed
- Provide clear instructions for converting SVG to PNG format
- Remove original SVG files that were converted to PNG
- Ensures format compatibility across all output formats
2025-08-27 16:47:27 +02:00
Vijay Janapa Reddi
19800b309e Adds a pre-commit hook to sanitize SVGs
Introduces a new pre-commit hook that removes control characters from SVG files.
This ensures that SVG files are clean and don't contain potentially malicious or problematic characters.
2025-08-15 13:57:07 -04:00
Vijay Janapa Reddi
fe50e403e5 fix(data-engineering): correct case sensitivity in filenames and references 2025-08-12 14:47:57 -04:00
Vijay Janapa Reddi
d3d95150eb Simplifies pre-commit configuration.
Refactors the pre-commit configuration file for improved readability and maintainability.

Removes unnecessary `ci` configuration, standardizes language specification for scripts,
and removes redundant `additional_dependencies`.
2025-08-12 14:29:43 -04:00
Vijay Janapa Reddi
7a4b118a97 Adds requests dependency for image validation
Updates the `validate-external-images` pre-commit hook to use the `python` language and explicitly declares the `requests` library as a dependency.

This ensures that the script can correctly validate external images by making HTTP requests.
2025-08-12 11:33:34 -04:00
Vijay Janapa Reddi
4b60639cda refactor(scripts): standardize script naming convention for clarity 2025-08-11 12:27:43 -04:00
Vijay Janapa Reddi
215c6f7805 chore(pre-commit): disable workflow change check 2025-08-10 16:54:17 -04:00
Vijay Janapa Reddi
6ad7ed79fd fix(workflows): optimize baremetal build to skip setup steps 2025-08-10 16:19:37 -04:00
Vijay Janapa Reddi
729a060f78 fix(workflows): ensure container build skips gracefully on all platforms 2025-08-10 16:07:57 -04:00
Vijay Janapa Reddi
b6387ba614 feat(precommit): optimize hook order for better efficiency
- Reorder hooks to run formatters first, then validators
- Group hooks by logical phases for better workflow
- Phase 1: Auto-formatters (whitespace, markdown, bibtex)
- Phase 2: Basic validators (YAML, structure)
- Phase 3: Content validators (labels, references, spelling)
- Phase 4: Asset validators (images)
- Phase 5: System checks (workflows, cleanup)

This prevents formatters from running after validators fail,
improving developer experience and reducing unnecessary work.
2025-08-09 15:20:11 -04:00
Vijay Janapa Reddi
fceaf1df19 fix(workflows): fix YAML syntax error and enhance pre-commit validation
- Fix YAML syntax error in publish-live.yml (shell script in wrong location)
- Separate workflow call from shell script monitoring
- Add specific GitHub workflow validation to pre-commit hooks
- Ensure all workflow YAML files are validated before commit
- Fix structural issue where shell script was inside 'with:' section

The publish-live.yml workflow now properly:
- Calls quarto-build-container.yml with correct parameters
- Has separate step for monitoring build completion
- Passes YAML validation with yamllint

Pre-commit now validates:
- All YAML files (general validation)
- GitHub workflow files specifically (targeted validation)
- Ensures workflow syntax is correct before commit
2025-08-08 09:38:21 -04:00
Vijay Janapa Reddi
b8eaa0c741 Replaces structure check with pre-commit workflow
Consolidates quality checks by replacing the dedicated
structure check workflow with a pre-commit workflow.

This change improves code quality and consistency by
automating checks for style, formatting, and project
structure using pre-commit hooks. It removes the separate
structure check workflow and incorporates its functionality
into the new pre-commit setup.
2025-08-07 10:50:05 -04:00
Vijay Janapa Reddi
938357f117 fix(workflows): update all path references from book/ to quarto/
- Update workflow files to use correct quarto/ directory paths
- Fix build-linux-container.yml container paths
- Update .all-contributorsrc file references
- Fix maintenance scripts path references
- Update clean_build.sh script paths
- Update pre-commit config file patterns

Resolves ENOENT error: book/contents/frontmatter/acknowledgements/acknowledgements.qmd
All workflows now reference the correct quarto/ directory structure.
2025-08-05 17:57:12 -04:00
Vijay Janapa Reddi
41ee77e40e fix(windows-container): use standard ltsc2022 base image and update paths 2025-08-05 15:27:17 -04:00
Vijay Janapa Reddi
e09ac08dc4 Improves content consistency and accuracy
Adds a pre-commit hook to check for duplicate labels in the book, ensuring consistency across figures, tables, and listings.

Corrects and updates figure references in the text for clarity and accuracy.

Removes redundant or outdated content to maintain focus and relevance.
2025-08-04 22:54:03 -04:00
Vijay Janapa Reddi
dc1b3fd28b fix(pre-commit): resolve YAML syntax error in workflow file check hook
- Move complex bash command to separate script file
- Fix YAML parsing issues in pre-commit config
- Ensure workflow file detection works properly
2025-08-04 10:29:53 -04:00
Vijay Janapa Reddi
2ad93e412d feat(workflow): improve publish-live workflow with better defaults and early detection
- Set AI mode as default for release notes
- Make description optional with good default
- Make dev commit optional (uses latest if empty)
- Change default release type to patch for frequent textbook updates
- Add pre-commit hook to detect workflow file changes early
- Add pre-push hook as additional safety net
- Add workflow check to prevent permission issues
- Improve user experience with fewer required fields
2025-08-04 10:27:49 -04:00
Vijay Janapa Reddi
2844fb38b8 fix(pre-commit): remove redundant build artifact check
- Remove check-build-artifacts hook that was causing failures
- Keep auto-cleanup-artifacts hook which handles cleanup properly
- Fixes the issue where .quarto cache was detected as build artifact
- Pre-commit now runs cleanly without false positives
2025-08-04 09:57:19 -04:00
Vijay Janapa Reddi
e37f53af9a refactor(validation): simplify to direct yamllint approach
- Remove custom Python script (validate_yaml.py)
- Use direct yamllint in pre-commit for simplicity
- Update .yamllint config to disable style checks
- Focus only on critical syntax errors
- Better performance and standard approach
- Maintain same validation coverage for all YAML files
2025-08-04 09:53:46 -04:00
Vijay Janapa Reddi
333a7e9599 feat(validation): expand YAML validation to all project files
- Rename validate_workflows.py to validate_yaml.py for clarity
- Validate ALL YAML files in the project, not just workflows
- Check GitHub Actions, Quarto configs, extensions, and other YAML files
- Remove duplicate file validation with improved filtering
- Found and validated 23 YAML files across the project
- Update pre-commit hook to use renamed script
- Maintain cross-platform compatibility and lenient rules
2025-08-04 09:52:45 -04:00
Vijay Janapa Reddi
70f98a10e3 feat(validation): add cross-platform YAML validation for workflows and Quarto files
- Add yamllint to requirements.txt for cross-platform compatibility
- Create validate_workflows.py script with auto-installation
- Add .yamllint config optimized for GitHub Actions and Quarto
- Integrate validation into pre-commit hooks
- Fix heredoc syntax errors in publish-live workflow
- Validate both GitHub Actions workflows and Quarto YAML files
- Focus on critical syntax errors, ignore style warnings
- Support macOS, Linux, and Windows platforms
2025-08-04 09:50:58 -04:00
Vijay Janapa Reddi
634bd79190 Disables markdownlint pre-commit hook
The markdownlint pre-commit hook is temporarily disabled.
This change allows for more flexibility during development and avoids
strict linting rules that may hinder initial prototyping.
2025-08-03 18:07:51 -04:00
Vijay Janapa Reddi
b71fab4b1c chore(pre-commit): enable auto-cleanup artifacts hook
Enable the Book Binder auto-cleanup hook to automatically clean build
artifacts before commit, improving repository cleanliness
2025-08-03 18:06:17 -04:00
Vijay Janapa Reddi
e05403460e Add comprehensive external image validation for pre-commit hooks
- Add validate_image_references.py to check all image references exist on disk
- Fix download_external_images.py to update markdown even when images already exist
- Add two pre-commit hooks: external image validation and missing image detection
- Download external images and update references to local paths
- Add documentation for pre-commit external image validation workflow
- Ensure build reliability by preventing external dependencies and missing images
2025-07-31 16:22:35 -04:00