Commit Graph

144 Commits

Author SHA1 Message Date
Vijay Janapa Reddi
e8c2bb461c feat: add footnote validation script and pre-commit hook
- Create scripts/validate_footnotes.py to check footnote consistency
- Validates all footnote references have definitions
- Validates all footnote definitions are actually used
- Detects duplicate footnote definitions
- Add to pre-commit hooks for automatic validation
- Currently reports 28 issues to be fixed in future PRs
2025-09-06 16:02:16 -04:00
Vijay Janapa Reddi
88b88a981a feat(tools): add SVG conversion helper tool
- Create convert_svg_to_png.py helper tool for manual SVG conversion
- Tool finds QMD references and assists with conversion process
- User controls all changes - nothing happens automatically
- Update pre-commit hook to mention the helper tool
- Remove automatic documentation file as requested
2025-08-28 11:38:08 +02:00
Vijay Janapa Reddi
80082cadc1 fix(pre-commit): restore SVG sanitization and improve SVG prevention
- Restore sanitize-svgs hook to clean control chars from existing SVG files
- Modify prevent-svg-files hook to exclude generated mediabag SVG files
- Convert and remove misplaced st_water_cycle.svg from sustainable_ai
- Allow generated SVG files while preventing manual SVG additions
2025-08-28 11:36:13 +02:00
Vijay Janapa Reddi
aed344bfb9 feat(pre-commit): add SVG file prevention hook
- Add pre-commit hook to prevent SVG files from being committed
- Provide clear instructions for converting SVG to PNG format
- Remove original SVG files that were converted to PNG
- Ensures format compatibility across all output formats
2025-08-27 16:47:27 +02:00
Vijay Janapa Reddi
19800b309e Adds a pre-commit hook to sanitize SVGs
Introduces a new pre-commit hook that removes control characters from SVG files.
This ensures that SVG files are clean and don't contain potentially malicious or problematic characters.
2025-08-15 13:57:07 -04:00
Vijay Janapa Reddi
fe50e403e5 fix(data-engineering): correct case sensitivity in filenames and references 2025-08-12 14:47:57 -04:00
Vijay Janapa Reddi
d3d95150eb Simplifies pre-commit configuration.
Refactors the pre-commit configuration file for improved readability and maintainability.

Removes unnecessary `ci` configuration, standardizes language specification for scripts,
and removes redundant `additional_dependencies`.
2025-08-12 14:29:43 -04:00
Vijay Janapa Reddi
7a4b118a97 Adds requests dependency for image validation
Updates the `validate-external-images` pre-commit hook to use the `python` language and explicitly declares the `requests` library as a dependency.

This ensures that the script can correctly validate external images by making HTTP requests.
2025-08-12 11:33:34 -04:00
Vijay Janapa Reddi
4b60639cda refactor(scripts): standardize script naming convention for clarity 2025-08-11 12:27:43 -04:00
Vijay Janapa Reddi
215c6f7805 chore(pre-commit): disable workflow change check 2025-08-10 16:54:17 -04:00
Vijay Janapa Reddi
6ad7ed79fd fix(workflows): optimize baremetal build to skip setup steps 2025-08-10 16:19:37 -04:00
Vijay Janapa Reddi
729a060f78 fix(workflows): ensure container build skips gracefully on all platforms 2025-08-10 16:07:57 -04:00
Vijay Janapa Reddi
b6387ba614 feat(precommit): optimize hook order for better efficiency
- Reorder hooks to run formatters first, then validators
- Group hooks by logical phases for better workflow
- Phase 1: Auto-formatters (whitespace, markdown, bibtex)
- Phase 2: Basic validators (YAML, structure)
- Phase 3: Content validators (labels, references, spelling)
- Phase 4: Asset validators (images)
- Phase 5: System checks (workflows, cleanup)

This prevents formatters from running after validators fail,
improving developer experience and reducing unnecessary work.
2025-08-09 15:20:11 -04:00
Vijay Janapa Reddi
fceaf1df19 fix(workflows): fix YAML syntax error and enhance pre-commit validation
- Fix YAML syntax error in publish-live.yml (shell script in wrong location)
- Separate workflow call from shell script monitoring
- Add specific GitHub workflow validation to pre-commit hooks
- Ensure all workflow YAML files are validated before commit
- Fix structural issue where shell script was inside 'with:' section

The publish-live.yml workflow now properly:
- Calls quarto-build-container.yml with correct parameters
- Has separate step for monitoring build completion
- Passes YAML validation with yamllint

Pre-commit now validates:
- All YAML files (general validation)
- GitHub workflow files specifically (targeted validation)
- Ensures workflow syntax is correct before commit
2025-08-08 09:38:21 -04:00
Vijay Janapa Reddi
b8eaa0c741 Replaces structure check with pre-commit workflow
Consolidates quality checks by replacing the dedicated
structure check workflow with a pre-commit workflow.

This change improves code quality and consistency by
automating checks for style, formatting, and project
structure using pre-commit hooks. It removes the separate
structure check workflow and incorporates its functionality
into the new pre-commit setup.
2025-08-07 10:50:05 -04:00
Vijay Janapa Reddi
938357f117 fix(workflows): update all path references from book/ to quarto/
- Update workflow files to use correct quarto/ directory paths
- Fix build-linux-container.yml container paths
- Update .all-contributorsrc file references
- Fix maintenance scripts path references
- Update clean_build.sh script paths
- Update pre-commit config file patterns

Resolves ENOENT error: book/contents/frontmatter/acknowledgements/acknowledgements.qmd
All workflows now reference the correct quarto/ directory structure.
2025-08-05 17:57:12 -04:00
Vijay Janapa Reddi
41ee77e40e fix(windows-container): use standard ltsc2022 base image and update paths 2025-08-05 15:27:17 -04:00
Vijay Janapa Reddi
e09ac08dc4 Improves content consistency and accuracy
Adds a pre-commit hook to check for duplicate labels in the book, ensuring consistency across figures, tables, and listings.

Corrects and updates figure references in the text for clarity and accuracy.

Removes redundant or outdated content to maintain focus and relevance.
2025-08-04 22:54:03 -04:00
Vijay Janapa Reddi
dc1b3fd28b fix(pre-commit): resolve YAML syntax error in workflow file check hook
- Move complex bash command to separate script file
- Fix YAML parsing issues in pre-commit config
- Ensure workflow file detection works properly
2025-08-04 10:29:53 -04:00
Vijay Janapa Reddi
2ad93e412d feat(workflow): improve publish-live workflow with better defaults and early detection
- Set AI mode as default for release notes
- Make description optional with good default
- Make dev commit optional (uses latest if empty)
- Change default release type to patch for frequent textbook updates
- Add pre-commit hook to detect workflow file changes early
- Add pre-push hook as additional safety net
- Add workflow check to prevent permission issues
- Improve user experience with fewer required fields
2025-08-04 10:27:49 -04:00
Vijay Janapa Reddi
2844fb38b8 fix(pre-commit): remove redundant build artifact check
- Remove check-build-artifacts hook that was causing failures
- Keep auto-cleanup-artifacts hook which handles cleanup properly
- Fixes the issue where .quarto cache was detected as build artifact
- Pre-commit now runs cleanly without false positives
2025-08-04 09:57:19 -04:00
Vijay Janapa Reddi
e37f53af9a refactor(validation): simplify to direct yamllint approach
- Remove custom Python script (validate_yaml.py)
- Use direct yamllint in pre-commit for simplicity
- Update .yamllint config to disable style checks
- Focus only on critical syntax errors
- Better performance and standard approach
- Maintain same validation coverage for all YAML files
2025-08-04 09:53:46 -04:00
Vijay Janapa Reddi
333a7e9599 feat(validation): expand YAML validation to all project files
- Rename validate_workflows.py to validate_yaml.py for clarity
- Validate ALL YAML files in the project, not just workflows
- Check GitHub Actions, Quarto configs, extensions, and other YAML files
- Remove duplicate file validation with improved filtering
- Found and validated 23 YAML files across the project
- Update pre-commit hook to use renamed script
- Maintain cross-platform compatibility and lenient rules
2025-08-04 09:52:45 -04:00
Vijay Janapa Reddi
70f98a10e3 feat(validation): add cross-platform YAML validation for workflows and Quarto files
- Add yamllint to requirements.txt for cross-platform compatibility
- Create validate_workflows.py script with auto-installation
- Add .yamllint config optimized for GitHub Actions and Quarto
- Integrate validation into pre-commit hooks
- Fix heredoc syntax errors in publish-live workflow
- Validate both GitHub Actions workflows and Quarto YAML files
- Focus on critical syntax errors, ignore style warnings
- Support macOS, Linux, and Windows platforms
2025-08-04 09:50:58 -04:00
Vijay Janapa Reddi
634bd79190 Disables markdownlint pre-commit hook
The markdownlint pre-commit hook is temporarily disabled.
This change allows for more flexibility during development and avoids
strict linting rules that may hinder initial prototyping.
2025-08-03 18:07:51 -04:00
Vijay Janapa Reddi
b71fab4b1c chore(pre-commit): enable auto-cleanup artifacts hook
Enable the Book Binder auto-cleanup hook to automatically clean build
artifacts before commit, improving repository cleanliness
2025-08-03 18:06:17 -04:00
Vijay Janapa Reddi
e05403460e Add comprehensive external image validation for pre-commit hooks
- Add validate_image_references.py to check all image references exist on disk
- Fix download_external_images.py to update markdown even when images already exist
- Add two pre-commit hooks: external image validation and missing image detection
- Download external images and update references to local paths
- Add documentation for pre-commit external image validation workflow
- Ensure build reliability by preventing external dependencies and missing images
2025-07-31 16:22:35 -04:00
Vijay Janapa Reddi
8cbccf2750 Adds check for external images in Quarto files
Ensures that all images used in Quarto files are local.
This increases the build reliability by preventing external
dependencies during the build process.
2025-07-31 15:25:09 -04:00
Vijay Janapa Reddi
11f76f237c Moves pre-commit scripts to the tools directory
Updates the pre-commit configuration to reference the scripts located in the `tools/scripts` directory. This change helps to organize the project structure by centralizing tooling-related scripts in a dedicated directory.
2025-07-31 11:49:13 -04:00
Vijay Janapa Reddi
14c37b846b Add pre-commit part key validation system
- Add validate-part-keys hook to .pre-commit-config.yaml
- Create validate_part_keys.py script for comprehensive validation
- Enhance inject-parts.lua with better error handling and pre-scan
- Add documentation for part key validation system
- Remove validation from GitHub workflow (moved to pre-commit)
- Add utility scripts for key checking and build cleanup

This catches invalid part keys before commit rather than in CI/CD
2025-07-31 01:35:46 -04:00
Vijay Janapa Reddi
ef46959bf2 Fix image format validation and organization
- Update pre-commit config to validate all images in contents/ (not just core/)
- Fix format mismatches: convert MPO to JPEG, incorrect JPEG to PNG
- Reorganize misplaced images to correct format directories
- Update QMD references for all moved image files
- Create missing jpeg directories where needed
- All images now properly validated and organized by format
2025-07-30 03:04:44 -04:00
Vijay Janapa Reddi
94ff74c0d3 Removes unnecessary language version specification
The 'language_version' specification is redundant for the 'check_images.py' hook. The tool implicitly utilizes the environment's default Python version. Removing this declaration simplifies the configuration.
2025-07-29 09:53:20 -04:00
Vijay Janapa Reddi
98002845e7 Refactors Quarto config selection in CI
Updates the CI build process to remove the existing `_quarto.yml` file before copying the appropriate configuration.

This ensures a clean slate and prevents potential conflicts between different build configurations (HTML, PDF, EPUB). It also changes the location of the config files to reside in a dedicated `config` directory.
2025-07-29 09:16:05 -04:00
Vijay Janapa Reddi
ec121fe758 Temporarily disables section ID check
Disables the section ID check pre-commit hook.

This change is temporary and the hook will be re-enabled in
a future commit.
2025-07-29 06:48:59 -04:00
Vijay Janapa Reddi
6018830d4c 🔗 Complete pre-commit integration: Book Binder + TikZ cleanup
 Perfect pre-commit integration achieved:
- Added check-build-artifacts hook using ./binder check
- Fixed exit codes: returns 1 when artifacts found, 0 when clean
- Removed tracked TikZ diagram PDFs from git (build artifacts)
- Enhanced artifact detection and cleanup patterns

🧹 TikZ diagram management:
- Removed diagram-*.pdf files from git tracking
- Added cleanup patterns for generated diagrams
- Pre-commit now properly detects and blocks artifact commits

🚀 Complete workflow proven:
1. git commit → pre-commit runs binder check
2. If artifacts found → commit blocked with guidance
3. User runs ./binder clean → artifacts removed
4. Retry commit → pre-commit passes 

🎉 Book Binder: Complete MLSysBook development CLI
- Single tool for all development operations
- Perfect pre-commit integration
- Bulletproof artifact detection and cleanup
- Professional git workflow integration
2025-07-28 12:03:13 -04:00
Vijay Janapa Reddi
e2b91dd3bb feat: improve website navigation structure
- Implement floating sidebar with complete chapter hierarchy
- Add all book chapters to sidebar navigation
- Organize labs by platform (Arduino, Seeed, Raspberry Pi)
- Create proper RESOURCES part for PhD guide and references
- Fix duplicate naming in Raspberry Pi section
- Change navbar background from blue primary to dark
- Add clean navbar with Labs/Kits + GitHub tools
- Update pre-commit config paths for new structure
2025-07-25 17:41:42 -04:00
Vijay Janapa Reddi
1fdf3749e4 Fix pre-commit configuration paths and create scripts symlink
- Create symlink from scripts/ to tools/scripts/ for pre-commit hooks
- Update all script paths in .pre-commit-config.yaml to correct locations:
  - find_unreferenced_labels.py -> content/find_unreferenced_labels.py
  - section_id_manager.py -> content/manage_section_ids.py
  - collapse_blank_lines.py -> content/collapse_blank_lines.py
  - check_images.py -> utilities/check_images.py
- Update all file patterns from contents/ to book/contents/
- Fix trailing whitespace detected by hooks
2025-07-25 14:07:55 -04:00
Vijay Janapa Reddi
c3e8ef65ae Removes emphasis in figure captions
Removes redundant bold formatting from figure captions to improve clarity and consistency.

Updates the "Free Models" caption to "AlphaFold" for better context.
2025-07-23 22:54:00 -04:00
Vijay Janapa Reddi
9a556814ea Removes check for large files
Removes the pre-commit hook that checks for large files (>1MB).
This is done to accommodate larger figure sizes.
2025-07-23 22:52:31 -04:00
Vijay Janapa Reddi
6f5916e203 Refines figure and table descriptions
Updates descriptions for the AlphaFold figure and the software vs. ML development table to improve clarity and conciseness.
2025-07-23 19:14:54 -04:00
Vijay Janapa Reddi
aa99fb5a99 Enhances descriptions and clarifies key concepts
Refines figure captions and table descriptions across multiple core concept documents to improve clarity and understanding.

These changes aim to provide more context and deeper insights into AI concepts, architectural patterns, and benchmarking methodologies. The focus is on explaining the "why" behind design choices and the trade-offs involved in different approaches.
2025-07-23 18:02:13 -04:00
Vijay Janapa Reddi
1a23a05e9c Reduce large file limit in pre-commit from 10MB to 1MB
Prevents accidental commits of large files to avoid repository cleanup issues.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-07-22 18:56:58 -04:00
Vijay Janapa Reddi
631a706f73 Adds pre-commit hook for large file check
Adds a pre-commit hook to prevent committing files larger than 10MB.
This helps to avoid including large, unnecessary files in the repository.
2025-07-22 10:54:27 -04:00
Vijay Janapa Reddi
c2ff01dfda Improves pre-commit hook configuration
Organizes and enhances the pre-commit configuration for better code quality and consistency.

- Groups hooks into logical sections (Spellchecking/Linting, Bibliography, Local Custom Hooks) for improved readability.
- Adds markdown linting and formatting hooks to ensure consistent markdown style.
- Refines local hook descriptions and organization for clarity.
- Validates images using pillow and rich libraries
2025-07-11 18:23:04 -04:00
Vijay Janapa Reddi
2255621f9f Adds pre-commit hook to detect locked files
Adds a pre-commit hook that checks for locked files (uchg flag) within the contents/ directory. This prevents accidental commits of locked files on macOS.

The hook uses `find` and `ls` to identify locked files and exits with an error if any are found, prompting the user to unlock them.
2025-07-11 18:11:30 -04:00
Vijay Janapa Reddi
8d72833ecf Adds image validation pre-commit hook
Adds a pre-commit hook to validate image files.
This ensures that images are not corrupted and have the correct format.

Also, renames several png images to jpg or gif and updates references in the qmd files.
2025-07-11 18:09:15 -04:00
Vijay Janapa Reddi
b1078b7d2a Removes unused hook
Removes the 'update-texlive-packages' hook from the pre-commit configuration.
This hook is no longer needed, simplifying the configuration and
reducing unnecessary processing time during pre-commit checks.
2025-07-02 15:06:43 -04:00
Vijay Janapa Reddi
d1e798a2b8 Automates TeX Live package management
Replaces the manual package extraction script with an automated tool.

This tool analyzes project files, extracts LaTeX package dependencies,
and generates a list of required TeX Live packages and collections.
It improves maintainability and ensures all necessary packages are
installed. The pre-commit configuration is updated to run this tool
automatically before committing changes.
2025-07-02 15:04:20 -04:00
Vijay Janapa Reddi
2a6d83107b Replaces sed with python script for blank line collapsing
Refactors the blank line collapsing logic from a bash command to a dedicated python script.

This change improves cross-platform compatibility, particularly addressing issues with sed implementations on different operating systems. It also enhances code readability and maintainability by using a python script.
2025-06-30 11:08:25 -04:00
Vijay Janapa Reddi
7827e0051a Updates pre-commit hooks and adds label checking
Updates versions of codespell and markdownlint-cli pre-commit hooks.
The mdformat hook is downgraded to address compatibility issues.

The unreferenced label check is re-enabled, enhancing content integrity
by identifying unused labels. Also standardizes the Python shebang and adds a missing dependency.

Updates the section ID checker to utilize `nltk` for improved
accuracy and reliability.

Improves the Markdown header detection to cover more cases.
2025-06-30 10:21:40 -04:00