All compression steps (PDF and EPUB, both Linux and Windows) now use the correct
working-directory path by prepending BOOK_ROOT to the output_dir. This fixes the
'No such file or directory' errors when trying to compress build artifacts.
The output_dir from the build step is relative (e.g., '_build/epub'), but needs to
be accessed from BOOK_ROOT (e.g., 'book/_build/epub') after the repository restructuring.
Updated both Linux and Windows Dockerfiles to use ARG variables for path
configuration, providing a single source of truth for book/ structure.
Changes:
- Added ARG variables at top of both Dockerfiles (BOOK_ROOT, BOOK_TOOLS, BOOK_DEPS, BOOK_DOCKER)
- Updated all COPY commands to use ARG variables instead of hardcoded paths
- Updated workflow files to use vars.BOOK_DOCKER for Dockerfile paths
- Maintains compatibility with existing build process
Benefits:
- Single location to update if directory structure changes
- Can override paths at build time with --build-arg
- Self-documenting path structure
- Consistent with GitHub Actions repository variables pattern
Repository Variables renamed for clarity:
- BOOK_ROOT = book
- BOOK_DOCKER = book/docker
- BOOK_TOOLS = book/tools
- BOOK_QUARTO = book/quarto
- BOOK_DEPS = book/tools/dependencies
All paths under book/ now use BOOK_ prefix, making it clear
these are book-related paths (vs future TinyTorch paths).
All workflows now use GitHub Repository Variables for path configuration:
- ${{ vars.BOOK_ROOT }} = book
- ${{ vars.DOCKER_DIR }} = book/docker
- ${{ vars.TOOLS_DIR }} = book/tools
- ${{ vars.QUARTO_DIR }} = book/quarto
- ${{ vars.DEPS_DIR }} = book/tools/dependencies
Benefits:
- Single source of truth (GitHub Settings > Variables)
- Works in all contexts (including job-level env blocks)
- Easy to update without code changes
- Consistent across all workflows
- Add BOOK_ROOT, QUARTO_DIR, TOOLS_DIR, DEPS_DIR, DOCKER_DIR env vars to all workflows
- Update path references to use centralized env vars
- Fix requirements.txt to reference book/tools/dependencies
- Fix pyproject.toml entry points: cli.main -> book.cli.main
- Add __init__.py to book/ and book/tools/ for Python package support
- Fix Dockerfile path casing (dockerfile -> Dockerfile)
- Fix publish-live.yml relative path after cd
This enables TinyTorch to coexist at repo root alongside book/ directory.
- Update working-directory from quarto to book/quarto
- Update Windows container workspace paths
- Update cd quarto commands to cd book/quarto
- Fix config paths to be relative to working directory
- Ensures all workflows work with new book/ structure
- Change book/docker/ back to docker/ for GitHub Actions
- These are external actions (docker/setup-buildx-action, etc.)
- Not our local book/docker/ directory
Fixes container build failures
* Restructure: Move book content to book/ subdirectory
- Move quarto/ → book/quarto/
- Move cli/ → book/cli/
- Move docker/ → book/docker/
- Move socratiQ/ → book/socratiQ/
- Move tools/ → book/tools/
- Move scripts/ → book/scripts/
- Move config/ → book/config/
- Move docs/ → book/docs/
- Move binder → book/binder
Git history fully preserved for all moved files.
Part of repository restructuring to support MLSysBook + TinyTorch.
Pre-commit hooks bypassed for this commit as paths need updating.
* Update pre-commit hooks for book/ subdirectory
- Update all quarto/ paths to book/quarto/
- Update all tools/ paths to book/tools/
- Update config/linting to book/config/linting
- Update project structure checks
Pre-commit hooks will now work with new directory structure.
* Update .gitignore for book/ subdirectory structure
- Update quarto/ paths to book/quarto/
- Update assets/ paths to book/quarto/assets/
- Maintain all existing ignore patterns
* Update GitHub workflows for book/ subdirectory
- Update all quarto/ paths to book/quarto/
- Update cli/ paths to book/cli/
- Update tools/ paths to book/tools/
- Update docker/ paths to book/docker/
- Update config/ paths to book/config/
- Maintain all workflow functionality
* Update CLI config to support book/ subdirectory
- Check for book/quarto/ path first
- Fall back to quarto/ for backward compatibility
- Maintain full CLI functionality
* Create new root and book READMEs for dual structure
- Add comprehensive root README explaining both projects
- Create book-specific README with quick start guide
- Document repository structure and navigation
- Prepare for TinyTorch integration
Addresses Dependabot security alert for arbitrary code injection
vulnerability in lycheeverse/lychee-action < 2.0.2.
- Upgrade from v1.9.3 to v2.0.2
- Fixes potential attack vector via lycheeVersion input
- Impact: Low, vulnerability in link checker action
Addresses #1016
Changes:
- Added pdf.html and epub.html as client-side redirect files
- These use meta refresh and JavaScript redirect for GitHub Pages
- Removed _redirects file copying (Netlify-only feature)
- HTML redirects work on GitHub Pages which doesn't support server redirects
GitHub Pages will serve these HTML files at /pdf and /epub URLs, which
then redirect to the actual files in assets/downloads/.
Addresses #1016
Changes:
- Changed PDF/EPUB navbar links from absolute URLs (mlsysbook.ai) to
relative URLs (/pdf, /epub) so they work on both main and dev sites
- Updated deploy-preview workflow to download PDF and EPUB artifacts
in addition to HTML artifact
- Added step to copy PDF and EPUB files to assets/downloads directory
- Added _redirects file to dev deployment for proper routing
This ensures dev preview site serves its own PDF/EPUB versions rather
than redirecting users to the main production site.
Change default build method from 'both' to 'container' to improve
efficiency and reduce unnecessary baremetal builds. Baremetal and both
options remain available for manual workflow dispatch when needed.
Changes:
- Set default build_method to 'container'
- Auto-triggered builds now use container only
- Manual dispatch still supports all three options
- Updated summary messages to reflect container default
Adds resilience to GitHub API status update calls in the baremetal workflow
to prevent Windows build failures due to network timeouts.
Changes:
- Add 10 second timeout to curl commands (down from 21+ seconds)
- Add retry logic with 2 attempts and 2 second delays
- Make status updates non-fatal with continue-on-error flag
- Add clear error messaging when API calls fail
The build status updates are now resilient to transient network issues
while preserving the custom status checks that may be used by branch
protection rules or external monitoring.
Add automatic version management to publish-live workflow:
- Update version in index.qmd during publish workflow
- Version calculated based on release type (patch/minor/major)
- Committed to dev branch before merge to main
- Version displayed in title metadata and links to releases page
Changes:
- Add update-version job to publish-live.yml workflow
- Create version-link.js to make version link to GitHub releases
- Add comments in index.qmd documenting automation
- Update PUBLISH_LIVE_WORKFLOW.md with version automation docs
- Include version-link.js in HTML builds
This ensures version number stays in sync with GitHub releases
automatically without manual editing.
The issue: scoop prefix python returned empty because the installed app
might be named python313, python3.13, or python depending on the package.
Solution:
- Try multiple app names: python313, python3.13, python
- Show scoop list output to see what was actually installed
- Better error messages showing which app names were tried
- Consistent app name detection in both install and dependencies steps
- Fallback to main/python if versions/python313 not available
This should find the Python installation regardless of how Scoop named it.
The issue: Scoop Python 3.13 was installed but subsequent steps were still
using GitHub Actions' pre-installed Python 3.9, causing betterbib to fail.
Root cause:
- PATH changes within a step don't persist to next steps in GitHub Actions
- Need to use GITHUB_PATH to persist PATH changes
- We were setting PATH in the Python install step but not GITHUB_PATH
Solution:
- Add Scoop Python path to GITHUB_PATH during installation
- Add both python directory and Scripts directory
- Add verification in Python dependencies step that fails if wrong version
- Show exactly which Python executable is being used
This ensures Python 3.13 from Scoop is used in all subsequent steps.
Streamlined the Scoop Python installation to be more direct and debuggable:
Changes:
- Directly attempt 'scoop install versions/python313' without conditionals
- Check LASTEXITCODE for installation success
- List available Python packages for debugging visibility
- Clear fallback logic if python313 not available in versions bucket
- Strict verification that Python 3.13 was installed (fail if not)
This approach is cleaner and will show us exactly what's available in the
versions bucket if python313 doesn't exist.
Reverting back to Scoop for Python installation (per user preference) but with
proper version management using Scoop's versions bucket.
Strategy:
- Add 'versions' bucket which contains older/specific Python versions
- Search for python313 package (Python 3.13.x)
- Install from versions/python313 if available
- Fallback to checking if latest is 3.14 and handling accordingly
- Ensure Scoop's Python path is prioritized in subsequent steps
This approach uses Scoop's built-in version management rather than trying
to use GitHub Actions' setup-python. The versions bucket should have
Python 3.13 available even after 3.14 is released to main bucket.
Python 3.14.0 is TOO NEW for pydantic-core, which uses PyO3 Rust bindings.
The error was clear:
error: the configured Python interpreter version (3.14) is newer than
PyO3's maximum supported version (3.13)
Root cause:
- Scoop installed Python 3.14.0 (latest available)
- pydantic-core's PyO3 bindings don't support 3.14 yet
- Build failed when trying to compile pydantic-core from source
Solution:
- Pin Scoop installation to Python 3.13: 'scoop install python@3.13'
- Update version check to require exactly 3.13 (not 3.14+)
- Add explanation in error messages about PyO3 limitation
This matches the Windows container's expectation of Python 3.13.1 and
ensures all Rust-based Python packages can build successfully.
Python 3.14 is too bleeding-edge for production use right now.
The version check was too strict, requiring exactly Python 3.13. Scoop now
installs Python 3.14.0, which is perfectly fine and newer.
Updated version validation logic:
- Reject Python 3.9 (GitHub Actions pre-installed version)
- Require Python >= 3.10 (minimum for betterbib>=7.5.0)
- Accept 3.13, 3.14, or any future 3.10+ version from Scoop
- Parse version string properly to extract major.minor numbers
This makes the check forward-compatible with future Python versions while
still ensuring we're not using the outdated pre-installed Python 3.9.
The Python dependencies installation was failing because it was using
GitHub Actions' pre-installed Python 3.9.13 instead of the Scoop-installed
Python 3.13. Package betterbib>=7.5.0 requires Python 3.10+.
Root cause:
- GitHub Actions runners have Python 3.9 pre-installed and in PATH
- This Python was found first, before Scoop's Python 3.13
- Cross-platform step didn't specify which Python to use
Solution:
- Split Python dependencies into Windows and Linux steps
- Windows: Use 'scoop prefix python' to get Scoop's Python path
- Explicitly prepend Scoop Python path to PATH for the step
- Add version verification (must be 3.13) with fail-fast
- Show Python location and version for debugging
This ensures Windows uses the same Python version (3.13) as the container,
which should satisfy all package requirements.
After reviewing the Windows container Dockerfile, added several important
features that were missing from bare-metal to improve reliability and
debugging capabilities.
Enhancements to Scoop installation:
- Set UTF-8 encoding explicitly before installing Scoop
- Download Scoop installer script first, then run it (more reliable)
- Use -RunAsAdmin flag matching container behavior
- Use Join-Path for cross-platform path construction
- More verbose logging at each step
New comprehensive verification step (Windows only):
- Prints full PATH for debugging
- Checks for Visual C++ Redistributable DLLs
- Runs 'quarto check' for comprehensive validation (not just --version)
- Systematically verifies all installed tools
- Matches the container's FINAL CHECKS phase
This step will catch environment issues before the build starts, making
debugging much easier if something is misconfigured.
These changes bring bare-metal Windows installation to exact parity with
the container's approach while adapting for GitHub Actions environment.
The Windows container uses 'R --version' which works because the container
environment doesn't have PowerShell's default 'r' alias for Invoke-History.
GitHub Actions runners do have this alias, so we need to use 'R.exe' explicitly.
Changes:
- Use 'R.exe --version' on Windows (matches container intent)
- Split R Setup Info into separate Windows/Linux steps
- Windows uses R.exe with PowerShell
- Linux uses R with bash (no alias conflict)
This exactly replicates the container's R verification approach while
accounting for GitHub Actions runner environment differences.
PowerShell has a built-in alias 'r' for 'Invoke-History', which conflicts
with running the R executable directly. When running 'R --version', it
tries to invoke history instead of R.
Fixes:
- Use 'Rscript --version' instead of 'R --version' in PowerShell
- Use call operator & for explicit command execution
- Update cross-platform R info step to use Rscript consistently
- Use Rscript -e for R.home() to avoid alias conflicts
This ensures R version checks work correctly on Windows runners.
The Scoop installation was completing successfully in its own step, but
subsequent steps couldn't find the scoop command because PATH changes
don't persist across GitHub Actions steps automatically.
Fixes:
- Add Scoop shims directory to GITHUB_PATH for future steps
- Add Scoop to current session PATH for immediate use
- Verify scoop is available before proceeding with bucket setup
- Add refreshenv for Chocolatey to ensure it's available
- Add debugging output showing Python location after Scoop install
This matches how the container handles PATH but adapted for GitHub
Actions' step isolation model.
Changes Windows bare-metal workflow to use Scoop and Chocolatey package
managers, matching the Windows container setup exactly. This should help
isolate the LaTeX build failures that occur on bare-metal but not containers.
Installation changes for Windows builds:
- Add Scoop package manager with git, r-bucket, and extras buckets
- Add Chocolatey package manager
- Install Python via Scoop (was GitHub Action)
- Install R via Scoop (was r-lib/actions)
- Install Quarto via Scoop (was quarto-dev/actions)
- Install Inkscape via Scoop (was winget)
- Install Ghostscript via Scoop (consolidated steps)
- Install TeX Live via Chocolatey 2025.20251008.0 with manual tlmgr
package installation (was zauguin/install-texlive action)
- Add Visual C++ Redistributable via Chocolatey (was missing)
Linux builds remain unchanged, using GitHub Actions as before.
This creates installation parity between container and bare-metal for
debugging purposes. Container builds work reliably, so matching their
approach should help identify what causes bare-metal LaTeX failures.
The build-baremetal job was missing the always() condition and explicit
success check that the build-container job has. This caused the baremetal
job to be skipped during manual workflow dispatch even when build_method
was set to 'both'.
Now both jobs use consistent condition patterns:
- always() to ensure evaluation regardless of dependency states
- explicit check for build-config success
- check for appropriate build_method value
This ensures when users select 'both' build methods, both container and
baremetal builds will actually run.
The Windows container build was failing with PowerShell syntax errors
because variables like $py_version were being evaluated by the outer
shell instead of being passed into the Docker container. This caused
the variable names to be stripped, leaving invalid syntax like
'= (python --version)'.
Fixed by escaping all PowerShell variables with backticks (`$) in
both the PDF and EPUB compression steps, ensuring they are passed
literally into the container's PowerShell environment.
Addresses the syntax errors preventing Windows PDF and EPUB builds
from completing successfully.
- Add user-focused language guidelines with clear examples
- Enhance structure with Educational Innovation section
- Remove development metrics from release note generation
- Focus on educational value and learning outcomes
- Follow established MLSysBook release note pattern
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Merges chapter 14 content updates including quiz refinements, bibliography improvements, and content enhancements across all chapters.
Also includes config cleanup: removes duplicate .codespell-ignore and .lycheeignore files from root (now centralized in config/linting/), adds SER to codespell ignore list for TikZ node names, and updates Vale technical terms.
- Add fixed position Netlify badge to bottom-right of HTML version
- Badge is small (30px), clickable, and links to netlify.com
- Only visible in HTML format, not PDF/EPUB
- Addresses Netlify hosting requirement for visible badge on main page
- Update changelog scripts to use correct 'quarto/contents/**/*.qmd' path
- Fix quarto config paths from 'book/config/' to 'quarto/config/'
- Update link-check workflow with correct content paths
- Resolves issue where scripts found 0 changes instead of 330+ commits
The previous implementation was missing an `apt-get update` command before attempting to install `ghostscript`. This change adds that command to ensure that the package list is up-to-date before attempting to install the package.
- Replace unreliable winget with Scoop package manager
- Add automatic Scoop installation if not present
- Update verification logic to use 'scoop which' commands
- Implement robust fallback detection for better reliability
- Align baremetal workflow with proven Windows container method
- Resolves path detection issues with winget installations
This matches the approach used successfully in Windows containers
and provides more predictable Ghostscript installation paths.
- Remove complex fallback logic now that we have confirmed package IDs
- Use direct winget commands with exact package identifiers
- Simplify verification steps to essential checks only
- Clean up verbose diagnostic output and error handling
- Streamline installation process for better maintainability
Much cleaner implementation using reliable winget -e --id commands.
- Add -e --id flags to Inkscape and Ghostscript installations
- Prioritize ArtifexSoftware.GhostScript as confirmed working package ID
- Improve package resolution reliability and prevent fuzzy matching issues
- Reduces ambiguity in winget package installation process
Thanks to user feedback for the reliable winget command format.
- Fix winget package identifier issues by trying multiple package names
- Add robust fallback to chocolatey if winget packages fail
- Implement flexible path detection for different installation locations
- Add comprehensive verification step to ensure Ghostscript works
- Provide detailed diagnostics for troubleshooting installation issues
Resolves 'No package found matching input criteria' and missing
directory errors that started occurring due to winget package changes.
- Fix immediate verification by using full path to inkscape.exe
- Add PATH verification step to ensure Inkscape is available for Quarto
- Addresses recent winget behavior changes that broke PATH updates
- Ensures TikZ diagram processing works correctly in subsequent steps
- Includes comprehensive diagnostics for troubleshooting PATH issues
Resolves sudden build failures where Inkscape was installed successfully
but not immediately available in PATH for verification and Quarto usage.
- Add acknowledgements.qmd to git status check and commit step
- Ensures all-contributors-cli changes are properly committed
- Should fix issue where taunoe appears in .all-contributorsrc but not in rendered files
- Add contributors workflow to validate-dev.yml after pre-commit validation
- Fix contributors script to use GITHUB_TOKEN environment variable
- Contributors now update automatically on every dev push
- Ensures acknowledgements are current before build matrix runs
- Resolves issue where taunoe and other recent contributors were missing
