Commit Graph

1098 Commits

Author SHA1 Message Date
Vijay Janapa Reddi
45db067464 feat: update publish-live workflow for two-volume release assets
Replace single-file PDF/EPUB upload with per-volume release assets:
- Collect Vol1 + Vol2 PDFs and EPUBs into release-assets/ directory
- Upload all assets via loop with automatic content-type detection
- Update download URLs and summary to reference volume-specific files
- Remove hardcoded Machine-Learning-Systems.pdf/epub references
2026-03-06 18:14:45 -05:00
Vijay Janapa Reddi
9f11116c4b fix: use find-based file discovery in compress steps and preview deploy
Replace hardcoded Machine-Learning-Systems.pdf/.epub filenames with
find-based discovery in all 4 compress steps (Linux/Windows PDF/EPUB)
and the dev preview deploy workflow. This ensures correct packaging
regardless of the output filename set in Quarto YAML configs.
2026-03-06 18:11:11 -05:00
Vijay Janapa Reddi
059291f243 fix: resolve Windows Unicode encoding errors in PDF builds
- Add PYTHONUTF8=1 env var to all Windows Docker run commands (PEP 540)
- Fix generate_figure_list.py to explicitly use encoding='utf-8' in
  write_text() instead of relying on system default (cp1252 on Windows)
- The ≈ character (\u2248) in Vol I content triggered charmap codec errors
2026-03-06 17:18:37 -05:00
Vijay Janapa Reddi
c492de47a4 fix: set UTF-8 encoding in Windows Docker containers
Add PYTHONIOENCODING=utf-8 env var to all Windows Docker run commands,
and set console/output encoding to UTF-8 in the build script. Fixes
UnicodeEncodeError for characters like ≈ (\u2248) that the default
Windows code page (cp1252) cannot encode.
2026-03-06 16:09:42 -05:00
Vijay Janapa Reddi
e3c647e1c3 fix: convert preflight here-string from expandable to literal for file-based execution
The preflight script used @"..."@ (expandable here-string) with backtick-escaped
variables, which produced invalid PowerShell when written to a .ps1 file. Switch
to @'...'@ (literal here-string) with normal PowerShell syntax. Format-specific
checks (PDF/EPUB) are appended conditionally using GitHub Actions expressions.
2026-03-06 15:16:16 -05:00
Vijay Janapa Reddi
4a58fce6ea fix: rewrite all Windows Docker steps to use file-based script execution
Replace pipe-based PowerShell execution (`$script | docker run ... -Command -`)
with file-based execution (`docker run ... -File script.ps1`) for all Windows
Docker steps: preflight, build, PDF compress, and EPUB compress.

The pipe pattern swallows container stdout and doesn't propagate exit codes,
causing builds to appear successful without actually rendering content.
2026-03-06 14:47:55 -05:00
Vijay Janapa Reddi
7183fb6087 fix: use full path for pwsh in Windows Docker containers
The Windows container image installs PowerShell 7 at a fixed path
but the short name 'pwsh' is not resolving in docker run commands.
Use the full path 'C:\Program Files\PowerShell\7\pwsh.exe' instead.
2026-03-06 12:41:34 -05:00
Vijay Janapa Reddi
74000e6077 ci: restore Windows build infrastructure
Restore Windows container build support that was accidentally removed
in commits a76aab467..a90c8803f. This restores:

- Windows Docker infrastructure (book/docker/windows/)
- Windows container build workflow (infra-container-windows.yml)
- Windows matrix entries in book-build-container.yml
- Windows health check support in infra-health-check.yml
- Windows build flags in book-validate-dev.yml and book-publish-live.yml

Restored from pre-removal state at f85e319d6.
2026-03-06 12:13:02 -05:00
Vijay Janapa Reddi
5fb6d6d354 fix: remove stray closing braces in infra-health-check.yml
Remove leftover closing braces that were causing GitHub Actions
workflow parse errors.
2026-03-06 12:00:13 -05:00
Vijay Janapa Reddi
3234a4fbc0 fix: remove trailing OR operators in infra-health-check.yml 2026-03-06 10:14:56 -05:00
Vijay Janapa Reddi
a90c8803ff fix: remove remaining windows outputs from book-build-container.yml 2026-03-06 10:14:00 -05:00
Vijay Janapa Reddi
702a97d00a fix: remove reference to deleted inputs.build_windows 2026-03-06 10:13:12 -05:00
Vijay Janapa Reddi
d02bd6ddd7 fix: remove duplicate shell key in infra-health-check.yml 2026-03-06 10:11:44 -05:00
Vijay Janapa Reddi
a76aab4676 ci: completely remove Windows build infrastructure
Per request, removed all traces of Windows container builds from the project.
This simplifies the CI pipeline to be Linux-only.

- Deleted `book/docker/windows/` directory and its Dockerfile
- Deleted `.github/workflows/infra-container-windows.yml`
- Removed Windows matrix jobs and steps from `book-build-container.yml`
- Removed Windows inputs and outputs from `book-build-container.yml`
- Removed Windows health checks from `infra-health-check.yml`
- Removed Windows references from `book-publish-live.yml`
- Removed Windows references from `book-validate-dev.yml`
2026-03-06 10:04:48 -05:00
Vijay Janapa Reddi
2ae9eb4a40 ci: deprecate Windows container build in dev validation
Removed Windows from the `book-validate-dev.yml` workflow. The Windows
container build is no longer supported or required for the dev validation
phase.

- Removed `build_os` input option for Windows
- Disabled Windows health check
- Removed Windows build matrix configuration
- Removed Windows success checks and reporting
2026-03-06 09:59:45 -05:00
Vijay Janapa Reddi
f85e319d64 fix: add retry logic to GitHub API requests in update_contributors.py
The update_contributors script was failing with intermittent 502 Bad Gateway
errors from the GitHub API when fetching commit history. Added retry logic
with exponential backoff (up to 3 retries) to handle transient 502, 503,
and 504 server errors gracefully.
2026-03-06 09:30:42 -05:00
Vijay Janapa Reddi
b77c8cc8ab fix: resolve PowerShell ScriptBlock parsing error in Windows docker run commands
Refactored Windows container build steps to pipe PowerShell scripts via stdin
instead of passing them as command-line arguments to `docker run`. This prevents
PowerShell from incorrectly interpreting curly braces inside the script string
as a ScriptBlock argument.

Applied this fix to:
- Preflight toolchain (Windows)
- Build format (Windows)
- Compress PDF (Windows)
- Compress EPUB (Windows)
2026-03-06 09:15:01 -05:00
Vijay Janapa Reddi
1d3bcddd0d Harden remaining Windows PowerShell interpolations.
Replace subexpression-based OS/architecture logging in dockerized Windows compression steps with format strings to avoid escaping-related parse failures.
2026-03-06 08:06:16 -05:00
Vijay Janapa Reddi
60cffb80b7 Fix Windows preflight command-source logging escapes.
Replace embedded subexpression interpolation in the dockerized PowerShell preflight checks with format strings so command source logging does not break script parsing.
2026-03-06 08:04:36 -05:00
Vijay Janapa Reddi
2ce2919be8 Ensure Windows runner starts Docker daemon before pulls.
Add a pre-pull Windows step that starts the docker service and waits until docker version succeeds so container image pulls do not fail on missing docker_engine pipe.
2026-03-06 07:52:23 -05:00
Vijay Janapa Reddi
3f88e2a89b Fix container preflight checks across Linux and Windows.
Make pandoc preflight succeed when only Quarto-bundled pandoc is available, and harden Windows error logging to avoid PowerShell interpolation failures in catch blocks.
2026-03-06 07:38:00 -05:00
Vijay Janapa Reddi
4f44655224 Grant actions read permission to dev validation workflow.
Allow reusable container workflow status queries from the caller and fix a stylesheet spelling issue required by pre-commit checks.
2026-03-05 17:35:07 -05:00
Vijay Janapa Reddi
72d714ead8 Improve container preflight diagnostics and docs references.
Add explicit per-check preflight logging and matrix failure instance reporting in the container build workflow, and update stale documentation links and workflow/file path references.
2026-03-05 16:03:00 -05:00
Vijay Janapa Reddi
7992c8ff13 Switch book validation to container-only fail-fast builds.
Remove baremetal workflow usage and add explicit Linux/Windows preflight toolchain checks so missing dependencies fail immediately before render.
2026-03-05 15:33:19 -05:00
Vijay Janapa Reddi
9314128cf7 Fix Windows python3 availability in container build paths.
Preserve container PATH during Windows docker-run steps and create/verify a python3 alias from Scoop Python so Quarto/Jupyter kernels that invoke python3 work reliably in both install-time and final verification checks.
2026-03-05 07:42:29 -05:00
Vijay Janapa Reddi
9ff63264cf Fix Linux baremetal TeX install permissions.
Install TeX Live under the runner user home instead of /usr/local and resolve tlmgr from that location so direct install-tl succeeds without root write access.
2026-03-04 20:07:21 -05:00
Vijay Janapa Reddi
887fc9eb81 Harden final Windows tool checks for existence and exit status.
Update Dockerfile and workflow final verification steps to normalize native command exit detection and fail explicitly on non-zero exits, avoiding false positives and false negatives in PowerShell checks.
2026-03-04 19:46:38 -05:00
Vijay Janapa Reddi
160922f50a Fix Linux TeX installer cleanup glob in baremetal workflow.
Avoid deleting /tmp/install-tl-unx.tar.gz before extraction by restricting cleanup to extracted install-tl directories.
2026-03-04 19:43:17 -05:00
Vijay Janapa Reddi
e015846c3e Harden baremetal TeX Live installs with mirror fallback.
Replace Linux action-based and Windows Chocolatey-based TeX installation with direct install-tl flows that retry across multiple CTAN mirrors and pin tlmgr to a stable repository.
2026-03-04 18:57:50 -05:00
Vijay Janapa Reddi
3fdae4a365 Defer Ghostscript runtime checks to final verification.
Keep fail-fast installation checks for Ghostscript command presence, but defer executing Ghostscript until the final verification phase where full runtime dependencies and PATH are in place.
2026-03-04 18:03:23 -05:00
Vijay Janapa Reddi
77a37a7707 Harden Windows build tool installation and verification flow.
Add immediate per-tool post-install checks with explicit command resolution and exit handling, keep end-of-job final verification with isolated per-tool reporting, and ensure rsvg-convert is installed/verified for Quarto PDF SVG conversion.
2026-03-04 17:21:20 -05:00
Vijay Janapa Reddi
12268748b2 Overhaul Windows container build for PATH and tool reliability
- Add explicit ENV PATH directive (Phase 15) so Docker layers
  inherit tool paths instead of relying on registry writes
- Reorder phases: TeX Live moved last (slowest, fail last)
- Create stable symlink C:\texlive\bin\windows for year-agnostic PATH
- Skip pip self-upgrade to avoid WinError 3 shim lock
- Use gswin64c (correct Scoop binary name) instead of gs
- Add rsvg-convert fallback to Chocolatey if Scoop fails
- Replace fragile verification loop with Test-Tool function
- Relax ErrorActionPreference for Chocolatey TeX Live in baremetal
2026-03-04 17:16:34 -05:00
Vijay Janapa Reddi
226b68ad1f Fix CI tool verification and switch Quarto to Scoop install
- Baremetal workflow: verification step now tracks failures and exits
  non-zero when tools are missing (previously always reported success)
- Baremetal workflow: R.exe --version replaced with Rscript --version
  to avoid PowerShell Invoke-History alias collision
- Windows Dockerfile: Quarto install switched from direct zip download
  (C:\quarto-1.9.27\bin) to scoop install extras/quarto for consistent
  PATH handling via Scoop shims
- Windows Dockerfile: final verification rewritten with failure tracking
  and exit 1 on missing tools
2026-03-04 13:52:02 -05:00
Vijay Janapa Reddi
ddfa753aa5 pwsh 2026-03-04 13:11:33 -05:00
Vijay Janapa Reddi
d9c2906e40 Adds Windows book builds and refines Quarto content
Introduces Windows HTML, PDF, and EPUB build configurations for both Volume I and Volume II in the GitHub Actions workflow, expanding the available output formats for the book.

Updates Quarto callout and figure syntax from `::::` to `:::` across numerous content files for consistency and compatibility.

Removes unreferenced `war_stories.bib` and `data_engineering.bib` bibliography files and their corresponding entries in Quarto configuration.

Standardizes internal references to the 'Responsible AI' chapter by updating `@sec-responsible-engineering` to `@sec-responsible-ai` for improved linking accuracy throughout the text.
2026-03-03 14:50:20 -05:00
Vijay Janapa Reddi
2349e63094 fix(ci): consolidate black version — drop workflow pin, floor to >=24.0.0
The CI workflow hard-pinned black==24.10.0 separately from requirements.txt
(which said >=23.0.0), causing version skew that reformatted 11 QMD files
on every CI run. Remove the override and let requirements.txt be the single
source of truth, bumped to >=24.0.0 to align with current latest.
2026-03-03 07:39:31 -05:00
Vijay Janapa Reddi
6cb39f40ab fix(build): set PYTHONPATH for mlsysim, move output-file to book:, add volume to job name 2026-03-03 07:32:40 -05:00
Vijay Janapa Reddi
213a9cf3b5 fix: remove empty env block from book-preview-dev workflow 2026-03-02 20:27:47 -05:00
Vijay Janapa Reddi
88e6922c56 fix: pin black in pre-commit CI workflow
Pin Black in the dev validation workflow so book-format-python produces deterministic output across local and CI environments, preventing first-pass auto-format churn from failing pre-commit.
2026-03-02 20:24:30 -05:00
Vijay Janapa Reddi
8129e4b31f Improves artifact verification and output naming
Updates the book publishing workflow to conditionally verify downloaded artifacts based on the `deploy_target` input, preventing failures during partial deployments.

Explicitly sets the output filenames for EPUB and PDF builds in Quarto configurations, ensuring consistent naming for generated book artifacts.
2026-03-02 17:51:04 -05:00
Vijay Janapa Reddi
96fa7ac5e5 chore: bump Quarto to 1.9.27 and R to 4.5.2
- Quarto 1.9.27: Linux (.deb), Windows (direct download; Scoop Extras has 1.8.27)
- R 4.5.2: Linux (CRAN jammy-cran40), Windows (Scoop main/r)
- Baremetal: quarto-actions/setup for both Linux and Windows
- Remove ggrepel version pin (R 4.5.x supports ggrepel 0.9.7)
- Update docs: BUILD.md, CONTAINER_BUILDS.md, docker READMEs
2026-03-02 17:36:35 -05:00
Vijay Janapa Reddi
5c19052d2a Add pandas fallback for pre-commit hooks
Include pandas in validate-dev fallback dependencies to satisfy mlsysim.sim imports during book-mlsys-test-units when base requirements install partially fails.
2026-03-02 10:35:21 -05:00
Vijay Janapa Reddi
fdd90ce139 Stabilize dev pre-commit workflow
Add fallback hook dependencies in validate-dev and apply trailing-whitespace fixes to lab plan files so pre-commit no longer fails on auto-modifications.
2026-03-02 10:22:41 -05:00
Vijay Janapa Reddi
a342170b67 Fix pre-commit fallback dependency install
Install rich in validate-dev pre-commit step so book hooks still run when full requirements install partially fails.
2026-03-02 10:16:09 -05:00
Vijay Janapa Reddi
1052b2be31 Update book workflows for volume-only builds
Switch container/baremetal/validate/preview/live flows to vol1+vol2 artifacts, keep baremetal in dev validation, and add stable single-book navbar link.
2026-03-02 09:45:40 -05:00
Vijay Janapa Reddi
a7f9367e42 Merge dev into feature/book-volumes: CI, contributors, workflows
# Conflicts:
#	README.md
2026-03-02 09:38:47 -05:00
Vijay Janapa Reddi
48b519c42e Merge feature/tinytorch-core into feature/book-volumes
# Conflicts:
#	README.md
#	tinytorch/src/01_tensor/01_tensor.py
#	tinytorch/src/15_quantization/ABOUT.md
2026-03-02 09:38:08 -05:00
Vijay Janapa Reddi
0ae4545bbc Enables multi-project contributor additions
Allows a single `@all-contributors` comment to add or update a contributor across multiple projects simultaneously.

Updates the workflow to:
- Detect multiple projects from explicit mentions in the trigger comment.
- Iterate over all detected projects to update their respective `.all-contributorsrc` files and project `README.md` tables.
- Adapt commit messages and bot replies to reflect multi-project changes.

This improves efficiency for managing contributors in multi-project repositories by reducing repetitive commands.
2026-03-02 09:27:59 -05:00
Vijay Janapa Reddi
8abbf533d8 fix(ci): bump R package cache to fix missing ggrepel on Linux builds
- Add -v2 to cache key to invalidate stale R package cache
- Add ggrepel to verification step to catch missing packages early
- Fixes hw_acceleration.qmd build failure (fig-processor-trends chunk)
2026-03-02 08:49:07 -05:00
Vijay Janapa Reddi
7e6cbc96ca Merge remote-tracking branch 'origin/dev' into dev 2026-03-01 18:43:22 -05:00