for not-installed lookups too
Caught by local TL 2026 verification: 'tlmgr info --only-installed bogus-xyz'
prints
package: bogus-xyz
installed: No
so the previous grep '^package:' matched both installed and not-installed,
which would silently skip *every* collection — including fontsextra and
latexextra that genuinely need network install. The container build would
exit 0 with no install attempted and produce a :latest image still missing
newpx, recreating exactly the bug this branch is trying to fix.
Anchor on 'installed: Yes' (after the colon, allowing for whitespace) so
not-installed collections fall through to the install loop as intended.
The 16:23 container rebuild caught a real flaky failure with the previous
commit's stricter exit-non-zero behavior: collection-fontsrecommended
failed twice when tlmgr's mirror.ctan.org redirect landed on stale mirrors
(ctan.math.illinois then mirrors.mit). On a stale mirror tlmgr refuses
with a silent 'Remote database at <url>' / exit 1, never reaching the
'package already present' fast-path that would have succeeded against a
fresh mirror.
install-tl's scheme-medium already installs basic, fontsrecommended,
fontutils, latex, latexrecommended, luatex, pictures, plus most language
collections — 7 of the 9 entries in tl_packages. Only fontsextra and
latexextra genuinely need a tlmgr install operation. Query the local
tlpdb with 'tlmgr info --only-installed' (no network) and skip the
network call entirely when the collection is already present, sidestepping
the random-mirror staleness for the redundant entries.
Previously install-tl-collections.sh exited 0 even when collections in
tl_packages failed to install, so the Linux Docker image would tag and
publish as :latest with missing fonts/packages. The failure surfaced
hours later as a downstream PDF render error
('LaTeX Error: File `newpxtext.sty` not found') in book-build-container,
making the chain of causation hard to spot.
Every collection listed in tl_packages is required by the book PDF
build — there is no soft-dependency tier. If any of them cannot be
installed, exit non-zero so the container build fails fast and the
broken image is never published.
Also tighten the 'tlmgr not available' branch to fail rather than skip:
no tlmgr means no PDF build, so silently moving on is wrong.
The previous retry loop used 'if tlmgr install ...; then' which discarded
tlmgr's actual error output, leaving only '❌ Attempt N failed' in the CI
log. On the 2026-05-04 rebuild this masked the real reason
collection-fontsextra refused to install (the version-mismatch error fixed
in the previous commit) and made the failure invisible until a downstream
PDF build broke 3 hours later.
Capture tlmgr's combined stdout and stderr to a tmpfile, always emit it,
and report the exit code on failure so the diagnostic reaches the build
log on the first attempt.
When install-tl runs against a tlnet snapshot and minutes later the script
calls 'tlmgr install collection-fontsextra', tlnet has often rolled forward
and tlmgr refuses with "Local TL version is incompatible with the
repository", which on the 2026-05-04 rebuild left newpx (and the rest of
collection-fontsextra) absent from the published :latest image, producing
a Vol II PDF build failure for newpxtext.sty 3 hours later.
Run 'tlmgr update --self' once before the collection install loop so the
local tlmgr matches whatever revision the remote currently advertises.
Failure here is non-fatal so a flaky update doesn't block the build.
Default RUN /bin/sh doubles 82414 inside double quotes to the shell PID, so
command substitution with Docker escaped dollar-paren was mis-parsed
(Phase 10: syntax error on open paren). Use POSIX backticks for command
substitution, brace variables for echo, and a short file-header note.
fs configure requires libuv headers/pkg-config; without libuv1-dev the build
fails on fs then cascades to rmarkdown. Pre-flight: pkg-config --exists libuv.
- Add book/tools/dependencies/required_r_packages.R for a single source of truth
- install_packages.R: fail if any package not loadable after install; source shared list
- verify: match R_LIBS_USER and requireNamespace only (no re-source of full install/tinytex)
- Dockerfile: COPY and cleanup /tmp/required_r_packages.R with install/verify scripts
Phase 7 already runs install_packages.R and verify, then removes /tmp copies. The old steps 9.1–9.3 only re-ran a minimal fallback. Remove redundant R library RUN; renumber phases 12→11 and 13→12.
RUN is preprocessed: bare $collection/$i/$success etc. become empty. Double $ so dash sees the real loop variables. Same for package_count echo after assignment.
BuildKit can expand an inline PATH+= such that /usr/bin is lost (head, etc. missing). TeX step uses an explicit system PATH; drop redundant health-check exports; no tlmgr|head.
The sequence $$(( is parsed as one escaped $, leaving bash-style
(( which /bin/dash rejects (Syntax error: "(" unexpected). Use
$$$(( so the shell receives proper POSIX $$(( arithmetic
expansion.
Dockerfile treats $4, $PATH, $?, ${duration} in shell RUN as build-time; use $$ so the image script keeps correct dash/sh behavior. Bump TEXLIVE_INSTALLER_REVISION=6.
Dockerfile RUN must use $$( for literal $( and $$(( for $(( so dash sees command/arithmetic substitution. Fixes Step 5.5 (tlmgr collections) parse failure after TeX layer started building.
The multi-line RUN hit dash/buildx parsing issues (Syntax error: '(' unexpected). Move download + install-tl into /opt/install-texlive-base.sh and invoke with a short RUN; bump TEXLIVE_INSTALLER_REVISION to invalidate cache.
CI /bin/sh (dash) failed with 'Syntax error: ( unexpected' on the TeX layer.
Use cut -c12-15 for year (install-tl-YYYYMMDD), separate case arms for
2026/2027, and error lines without a single echo that could confuse the
parser. REVISION=4 to refresh build cache.
mirrors still serve install-tl-unx for TL2025 while main tlnet is 2026; a fixed
-repository caused local 2025 / repository 2026. After extract, derive year from
install-tl-YYYY* path, use Utah historic tlnet-final for 2025 and mirror.ctan
for 2026+, save URL for tlmgr, bump install revision.
Merge download/extract and profile/install into one RUN so buildx cannot cache
a stale install-tl-unx (wrong TL year) separately from -repository. Bump
TEXLIVE_INSTALLER_REVISION when forcing a fresh installer after CTAN rollover.
Historic TL2025 repo breaks when install-tl-unx is already TL2026 (local 2026 vs
repo 2025). Use main systems/texlive/tlnet with the same installer download.
CTAN main tlnet moved to 2026 while install-tl-unx still targets TL2025, so
install-tl failed immediately with local/repository year mismatch. Use Utah
historic tlnet-final and set tlmgr repository to match for collection installs.
- mkdir R_LIBS_USER before Rscript; set .libPaths + cloud.r-project.org in install_packages.R
- Pre-flight step fails in seconds if cmake or pkg-config missing (avoids 30–60m then fs/rmarkdown/tidyverse verify failure)
- Ncpus=1 for R package compiles; realistic time hint for the long CRAN compile step
R package 'fs' (transitive dep of tidyverse/rmarkdown/sass/bslib) bumped to
1.6.x in CRAN. The new version bundles libuv-v1.52.0 source and requires
cmake to build it. The Linux container apt install list had neither cmake
nor pkg-config, so every clean rebuild failed at:
/bin/bash: line 2: cmake: command not found
make: *** [Makevars:44: libuv] Error 127
ERROR: compilation failed for package 'fs'
The 'fs' failure cascaded into sass, gargle, bslib, rmarkdown, tidyverse,
googledrive, googlesheets4, reprex -- causing verify_r_packages.R to fail
on 'rmarkdown, tidyverse' and aborting the weekly container rebuild.
Adds cmake and pkg-config to the Phase 2 apt install list. Restores the
ability to build R source packages that vendor C/C++ libraries.
Quarto's Lua filter calls rsvg-convert to convert SVG figures to PDF
during PDF builds. librsvg2-dev was present (C headers/lib) but the
binary package librsvg2-bin was missing, causing a FATAL build error:
'Could not convert a SVG to a PDF. Please ensure rsvg-convert is on path'
Also adds rsvg-convert to the Phase 2 verification checks so missing
tools are caught at image build time, not at render time.
- Quarto 1.9.27: Linux (.deb), Windows (direct download; Scoop Extras has 1.8.27)
- R 4.5.2: Linux (CRAN jammy-cran40), Windows (Scoop main/r)
- Baremetal: quarto-actions/setup for both Linux and Windows
- Remove ggrepel version pin (R 4.5.x supports ggrepel 0.9.7)
- Update docs: BUILD.md, CONTAINER_BUILDS.md, docker READMEs
ARG variables have scope issues across RUN commands in Docker, causing
COPY commands to fail. Since these paths are stable, hardcoding them
is simpler and more reliable.
- Moved RUN echo statements after COPY commands in both Dockerfiles
- ARG variables lose scope after RUN commands, causing COPY to fail
- Both Linux and Windows Dockerfiles now follow consistent pattern:
1. Re-declare ARG variables
2. Execute COPY commands immediately
3. Run any status/logging commands after
Docker ARG variables have stage-level scope and must be re-declared
after RUN commands to be available for COPY commands. This fixes the
build error where Docker couldn't find the dependency files.
Changes:
- Moved ARG BOOK_DEPS and ARG BOOK_DOCKER declarations before COPY commands
- Added comment explaining ARG scope behavior
- Both Linux and Windows Dockerfiles updated consistently
Fixes: COPY failed: file not found in build context
Updated both Linux and Windows Dockerfiles to use ARG variables for path
configuration, providing a single source of truth for book/ structure.
Changes:
- Added ARG variables at top of both Dockerfiles (BOOK_ROOT, BOOK_TOOLS, BOOK_DEPS, BOOK_DOCKER)
- Updated all COPY commands to use ARG variables instead of hardcoded paths
- Updated workflow files to use vars.BOOK_DOCKER for Dockerfile paths
- Maintains compatibility with existing build process
Benefits:
- Single location to update if directory structure changes
- Can override paths at build time with --build-arg
- Self-documenting path structure
- Consistent with GitHub Actions repository variables pattern
Updated all COPY commands in Dockerfiles to use correct paths with book/ prefix:
- book/tools/dependencies/requirements.txt
- book/tools/dependencies/install_packages.R
- book/tools/dependencies/tl_packages
- book/docker/linux/verify_r_packages.R
- book/docker/windows/verify_r_packages.R
This fixes container build failures where Docker couldn't find dependency files at the old paths after the monorepo restructure moved all content to the book/ subdirectory.
* Restructure: Move book content to book/ subdirectory
- Move quarto/ → book/quarto/
- Move cli/ → book/cli/
- Move docker/ → book/docker/
- Move socratiQ/ → book/socratiQ/
- Move tools/ → book/tools/
- Move scripts/ → book/scripts/
- Move config/ → book/config/
- Move docs/ → book/docs/
- Move binder → book/binder
Git history fully preserved for all moved files.
Part of repository restructuring to support MLSysBook + TinyTorch.
Pre-commit hooks bypassed for this commit as paths need updating.
* Update pre-commit hooks for book/ subdirectory
- Update all quarto/ paths to book/quarto/
- Update all tools/ paths to book/tools/
- Update config/linting to book/config/linting
- Update project structure checks
Pre-commit hooks will now work with new directory structure.
* Update .gitignore for book/ subdirectory structure
- Update quarto/ paths to book/quarto/
- Update assets/ paths to book/quarto/assets/
- Maintain all existing ignore patterns
* Update GitHub workflows for book/ subdirectory
- Update all quarto/ paths to book/quarto/
- Update cli/ paths to book/cli/
- Update tools/ paths to book/tools/
- Update docker/ paths to book/docker/
- Update config/ paths to book/config/
- Maintain all workflow functionality
* Update CLI config to support book/ subdirectory
- Check for book/quarto/ path first
- Fall back to quarto/ for backward compatibility
- Maintain full CLI functionality
* Create new root and book READMEs for dual structure
- Add comprehensive root README explaining both projects
- Create book-specific README with quick start guide
- Document repository structure and navigation
- Prepare for TinyTorch integration
