Remove 36 manual page breaks across all chapters to restore continuous text flow, reposition margin cards and figures to eliminate bottom overflows and collisions, and add contact sheet generation tool for multi-page visual verification.
Release-hardening and transformer-era release of TinyTorch, audited as one
tree. This commit brings tinytorch/ and the TinyTorch CI workflows to the
state of the audit/tinytorch-release branch, where the individual changes
are recorded commit by commit.
- Milestones: transformer-era parts (TinyGPT, sequence routing, TinyCopilot,
conversational Q&A), Milestone 06 "MLPerf to Generative Serving (2018)",
Milestone 07 kernels; per-part recording; NumPy-referenced losses and
causality, attention-content and position probes; try-it prompts after a
pass.
- Framework: dropout eval mode, target gradients, float32 stays float32,
layer validation, LoRA adapters.
- tito: loopback login callback with a state nonce, consented progress sync,
TLS-verified update check, Ctrl-C after a pass keeps the pass.
- Grading: graded tests compare values against NumPy references; tests that
could not fail now check something.
- CI: the publish preflight runs its tests and ships the commit it tested;
workflow inputs pass through env; third-party actions pinned.
- Guide and book: read line by line against the code; every demo is a real
recording.
See tinytorch/CHANGELOG.md, section 0.2.0.
- Add encoding='utf-8' to module file reads in test_module_integration, test_training_edge_cases, and test_backward_boundaries
- Add encoding='utf-8' to CLI test fixtures (test_completion_certification, test_progressive_integration_selection, test_community_flow, test_live_submission, test_user_journey)
- Add encoding='utf-8' and errors='replace' to SIMD compiler subprocess and book tools
- Add encoding='utf-8' to Jupyter PID file read/write in workflow.py
- Standardize opening sections of all 24 book chapters to the 4-beat
PyTorch deconstruction pattern (Standard PyTorch, The Illusion,
The Engineering Reality, The Contract).
- Redesign book cover in Swiss Modernist ETH Light palette with centered
Tiny🔥Torch wordmark, monospace inline code tagline, and 20-module
snaking runtime bus terminating in TinyGPT.
- Update book metadata and regenerate publication assets.
The module page drops 'not building col2im', shows the trainable call, and
lists the two failure messages a student will hit. Chapter 17 derives the
three gradients, quotes col2im and Im2colConv2dFunction.backward, reports
the measured training-step speedup, and replaces the write-col2im exercise
with one on recomputing the patch matrix instead of saving it.
The module page lists the new objective, the pattern, the prerequisite on
Conv2d, and the two failure messages a student will actually hit. Chapter
17 quotes im2col and im2col_conv2d, contrasts the offset loop with chapter
9's per-position model, traces the 3x3 example, reports the measured
speedups, and adds a col2im exercise. Chapter 9 points forward to it.
The chapter promised graph capture, systolic arrays, and Ring-AllReduce,
none of which TinyTorch implements. It now walks tinytorch.extensions:
the C++ kernels called through ctypes, the Triton fused bias + GELU, and
the Apple GPU matmul through MPS, quoting the real code and measuring
each against NumPy on one machine (M5 Max): compiled and vectorized C++
is 170x Python and still 75x slower than np.matmul; fusion only pays once
the tangent is vectorized; MPS breaks even near n=1024.
- listings.py quotes C/C++ functions by name and defs nested under an if.
- simd_ops builds with -fveclib=Accelerate on macOS so std::tanh
vectorizes (fused GELU 22.1 ms -> 3.1 ms on 4096x768).
- New extension-paths figure; margin cards no longer name a nonexistent
src/21_extensions/ or systolic TPUs.
- Preface, roadmap, part title, cross-chapter pointers, the Optimization
tier page, and INSTRUCTOR.md describe the chapter as it now is.
Milestone II was written around training TinyGPT on prose, which no
course script does; the attention milestone the course runs appeared
only as a side section, and the CNN milestone not at all. The chapter
now runs in three parts:
1. Convolution on TinyDigits: SimpleCNN.forward quoted from the
script, parameter and multiply-accumulate counts, a matched-budget
comparison with Milestone I's MLP (the MLP matches or beats it at
20, 50, and 100 epochs), the 32x32 arithmetic where convolution's
parameter advantage binds, and the measured 87x per-epoch cost of
Module 09's Python loops.
2. Attention on structured sequences: the reversal/copy/mixed tasks,
measured results over six runs (2, 2, and 4 epochs to threshold),
and why fixed permutations are easy while prefix-conditioned
routing is the real test.
3. Generation with TinyGPT, unchanged, because Part III's profiling,
KV-cache, and capstone chapters build on it.
Summary, exercises, and the welcome roadmap row follow suit.
The book and the lab guide shipped byte-identical cover art
(c4d795ca...), and that art is really the guide's: its tagline, "Don't
just import torch. Build it.", is the guide's subtitle.
Worse, the art's two subtitle lines, "The xv6 of Machine Learning
Systems" and "From Tensors to Systems Acceleration", match neither
artifact's current title. They predate the retitle in 1770d8ea2d, so
the book has been shipping a cover that contradicts its own title page.
The new cover carries the book's own title and subtitle in the brand
palette from guide/pdf/_quarto.yml: navy field against the guide's
white, the flame mark, and the twenty modules laid out as the four
tiers with the capstone accented. It is generated from
book/tools/build_cover.py rather than exported from a design tool, so
the next retitle is an edit to a source file instead of a re-export
nobody can reproduce.
The guide's cover is untouched.
Vijay Janapa Reddi