Plain words and formal posture across the landing site, labs, slides, TinyTorch
guide, MLS-im docs and Hardware Kits: removed hype and intensifiers, contractions
and em-dashes in expository prose, announcers and wordy verbs. Wording only; no
code, links, numbers or claims changed.
Release-hardening and transformer-era release of TinyTorch, audited as one
tree. This commit brings tinytorch/ and the TinyTorch CI workflows to the
state of the audit/tinytorch-release branch, where the individual changes
are recorded commit by commit.
- Milestones: transformer-era parts (TinyGPT, sequence routing, TinyCopilot,
conversational Q&A), Milestone 06 "MLPerf to Generative Serving (2018)",
Milestone 07 kernels; per-part recording; NumPy-referenced losses and
causality, attention-content and position probes; try-it prompts after a
pass.
- Framework: dropout eval mode, target gradients, float32 stays float32,
layer validation, LoRA adapters.
- tito: loopback login callback with a state nonce, consented progress sync,
TLS-verified update check, Ctrl-C after a pass keeps the pass.
- Grading: graded tests compare values against NumPy references; tests that
could not fail now check something.
- CI: the publish preflight runs its tests and ships the commit it tested;
workflow inputs pass through env; third-party actions pinned.
- Guide and book: read line by line against the code; every demo is a real
recording.
See tinytorch/CHANGELOG.md, section 0.2.0.
Callout titles were written five ways (Coming Up: X — Y, Coming Up — X: Y,
hyphens, and none on 01/15/16); each page now has an 'Up next' callout
where it had one, a linked Next: line, and a real 'How later modules use
this one' heading instead of assorted bold pseudo-headings. The
boilerplate '@tbl-... traces how this module is reused' sentence is gone.
Module Info prerequisite leads use one phrasing.
- Top of page: the tall audio and Binder cards become one compact row
(the Overview heading moves up about 150px); the Binder button says
the session is discarded when you leave.
- Commands: each command's comment sits on its own line so the block
survives a phone screen; the four steps `complete` runs move into
"How you know it works".
- How you know it works: unit tests and integration files are lists
(two columns on wide screens), and test names are escaped, since two
`__init__` names on Module 19 were rendering as bold "init".
- What you write: a stacked definition list instead of a two-column
table that clipped on phones.
- Headings in sentence case on all 20 pages; one "What you're not
building yet" label; H3/H4 a clearer step down from H2.
- Code blocks scroll instead of soft-wrapping, and the copy button no
longer covers the first line.
- Slides viewer: on a load failure the spinner stops and a plain
download link remains.
- Module 20: one name (Torch Olympics), Capstone in its info line, and
the Olympics section moved ahead of the troubleshooting sections.
- 13 and 20 say "module" for site pages; 15's lead sentence reworded.
Every module page now has the same shape: Overview, Start and finish,
Learning Objectives, What You'll Build, What you write, How you know
it works, When it fails, Finished? Read why, What's Next. Pages drop
from 530-760 source lines to about 110.
Four sections are generated from the repository at render time:
- module-checks: the unit tests the notebook runs (by their headings
in src/), the integration files `tito module complete` selects
(same rule as tito), and the milestones this module unlocks (parsed
from tito/commands/milestone.py).
- book-chapter: the companion chapter by its real title, with the PDF
link and a note that the book prints reference solutions.
- stub-failure: what a bare NotImplementedError means.
- (from earlier) module-workflow and what-you-write.
"When it fails" lists at most three messages per module, each one
reproduced by injecting the named bug into src/ and running that
module's unit tests; none is paraphrased.
Removed from every page: Core Concepts, API Reference, Production
Context with its PyTorch tabs, Check Your Understanding / Self-Check,
Key Takeaways, Further Reading (now in the book), Get Started (it
repeated the Binder link and linked the full solution), and the
implementation-roadmap tables, which listed pre-solved scaffold as
student work. The three optimization-tier narrative sections
(14, 17, 18) are kept aside for the tier hub.
Each of the 20 module pages carried an identical 298-line HTML block
(audio, Binder, source link, slide viewer) differing only in slug,
notebook, and slide name. Move it into a guide/_extensions/tinytorch
shortcode driven by page front matter (module, notebook, slides), and
move the H1 into a front-matter title.
Rendered DOM is identical for modules 02-20. Module 01 now gets the
same title block as the others (its H1 had not been promoted), and
every page's browser title reads 'Module NN: Name' instead of the bare
file slug.
Per-page review against src/ found four snippets that do not run and two
that run but do the wrong thing.
Do not run:
- 11_embeddings: `Tokenizer(vocab_size=50000)`. Tokenizer is the
interface base class with no __init__, so this raises TypeError.
BPETokenizer is the one that takes vocab_size.
- 06_autograd documented `set_grad_enabled(mode: bool)`. That name
appears nowhere in the repo. The row was also a duplicate: the real
API, is_grad_enabled(), was already documented on the line above.
- 19_benchmarking documented `compare_submissions`. Zero occurrences in
src/19_benchmarking; the only occurrence in the repo was that table
row.
- 09_convolutions promised module 10 builds a WordPiece tokenizer, in
two places. It builds CharTokenizer and BPETokenizer; WordPiece
appears only as an aside about BERT.
Run, but wrong:
- 04_losses showed `return LogSoftmax.apply(x)`, dropping the dim
argument the wrapper accepts (src passes `dim=dim`). A student gets a
log_softmax whose dim= is silently ignored, i.e. the wrong axis with
no error.
- 16_compression's structured_prune pruned every Linear. src collects
them and prunes `linears[:-1]`, deliberately excluding the classifier
because "its output channels are the classes, so zeroing them removes
classes, not neurons". Copying the listing silently deletes output
classes. Also switched to src's stable argsort.
Two reasoning errors:
- 19_benchmarking's Pareto example declared model C (12ms, 91%)
dominated by A (10ms, 90%). A is faster but less accurate, so neither
dominates and C was Pareto-optimal. The page's own Q4 analyses the
identical structure correctly. C is now 89% so the example teaches
what it claims, with a note on the near-miss case.
- 13_transformers assumed the output projection is weight-tied ("0
extra"). src builds `Linear(embed_dim, vocab_size, bias=False)` and
counts it, and the page's own figure alt text calls it a separate
head. Also: the causal mask was described as upper triangular filled
with -inf, while the listing directly beneath it is np.tril of 1.0/0.0.
And the measurement overclaims, all against milestones/06_2018_mlperf,
which states compression and speedup "are outcomes to measure, not
benefits the script assumes" and that a small workload "may become
slower": "compress 8x and speed up generation 10x" (the repo ceiling is
4x), INT8 "speeds up inference 2x", pruning "halves memory" (capstone's
own measure_memory sums param.data.nbytes, "zeros still occupy
storage"), "sparsity creates memory savings", pruning "while preserving
accuracy", profiling "routinely uncovers 10x speedups", and vectorised
NumPy "10-100x faster". 15_quantization also pointed at module 16 for
quantization-aware training, which module 16 does not implement and
which 15's own "What You're NOT Building" list already excluded.
Vijay Janapa Reddi