#1617 unseeded the module-level rng in tinytorch/src/03_layers/03_layers.py
to make the perceptron milestone produce different weights on every run.
But Linear.__init__ reads weight init from that rng, so the change made
Linear init nondeterministic across the entire codebase. The integration
test tests/integration/test_training_flow.py::test_deep_network_gradient_chain
specifically depends on Linear having stable init; empirical sweep over
5000 random draws shows the test fails 27.5% of the time under unseeded
init, which is why the Windows CI run failed in #1617.
Restore the seed=7 default in 03_layers.py and instead rebind layers.rng
to an unseeded RNG locally inside the perceptron milestone, mirroring
the pattern already used by the XOR milestone (#1618). This keeps the
perceptron's "different weights every run" promise without breaking
unrelated tests.
Milestone 3 (XOR Solved) was unconditionally announcing
'✅ Training Complete - XOR Solved!' and a green '🎉 Success! You Ended
the AI Winter!' panel showing 'Final accuracy: 75.0% (Perfect XOR
solution!)' even when the network was stuck at the classic 75%
dead-ReLU saddle point and had clearly not solved XOR.
Gate both messages on a 0.95 convergence threshold:
- On convergence: keep the existing celebration panel.
- Below threshold: show a yellow warning that training did not
converge, explain the dead-ReLU saddle-point symptom, and tell
the student to re-run the milestone or use a larger hidden
layer instead of moving on to Milestone 03 on a broken result.
Relates to #1613
The migration from np.random.seed() to default_rng() (commit d30257577c)
inadvertently broke the XOR convergence guarantee in milestone 3 (XOR
Solved). The intended seeding line became dead code:
rng = np.random.default_rng(7) # local var, never used
The active weight-init RNG lives at tinytorch.core.layers.rng. With its
default seed (7), the 4-unit hidden layer initializes into a dead-ReLU
saddle point: training stalls at 75% accuracy (one XOR case stuck at
p~=0.5) regardless of how long it runs.
Re-seed tinytorch.core.layers.rng to 1986 (the year of the backprop
paper) immediately before building the model. This restores deterministic
100% convergence in 500 epochs while preserving the original
hidden_size=4 pedagogy and all surrounding educational text.
Verified: before -> 75.0% test accuracy, after -> 100.0% test accuracy.
Relates to #1614
The module-level `rng = np.random.default_rng(7)` at the top of
`src/03_layers/03_layers.py` was reused for every `Linear(...)` weight
init, so every run produced identical weights. Milestone 1's UI text
("No random seed - each run will be different!") contradicted this, and
the issue reporter rightly expected weights to vary.
Drop the seed in two places:
- `src/03_layers/03_layers.py`: the module-level `rng` is now unseeded.
Local seeded RNGs inside test/demo blocks (around lines 829, 848, 981)
are unchanged - those want determinism for reproducible self-tests.
- `milestones/01_1958_perceptron/01_rosenblatt_forward.py`: the data-gen
RNG is also unseeded so the cluster points vary too, matching the
on-screen claim.
Verified by running the milestone twice: cluster positions, predictions,
and accuracy now differ between runs (e.g., 100% "got lucky" vs 0%
"random guessing"). `Linear(...)` weights also differ across fresh
constructions within a single run.
Relates to #1611
Bring the historical milestones into the Lab Guide as native Quarto
chapters, interleaved between the tier Parts so each milestone unlocks
right after the modules that power it. The seven legacy mystnb ABOUT.md
files under tinytorch/milestones/ are removed; their narrative content
now lives at tinytorch/quarto/milestones/ and is wired into both the
PDF book TOC and the website sidebar/navbar.
Verified locally: PDF builds (330 pages, all six milestones present at
expected positions, callouts render correctly), full website renders,
pre-commit run --all-files passes.
The 7 tinytorch/milestones/*/ABOUT.md files contained unique narrative
content (historical context, "aha moment" stories, prerequisite tables,
runnable instructions) but were stranded after the Jupyter Book retirement
— mystnb format, no Quarto renderer, no entry in any TOC.
Port them into the Lab Guide so the historical recreations finally have a
canonical home that ships with the book PDF and the website.
Layout (Option A — interleaved between tiers):
• Foundation Tier (Modules 01–08)
• Foundation Milestones (1958 Perceptron · 1969 XOR · 1986 MLP)
• Architecture Tier (Modules 09–13)
• Architecture Milestones (1998 CNN · 2017 Transformers)
• Optimization Tier (Modules 14–19)
• Optimization Milestones (2018 MLPerf)
• Capstone (Module 20)
Each milestone unlocks immediately after the modules that power it,
matching the "your code recreates history" pedagogy.
Conversion notes:
• mystnb {tip} blocks → :::{.callout-tip title="…"}
• mystnb admonitions → :::{.callout-note title="…"}
• Added a "Milestone Info" callout per chapter (tier · difficulty ·
time · prerequisites) to mirror the "Module Info" header used by the
module chapters.
• milestones/ABOUT.md became milestones/index.qmd (Part overview) so
the new "Foundation Milestones" Part starts with a landing page that
summarises all six milestones.
Wiring:
• tinytorch/quarto/pdf/_quarto.yml — three new Parts interleaved
between the existing tier Parts; comment explains the placement rule.
• tinytorch/quarto/_quarto.yml — sidebar gets three new sections
("🕰️ Foundation Milestones", etc.) and the navbar gains a top-level
"Historical Milestones" link to the new index.
• tito/milestones.qmd is unchanged: it documents the `tito milestone`
CLI (separate purpose from the new narrative chapters).
Verification:
• make pdf produces TinyTorch-Guide.pdf at 330 pages with all six
milestones at the expected logical positions; confirmed visually
that the "Milestone Info" + "What You'll Learn" callouts render
with proper Quarto styling.
• quarto render produces 7 new pages under _build/milestones/.
• pre-commit run passes (incl. the internal-link validator).
Add `width="100%"` to every HTML content and contributor table across all
project READMEs so they render full-width on GitHub instead of collapsing
to natural content width. Cell-level `width="X%"` percentages were already
in place but only take effect once the table itself has an explicit width.
Also update the contributor-sync scripts so the auto-generated tables stay
consistent on the next bot run:
- .github/workflows/contributors/generate_main_readme.py
- .github/workflows/contributors/generate_readme_tables.py
Scope: 27 files, 85 tables. Sub-project READMEs that already use the
"card" pattern (labs/, kits/ content sections with <table width="98%">
wrappers) are intentionally untouched.
- Update Python version requirement from 3.8+ to 3.10+ across badges,
docs, tests, and setup validation
- Rewrite Milestone 05 docs to reflect single synthetic-task script
(01_vaswani_attention.py) replacing old 3-script approach
- Fix repository URLs from placeholder VJ/TinyTorch to
harvard-edge/cs249r_book
- Update contact email to info@mlsysbook.ai
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace np.random.randn/rand/seed with np.random.default_rng(7) across
all 93 source modules, tests, and milestones for reproducible, isolated
random state.
- Fix pool_size=(2,2) → kernel_size=2, stride=2 in 02_lecun_cifar10.py
- Normalize all Conv2D/MaxPool2D/BatchNorm2D references to lowercase d
(Conv2d, MaxPool2d, BatchNorm2d) across comments, strings, and ASCII art
- Fix Conv2d error string in 09_convolutions.py source module
- Delete milestones/extras/ directory (drifted copies, 2 placeholders,
1 broken script — maintenance liability that caused the duplicate bug)
- Add test_milestones_smoke.py: 19 lightweight tests that import every
milestone script and construct every model class in 0.07s, catching
API drift without data downloads
Standardize table formatting across 25 README files to use
HTML tables with consistent styling (thead/tbody, column widths,
bold labels) matching the main README's presentation.
Refresh top-level README with clearer volume naming, HTML branch
table, and consolidated support section. Replace ASCII art with
SVG diagrams in tinytorch milestones and labs. Streamline slides
and kits READMEs. Remove retired challenges/ directory.
Fixed NameError in step_1_profile and step_5_accelerate by adding
Tensor as a parameter. These functions create sample tensors for
profiling/testing but the Tensor class is imported in main(), so
they need it passed as an argument.
Each challenge function now has visual documentation showing:
- The task input/output example
- The ideal attention weight matrix pattern
- Why the pattern is required for the task
Challenge 3 also explains why a fresh model is needed
(sequential training causes "catastrophic forgetting").
Break the monolithic main() into clean, documented functions:
- CONFIG dict for shared hyperparameters
- build_model() for creating fresh model/optimizer/loss
- challenge_1_reversal() - anti-diagonal attention patterns
- challenge_2_copying() - diagonal attention patterns
- challenge_3_mixed() - prefix-conditioned behavior (fresh model)
- print_final_results() - summary table and messages
This makes the code much easier for students to understand
and clearly shows why challenge 3 needs a fresh model.
The transformer was being trained sequentially on reversal then copying,
which caused it to "forget" reversal before the mixed task. Now we
reinitialize the model before challenge 3 so it learns both tasks
together with proper prefix conditioning.
Updates milestone documentation across all site files to match the
actual MILESTONE_SCRIPTS configuration in tito/commands/milestone.py:
- Milestone 01 (Perceptron): requires modules 01-03 (not 01-04)
- Milestone 02 (XOR Crisis): requires modules 01-03 (not 01-02)
- Milestone 05 (Transformers): requires 01-08 + 11-13 (not 01-13)
- Milestone 06 (MLPerf): requires 01-08 + 14-19 (not 01-19)
Also fixes broken link to chapters/milestones.html (directory does not
exist) and corrects path to student notebooks.
Bug fixes:
- Move test_autograd_core.py from 05_dataloader/ to 06_autograd/ (fixes#1127)
- Fix integration test mapping: tests now only run after their dependencies
are available (module 4 loss tests moved to module 7+)
- Remove premature test_unit_function_classes() call in 06_autograd.py
that ran before enable_autograd() (fixes#1128)
- Handle EOFError in milestone prompts for non-interactive mode (fixes#1129)
Improvements:
- Read version from pyproject.toml as single source of truth
- Add try/except for sync prompt in milestone completion
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update remaining 1957→1958 references across all documentation
- Add tito dev commands (preflight, export, validate) to CLI reference
- Update CLI validation script to recognize new dev subcommands
- Fix milestone year references in tests and workflow code
- Update timeline visualization JavaScript
This completes the Perceptron year standardization to align with
the publication year and academic citation format (rosenblatt1958perceptron).
Cherry-picked from: ebf3fb17b (feature/tito-dev-validate)
- Rename milestone directory from 01_1957_perceptron to 01_1958_perceptron
- Update all references to use 1958 (publication year) for consistency
with academic citation format (rosenblatt1958perceptron)
- Changes affect: READMEs, docs, tests, milestone tracker
Rationale: Using 1958 aligns with the publication year and standard
academic citations, while 1957 was the development year.
Cherry-picked from: 28ca41582 (feature/tito-dev-validate)
- Add callout boxes for milestone info and prerequisites
- Standardize section naming (The Aha Moment)
- Add "YOUR Code Powers This" tables
- Fix tito commands to use proper module status checks
- Merge redundant sections for cleaner narrative flow
Test fixes:
- test_dataloader_integration.py: Fix import path (tinytorch.data → tinytorch.core)
- integration_mnist_test.py: Fix Linear import (was aliased but used wrong name)
- test_module_05_dense.py: Fix Dense vs Linear usage (was using wrong variable name)
Milestone fix:
- 01_vaswani_attention.py: Fix indentation in train_epoch function
- Changed 'Run after Module 07' to 'Run after Module 08' (Training)
- Added note that Module 05 (DataLoader) is intentionally skipped
for XOR since it only has 4 data points (no batching needed)
Update Milestone 05 (Transformer) to use DataLoader for batched training:
- Add SequenceDataset class using YOUR Dataset interface
- Update train_epoch to iterate through DataLoader batches
- Update run_challenge to create DataLoaders with configurable batch_size
- Update module requirements to include Module 05 (DataLoader)
This makes DataLoader usage consistent across all milestones from
Module 03 (MLP) onwards, reinforcing the batching pattern.
