The module declares rng = np.random.default_rng(7) at the top so that all
stochastic operations are reproducible for a given seed. Dropout ignored this
and called np.random.random(shape), which draws from the global unseeded state.
Masks were different on every run even when the module seed was set, making
Dropout unit tests non-reproducible and causing hard-to-debug numerical
differences between training runs.
Changed to rng.random(shape) so Dropout participates in the same reproducible
stream as every other random operation in the module.
#1617 unseeded the module-level rng in tinytorch/src/03_layers/03_layers.py
to make the perceptron milestone produce different weights on every run.
But Linear.__init__ reads weight init from that rng, so the change made
Linear init nondeterministic across the entire codebase. The integration
test tests/integration/test_training_flow.py::test_deep_network_gradient_chain
specifically depends on Linear having stable init; empirical sweep over
5000 random draws shows the test fails 27.5% of the time under unseeded
init, which is why the Windows CI run failed in #1617.
Restore the seed=7 default in 03_layers.py and instead rebind layers.rng
to an unseeded RNG locally inside the perceptron milestone, mirroring
the pattern already used by the XOR milestone (#1618). This keeps the
perceptron's "different weights every run" promise without breaking
unrelated tests.
The module-level `rng = np.random.default_rng(7)` at the top of
`src/03_layers/03_layers.py` was reused for every `Linear(...)` weight
init, so every run produced identical weights. Milestone 1's UI text
("No random seed - each run will be different!") contradicted this, and
the issue reporter rightly expected weights to vary.
Drop the seed in two places:
- `src/03_layers/03_layers.py`: the module-level `rng` is now unseeded.
Local seeded RNGs inside test/demo blocks (around lines 829, 848, 981)
are unchanged - those want determinism for reproducible self-tests.
- `milestones/01_1958_perceptron/01_rosenblatt_forward.py`: the data-gen
RNG is also unseeded so the cluster points vary too, matching the
on-screen claim.
Verified by running the milestone twice: cluster positions, predictions,
and accuracy now differ between runs (e.g., 100% "got lucky" vs 0%
"random guessing"). `Linear(...)` weights also differ across fresh
constructions within a single run.
Relates to #1611
Audit (parent commit): every tinytorch/src/NN_xxx/ABOUT.md is a stale
duplicate of tinytorch/quarto/modules/NN_xxx.qmd, with the QMD strictly
newer/richer (each QMD has additional sections like Your TinyTorch,
PyTorch comparison, MLPerf section that ABOUT.md is missing).
Drift confirms ABOUT.md is unmaintained: e.g., 12_attention/ABOUT.md
still says "GPT-3 training (4x inference)" while the QMD was corrected
to 5x. Single-source-of-truth in quarto/modules/ stops the recurrence.
The retired Jupyter Book site (tinytorch/site-legacy/) was the only
historical consumer; it is deleted in the next commit.
Also deleted:
- tools/dev/fix_about_titles.py -- one-shot ABOUT.md title fixer
- tools/dev/fix_mermaid_diagrams.py -- one-shot ABOUT.md mermaid tweaker
Both operated on the now-deleted ABOUT.md files and have no other use.
Updated tinytorch/README.md so the documented repo tree no longer
shows ABOUT.md under each module folder, and repointed a stale
"setup guide" link from site/getting-started.md to quarto/getting-started.qmd.
Replace np.random.randn/rand/seed with np.random.default_rng(7) across
all 93 source modules, tests, and milestones for reproducible, isolated
random state.
M03 Layers:
- Sequential.forward/.__call__ now accept training= parameter and
pass it through to Dropout layers (eval mode was broken)
M12 Attention:
- Fix memory multiplier: was 7x (double-counted), now 4x incremental
(1x forward + 1x grad + 2x optimizer). Aligned ABOUT.md to match.
- Fix mask docstring: True/False -> 1/0 to match implementation
M14 Profiling:
- Add __enter__/__exit__ to Profiler (6 progressive tests used
context manager but class had no protocol support)
- Fix FLOP prose formula: remove batch_size to match implementation
M15 Quantization:
- Fix "symmetric" -> "asymmetric (min-max)" terminology throughout
- Fix Magic Formula prose to match code: q = x/s + zp (not (x-zp)/s)
- Add parentheses to zero-point hint formula for clarity
Notebooks use short names (tensor.ipynb, not 01_tensor.ipynb) but docs
and Binder postBuild scripts used the prefixed form. This caused broken
Binder links and incorrect paths in troubleshooting guides.
Fixes: harvard-edge/cs249r_book#1176
Replace hardcoded numerical values across all module ABOUT.md files with
Python-computed values using myst_nb glue() references. Each file is now a
MyST Markdown Notebook that executes inline code cells to compute memory
sizes, FLOPs, compression ratios, and other quantitative values.
Key changes:
- Add file_format: mystnb frontmatter and code-cell blocks to all 20 files
- All arithmetic (memory calculations, speedups, ratios) now computed inline
- Fix multiple arithmetic errors discovered during conversion
- Enable execute_notebooks: "cache" in PDF config for glue resolution
- Fix jupyter-book version constraint in Makefile
Add the TinyTorch VS Code extension source package and align module code/docs references so APIs, milestones, and progression notes remain consistent across the curriculum.
Audit and fix consistency issues across all module source files:
- Standardize ML Systems header to "ML Systems Reflection Questions" (01, 13)
- Fix section ordering: test_module before ML Systems in modules 16, 17
- Rename demo_spatial() to demo_convolutions() to match module name (09)
- Rename demo_*_with_profiler() to explore_*_with_profiler() (15, 16, 17)
- Fix test naming to use test_unit_* prefix consistently (03, 05, 11, 12)
- Add missing emojis in test_module/demo patterns (02, 15)
- Standardize tito command format to number-only (01, 03, 06, 07, 18)
- Fix implementation headers: hyphen to colon separator (09, 12)
- Add missing "Where This Code Lives" package section (13)
- Fix export command in module summary (05, 06)
Three categories of changes across 17 modules:
1. Function decomposition (Modules 01,03,05-15,18-19): Break large
monolithic functions into focused _helper + orchestrator pattern.
Each helper teaches one concept with its own unit test.
2. Naming convention fixes (Modules 08,09,11,18,19): Ensure underscore
convention is consistent — standalone _func in export cells renamed
to func (public API), monkey-patched method names match target
visibility, removed unnecessary #| export from internal helpers.
3. Progressive disclosure (Modules 02-05,08,11-15): Remove forward
references to future modules. Replace "you'll learn in Module N"
with concrete descriptions. Trim connection maps to only show
current and prior modules. Keep end-of-module "Next" teasers
as motivational breadcrumbs.
All 17 modified modules pass their test suites.
The formula sqrt(1/fan_in) is actually LeCun initialization (1998),
not Xavier/Glorot. True Xavier uses sqrt(2/(fan_in+fan_out)).
- Rename XAVIER_SCALE_FACTOR → INIT_SCALE_FACTOR
- Update all comments to say "LeCun-style initialization"
- Add note explaining difference between LeCun, Xavier, and He init
- Keep the simpler formula for pedagogical clarity
Fixes#1161
Replace the slide download card with an inline PDF viewer using PDF.js:
- Change grid layout from 4 cards (2x2) to 3 cards in a row
- Add embedded slide viewer with navigation, zoom, and fullscreen
- Load slides from local _static/slides/ for reliable CORS handling
- Add "· AI-generated" subtitle to match audio card pattern
- Use 🔥 icon consistently across all viewers
Affected: 20 module ABOUT.md files + big-picture.md
Improve ~43 error messages across 12 modules to follow the
What/Why/Fix pattern (❌💡🔧) that teaches students at the
moment they hit an error:
- ❌ What failed (with actual tensor shapes/values)
- 💡 Why it failed (conceptual insight)
- 🔧 How to fix (concrete code using their data)
Key improvements:
- Add anticipatory checks for common mistakes (e.g., 3D input
to Conv2D when 4D expected suggests adding batch dimension)
- Dropout error now explains p is DROP probability, not KEEP
- Shape mismatch errors show both dimensions and suggest fixes
- Abstract method errors provide implementation templates
Modules updated: tensor, layers, dataloader, optimizers,
convolutions, tokenization, embeddings, attention, quantization,
acceleration, memoization, benchmarking
All 20 module tests pass.
Update unit test markers across all 20 modules for consistency:
- Changed header emoji from microscope to test tube
- Maintains visual consistency across module test sections
- Standardize test emoji usage (🔬 for unit tests, 🧪 for module tests)
- Restore missing NBGrader solution blocks in Module 15 (quantization)
- Fix missing #| default_exp directives in modules 05, 12, 13, 15, 17
- Remove duplicate #| default_exp directives
- Ensure all exports are only for core functionality (no tests/demos)
- Apply consistent styling across all 20 modules via module-developer agent
Module 03 (Linear layer) was incorrectly passing requires_grad=True
to Tensor constructor, which violates progressive disclosure design.
The requires_grad parameter is introduced in Module 06 via monkey
patching of Tensor.__init__. Module 03 should work independently
without depending on autograd functionality.
Changes:
- Remove requires_grad=True from weight/bias Tensor initialization
- Update ABOUT.md to clarify gradient tracking is enabled in Module 06
This fixes the issue reported where students working sequentially
through modules would get errors in Module 03 before completing
Module 06.
Closes#1101
Replace text-only card links with styled 54px pill buttons for
Binder (orange) and GitHub (gray) actions. Audio player height
now matches button height for visual consistency across all 20
module pages.
- Module 01: Remove requires_grad, grad, backward() from Tensor class
Students learn pure tensor math first without gradient concepts
- Module 02: Remove requires_grad propagation from Softmax
Activations are now forward-only until autograd is enabled
- Module 03: Remove requires_grad=True from layer weights
Layers store parameters without gradient flags
- Module 05: Update enable_autograd() to ADD gradient infrastructure
Now adds requires_grad, grad, and backward() to Tensor class
Uses helper functions for tensors created before autograd
- Module 09: Remove Conv2dBackward, MaxPool2dBackward classes
Convolutions are now forward-only, no Module 05 import
- Module 11: Remove EmbeddingBackward import and usage
Embeddings are now forward-only
- Modules 12, 13: Remove requires_grad from mask/param tensors
This implements true progressive disclosure: concepts are introduced
only when students are ready to learn them. Gradient tracking is now
completely absent from Modules 01-04 and added in Module 05.
Adds interactive launch cards to all 20 TinyTorch module ABOUT.md files:
- Launch Binder button for browser-based exploration
- View Source link to GitHub implementation
- Audio Overview placeholder for NotebookLM content
Cards wrapped in {only} html directive to exclude from PDF output.
Also removes duplicate README.md files from src modules.
Add machine-readable module.yaml files to each of the 20 modules with
title, subtitle, and description fields. Update tito CLI to read from
these files instead of parsing Python files.
- Create module.yaml in src/NN_*/ directories
- Add YAML parser with validation in tito/core/modules.py
- Update list_modules() to display descriptions from YAML
- Update Binder URLs to use urlpath parameter pointing to generated
notebooks in modules/ directory instead of raw .py files
- Remove Colab links since they cannot run postBuild to generate notebooks
- Fix View Source links to include tinytorch/ prefix
- Fix broken symlink for 09_convolutions_ABOUT.md
- Fix header level warnings (H1 to H3 jumps) in community.md and intro.md
- Remove broken cross-references to deleted files across site pages
- Fix lexing errors by using text blocks for Unicode characters
- Update mermaid diagram in big-picture.md to use light fill colors
for PDF compatibility (mermaid-cli does not respect inline color styles)
- Update all repository references to point to harvard-edge/cs249r_book
- Fix Binder URLs to include tinytorch/ path prefix
- Fix Colab URLs to include tinytorch/ path prefix
- Update marimo-badges.js with correct repo and path
- Fix dataset documentation URLs
- Update module ABOUT.md files with correct source links
🤖 Generated with [Claude Code](https://claude.com/claude-code)
- Add sphinxsetup configuration with distinct colors for each admonition type
(tip=green, warning=orange, note=blue, caution=red, etc.)
- Convert {admonition} with :class: attribute to native {tip}, {warning},
{seealso} directives for proper Sphinx type detection in LaTeX output
- Remove unsupported emojis from site markdown files for LaTeX compatibility
- Update codespell ignore list for font names (Heros) and PDF options (FitH)
- Update 20 ABOUT.md files and 16 site/*.md files
- Replace site/modules/*_ABOUT.md files with symlinks to src/*/ABOUT.md
- src/ is now the single source of truth for module documentation
- Sync cleaned-up versions (emoji removal) from site/ back to src/
- Remove sync target from Makefile since symlinks handle everything
- Jupyter Book works with symlinks, no build changes needed
- Move 'Getting Started' section earlier (position 6, after Build → Use → Reflect)
- Add 'Common Pitfalls' section to all modules (3-5 pitfalls with code examples)
- Add 'Production Context' section to all modules (framework comparisons, real-world usage)
- Verify professional emoji usage (no emoji in section headers)
- Apply consistent structure across all 20 modules
Additional cleanup following module review:
- Removed redundant __call__ method from Linear (inherits from Layer)
- Fixed Dropout docstrings to correctly describe inference behavior
- Simplified Sequential.parameters() by removing unnecessary hasattr check
All 61 tests still passing after cleanup
Simplifies the layers module API by removing alias proliferation that could confuse students in a pedagogical framework.
Changes:
- Rename SimpleModel → Sequential (matches PyTorch naming)
- Remove create_mlp() and MLP alias (taught in milestones, not core modules)
- Remove input_size/output_size aliases from Linear (keep only in_features/out_features)
- Update all tests to use explicit Sequential composition
- Fix dtype test to validate float32 normalization (TinyTorch's design)
Module focus: Individual building blocks (Linear, Dropout, Sequential container)
MLP construction: Taught in Milestone 03 (1986 MLP) using manual composition
Rationale:
- Progressive disclosure: students learn explicit composition first
- API clarity: one way to do things reduces cognitive load
- Separation of concerns: modules teach primitives, milestones teach patterns
All tests passing: 48/48 in module 03, 214/221 across all modules
Add SimpleModel as a minimal container for explicit layer composition.
Used by quantization, compression, and capstone modules for:
- Collecting parameters from multiple layers
- Running integration tests
- Enabling optimization functions that need a model object
This consolidates SimpleModel definitions that were scattered across modules.
