40 Commits

Author SHA1 Message Date
Vijay Janapa Reddi
78dd6c1a92 chore(tinytorch): delete 20 stale src/*/ABOUT.md duplicates
Audit (parent commit): every tinytorch/src/NN_xxx/ABOUT.md is a stale
duplicate of tinytorch/quarto/modules/NN_xxx.qmd, with the QMD strictly
newer/richer (each QMD has additional sections like Your TinyTorch,
PyTorch comparison, MLPerf section that ABOUT.md is missing).

Drift confirms ABOUT.md is unmaintained: e.g., 12_attention/ABOUT.md
still says "GPT-3 training (4x inference)" while the QMD was corrected
to 5x. Single-source-of-truth in quarto/modules/ stops the recurrence.

The retired Jupyter Book site (tinytorch/site-legacy/) was the only
historical consumer; it is deleted in the next commit.

Also deleted:
  - tools/dev/fix_about_titles.py     -- one-shot ABOUT.md title fixer
  - tools/dev/fix_mermaid_diagrams.py -- one-shot ABOUT.md mermaid tweaker
Both operated on the now-deleted ABOUT.md files and have no other use.

Updated tinytorch/README.md so the documented repo tree no longer
shows ABOUT.md under each module folder, and repointed a stale
"setup guide" link from site/getting-started.md to quarto/getting-started.qmd.
2026-04-22 15:53:35 -04:00
Vijay Janapa Reddi
d30257577c refactor(tinytorch): migrate from legacy np.random to default_rng(7)
Replace np.random.randn/rand/seed with np.random.default_rng(7) across
all 93 source modules, tests, and milestones for reproducible, isolated
random state.
2026-04-03 17:57:51 -04:00
Vijay Janapa Reddi
be245aef22 fix(tinytorch): batch 3 critical fixes from module audit
M03 Layers:
- Sequential.forward/.__call__ now accept training= parameter and
  pass it through to Dropout layers (eval mode was broken)

M12 Attention:
- Fix memory multiplier: was 7x (double-counted), now 4x incremental
  (1x forward + 1x grad + 2x optimizer). Aligned ABOUT.md to match.
- Fix mask docstring: True/False -> 1/0 to match implementation

M14 Profiling:
- Add __enter__/__exit__ to Profiler (6 progressive tests used
  context manager but class had no protocol support)
- Fix FLOP prose formula: remove batch_size to match implementation

M15 Quantization:
- Fix "symmetric" -> "asymmetric (min-max)" terminology throughout
- Fix Magic Formula prose to match code: q = x/s + zp (not (x-zp)/s)
- Add parentheses to zero-point hint formula for clarity
2026-03-24 08:49:46 -04:00
Vijay Janapa Reddi
850a91adc6 fix(docs): align notebook filenames with tito convention across all docs
Notebooks use short names (tensor.ipynb, not 01_tensor.ipynb) but docs
and Binder postBuild scripts used the prefixed form. This caused broken
Binder links and incorrect paths in troubleshooting guides.

Fixes: harvard-edge/cs249r_book#1176
2026-02-17 18:31:44 -05:00
Vijay Janapa Reddi
e7f9223680 feat(site): convert all 20 ABOUT.md files to MyST notebooks with computed values
Replace hardcoded numerical values across all module ABOUT.md files with
Python-computed values using myst_nb glue() references. Each file is now a
MyST Markdown Notebook that executes inline code cells to compute memory
sizes, FLOPs, compression ratios, and other quantitative values.

Key changes:
- Add file_format: mystnb frontmatter and code-cell blocks to all 20 files
- All arithmetic (memory calculations, speedups, ratios) now computed inline
- Fix multiple arithmetic errors discovered during conversion
- Enable execute_notebooks: "cache" in PDF config for glue resolution
- Fix jupyter-book version constraint in Makefile
2026-02-17 18:11:31 -05:00
Vijay Janapa Reddi
910240a3f6 chore(tinytorch): add VS Code extension and sync module updates
Add the TinyTorch VS Code extension source package and align module code/docs references so APIs, milestones, and progression notes remain consistent across the curriculum.
2026-02-15 14:02:09 -05:00
Vijay Janapa Reddi
81cdbba67b refactor(tinytorch): function decomposition, naming conventions, and progressive disclosure
Three categories of changes across 17 modules:

1. Function decomposition (Modules 01,03,05-15,18-19): Break large
   monolithic functions into focused _helper + orchestrator pattern.
   Each helper teaches one concept with its own unit test.

2. Naming convention fixes (Modules 08,09,11,18,19): Ensure underscore
   convention is consistent — standalone _func in export cells renamed
   to func (public API), monkey-patched method names match target
   visibility, removed unnecessary #| export from internal helpers.

3. Progressive disclosure (Modules 02-05,08,11-15): Remove forward
   references to future modules. Replace "you'll learn in Module N"
   with concrete descriptions. Trim connection maps to only show
   current and prior modules. Keep end-of-module "Next" teasers
   as motivational breadcrumbs.

All 17 modified modules pass their test suites.
2026-02-14 16:52:15 -05:00
Vijay Janapa Reddi
3947b2defa fix(tinytorch): enforce progressive disclosure across 9 modules
Audit found docstrings/comments revealing concepts from later modules.
All edits are docstring/comment-only — no code, imports, or tests changed.

Module 01: Replace neural network terminology with generic math examples
Module 02: Remove gradient flow references, reframe layer terminology
Module 05: Remove optimizer/backward from pipeline diagrams
Module 06: Replace transformer/embedding references with general patterns
Module 07: Replace embedding/transformer terminology with generic terms
Module 10: Replace detailed embedding analysis with brief Module 11 teaser
Module 13: Fix swapped dependency numbers, trim KV-cache explanation
Module 14: Remove quantization/compression references from docstrings
Module 17: Fix factually wrong Module 18 teaser description

231 tests pass across all modified modules.
2026-02-13 10:00:50 -05:00
Vijay Janapa Reddi
0c7509ff35 feat(site): add embedded PDF slide viewer to all module pages
Replace the slide download card with an inline PDF viewer using PDF.js:
- Change grid layout from 4 cards (2x2) to 3 cards in a row
- Add embedded slide viewer with navigation, zoom, and fullscreen
- Load slides from local _static/slides/ for reliable CORS handling
- Add "· AI-generated" subtitle to match audio card pattern
- Use 🔥 icon consistently across all viewers

Affected: 20 module ABOUT.md files + big-picture.md
2026-02-03 12:06:52 -05:00
AndreaMattiaGaravagno
ad94870ed2 docs(slides): specify genai usage (#1149) 2026-02-01 10:01:47 -05:00
Vijay Janapa Reddi
d2b56498b3 Fix convolution FLOPs calculation in profiling example
112 × 112 × 7 × 7 × 3 × 64 × 2 = 236,027,904 (not 235,012,096)
2026-01-27 08:38:18 -05:00
Vijay Janapa Reddi
eab5122691 feat: add slide deck cards to all module pages
- Add 4th card (Slide Deck) to 2x2 grid layout on 18 modules
- Host PDFs via GitHub release (tinytorch-slides-v0.1.0)
- Card order: Audio → Binder → Source → Slides
- Colors: Orange (#f97316), Teal (#14b8a6), Sky Blue (#0ea5e9)
2026-01-25 13:21:06 -05:00
Vijay Janapa Reddi
56c05085d0 style(modules): standardize unit test emoji to test tube
Update unit test markers across all 20 modules for consistency:
- Changed header emoji from microscope to test tube
- Maintains visual consistency across module test sections
2026-01-24 19:04:04 -05:00
Vijay Janapa Reddi
aafd7a8c67 refactor(modules): standardize formatting and fix NBGrader directives
- Standardize test emoji usage (🔬 for unit tests, 🧪 for module tests)
- Restore missing NBGrader solution blocks in Module 15 (quantization)
- Fix missing #| default_exp directives in modules 05, 12, 13, 15, 17
- Remove duplicate #| default_exp directives
- Ensure all exports are only for core functionality (no tests/demos)
- Apply consistent styling across all 20 modules via module-developer agent
2026-01-24 10:07:18 -05:00
Vijay Janapa Reddi
c420fe7858 chore(tinytorch): bump version to v0.1.4
TinyTorch v0.1.4: Educational improvements and module path fixes

Breaking Changes:
- fix: correct module path from core.transformer to core.transformers (14 files)

Educational Enhancements:
- refactor: remove premature backward() methods for cleaner progressive learning
- feat: add educational scaffolding with TODO/hints in Module 20 Capstone
- docs: remove forward references to Module 06 in early modules

Bug Fixes:
- fix: TransformerBlock now supports ff_dim parameter for flexibility
- fix: wrap module print statements in if __name__ guards

Code Quality:
- refactor: reorganize Quantizer class export location
- refactor: improve module integration in tinytorch.__init__.py
- chore: remove outdated TINYTORCH_FORMATTING_STANDARDS.md (415 lines)

Stats: 29 files changed, 357 insertions(+), 711 deletions(-)
2026-01-17 10:25:59 -05:00
Vijay Janapa Reddi
0d076aee26 fix: update tier boundaries across all documentation
Comprehensive update to reflect correct module assignments:
- Foundation Tier: 01-08 (was incorrectly 01-07 in many places)
- Architecture Tier: 09-13 (was incorrectly 08-13 in many places)

Updated files:
- Site pages: intro.md, big-picture.md, getting-started.md
- Tier docs: olympics.md, optimization.md
- TITO docs: milestones.md
- Source ABOUT.md: 09, 10, 11, 12, 13, 14, 16
- Paper diagrams: module_flow.dot, module_flow_horizontal.tex
- Milestones: README.md, 02_1969_xor/ABOUT.md
- Tests: integration/README.md
- CLI: tito/commands/module/test.py
2025-12-19 20:12:24 -05:00
Vijay Janapa Reddi
66739228cd docs: add pedagogical rationale explaining tier structure
Each tier now explains WHY the modules are ordered as they are:

Foundation: Forward Pass (01-04) → Learning (05-07) → Training (08)
- Minimal dependency chain: can't build optimizers without autograd

Architecture: Vision (09) || Language (10-13)
- Parallel tracks for domain-specific needs
- Language: tokenization → embeddings → attention → transformers

Optimization: Measure → Model-Level → Runtime → Validate
- Model-level (15-16): Change the model (quantization, compression)
- Runtime (17-18): Change execution (acceleration, memoization)
- General techniques before specialized applications
2025-12-19 19:49:14 -05:00
Vijay Janapa Reddi
f51f87566b docs: update source module headers for new optimization order 2025-12-19 19:30:43 -05:00
Vijay Janapa Reddi
d203fba8b8 fix: complete module renumbering across entire codebase
Updated all references to reflect new module order:
- Module 05: DataLoader (was 08)
- Module 06: Autograd (was 05)
- Module 07: Optimizers (was 06)
- Module 08: Training (was 07)

Changes include:
- paper/paper.tex: 20+ references, tier descriptions, milestones
- src/: Export commands, dependency diagrams, docstrings
- tests/: Dependency chains, integration tests, README
- tito/: export_utils.py path mappings
- tinytorch/: Auto-generated package file headers

Foundation Tier is now Modules 01-08
Architecture Tier is now Modules 09-13
2025-12-19 17:43:41 -05:00
Vijay Janapa Reddi
bcf81d9490 style: standardize emoji section headers across all TinyTorch modules
- Update all 20 modules with consistent emoji section headers
- Use unique emojis for each section type:
  - 🔗 Prerequisites & Progress
  - 🎯 Learning Objectives
  - 📦 Package Location
  - 📋 Module Dependencies
  - 💡 Introduction/Motivation
  - 📐 Foundations/Math
  - 🏗️ Implementation
  - 🔧 Integration/Utilities
  - 📊 Systems Analysis
  - ⚠️ Warnings/Danger
  - 🧪 Module Integration Test
  - 🤔 ML Systems Thinking
  -  Aha Moment
  - 🚀 Module Summary
- Ensures visual consistency for student navigation across modules
2025-12-18 12:23:10 -05:00
Vijay Janapa Reddi
5321117f09 style: add consistent pill-shaped buttons to module cards
Replace text-only card links with styled 54px pill buttons for
Binder (orange) and GitHub (gray) actions. Audio player height
now matches button height for visual consistency across all 20
module pages.
2025-12-17 20:29:08 -05:00
Vijay Janapa Reddi
7196eb6f9c feat: add embedded audio players to module ABOUT pages
- Add HTML5 audio players for NotebookLM-generated overviews
- Fix tab-set directive nesting (remove stray backticks)
- Fix grid card fence structure (5 backticks for outer {only})
- Audio hosted on GitHub Release tinytorch-audio-v0.1.1
2025-12-17 19:28:44 -05:00
Vijay Janapa Reddi
08ff168328 feat: add Binder/Source/Audio grid cards to module pages
Adds interactive launch cards to all 20 TinyTorch module ABOUT.md files:
- Launch Binder button for browser-based exploration
- View Source link to GitHub implementation
- Audio Overview placeholder for NotebookLM content

Cards wrapped in {only} html directive to exclude from PDF output.
Also removes duplicate README.md files from src modules.
2025-12-17 14:33:04 -05:00
Vijay Janapa Reddi
82984bbff2 refactor(tinytorch): remove section numbers from markdown headers
Standardize header format across all 20 module files by removing
numbered prefixes. Headers now use descriptive titles only, making
maintenance easier when reordering sections.

Changes:
- `## 1. Introduction` → `## Introduction`
- `## Part N: Title` → `## Title`
- `## N.N Subsection` → `## Subsection`

Header hierarchy preserved (H1 for module title, H2 for sections).
2025-12-16 07:18:29 -05:00
Vijay Janapa Reddi
a6f9bc3b0b feat(tinytorch): add module.yaml metadata files for CLI module descriptions
Add machine-readable module.yaml files to each of the 20 modules with
title, subtitle, and description fields. Update tito CLI to read from
these files instead of parsing Python files.

- Create module.yaml in src/NN_*/ directories
- Add YAML parser with validation in tito/core/modules.py
- Update list_modules() to display descriptions from YAML
2025-12-15 20:30:32 -05:00
Vijay Janapa Reddi
d6a96ca2d2 fix(tinytorch): correct Binder URLs and remove broken Colab links
- Update Binder URLs to use urlpath parameter pointing to generated
  notebooks in modules/ directory instead of raw .py files
- Remove Colab links since they cannot run postBuild to generate notebooks
- Fix View Source links to include tinytorch/ prefix
2025-12-15 13:45:31 -05:00
Vijay Janapa Reddi
eec214e335 fix: resolve Jupyter Book build warnings and PDF diagram readability
- Fix broken symlink for 09_convolutions_ABOUT.md
- Fix header level warnings (H1 to H3 jumps) in community.md and intro.md
- Remove broken cross-references to deleted files across site pages
- Fix lexing errors by using text blocks for Unicode characters
- Update mermaid diagram in big-picture.md to use light fill colors
  for PDF compatibility (mermaid-cli does not respect inline color styles)
2025-12-14 14:58:47 -05:00
Vijay Janapa Reddi
f5f1615840 fix: restore pedagogical prerequisites for proper learning progression
Prerequisites now reflect intended learning sequence rather than just
code imports. Understanding why matters as much as what imports.

Pedagogical structure:
- Phase 1 Foundation (01-04): Sequential chain building core concepts
- Phase 2 Training (05-07): Autograd needs loss context, optimizers need gradients
- Phase 3 Architecture: CV track (08-09), NLP track (10-13) branch from foundation
- Phase 4 Optimization (14-19): All require models to optimize (01-13)

Key insight: A student could technically run memoization code with only
Tensor knowledge, but would not understand WHY KV caching matters without
first building attention and transformers. Prerequisites guide learning,
not just code execution.
2025-12-14 14:05:57 -05:00
Vijay Janapa Reddi
672e21e30a fix: update module prerequisites to reflect actual import dependencies
Prerequisites now match code imports rather than pedagogical sequencing.
Each module lists only the modules it actually imports from, making
dependencies accurate and reducing unnecessary barriers to entry.

Key changes:
- Modules 05, 06, 08, 10, 17: Only require Module 01 (Tensor)
- Module 07 Training: Requires 01, 03, 04, 06 (not 01-06)
- Modules 09, 11: Require 01, 05 (Tensor + Autograd)
- Module 12 Attention: Requires 01, 02, 03 (not 01-05, 10-11)
- Module 13 Transformers: Requires 01, 02, 03, 11, 12
- Modules 15, 16: Require 01, 02, 03 (not full stack)
- Module 18: Requires 01, 14 (Tensor + Profiling)
- Module 19: Requires only 14 (Profiling)
2025-12-14 14:00:06 -05:00
Vijay Janapa Reddi
fcf3d8bd12 fix: update all GitHub URLs from mlsysbook/TinyTorch to harvard-edge/cs249r_book
- Update all repository references to point to harvard-edge/cs249r_book
- Fix Binder URLs to include tinytorch/ path prefix
- Fix Colab URLs to include tinytorch/ path prefix
- Update marimo-badges.js with correct repo and path
- Fix dataset documentation URLs
- Update module ABOUT.md files with correct source links

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2025-12-14 12:36:10 -05:00
Vijay Janapa Reddi
c2598e827d docs: improve orientation files based on expert review
Critical fixes:
- Add getting-started.md to PDF TOC (students need setup instructions)
- Move Course Orientation to position #2 in website TOC (better discoverability)
- Add prerequisites warning note at top of getting-started.md

High priority improvements:
- Add "What if validation fails?" troubleshooting section
- Add per-module time estimates table (60-80 hours total)
- Clarify autograd prerequisites (chain rule conceptual knowledge needed)
- Standardize Quick Start terminology (fix links to getting-started.md)
- Add persona-based routing in "What to Read Next" sections

Content cleanup:
- Remove redundant prerequisites from preface.md
- Remove tier overview duplication from preface.md
- Remove MLSysBook from prerequisites.md (already in preface)
- Convert prerequisites.md from 65% bullets to 90% prose
- Simplify learning-journey.md intro (remove awkward meta-section)
- Fix checkpoints -> modules terminology in learning-journey.md
- Restore instructor "coming soon" section in getting-started.md

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-13 16:48:59 -05:00
Vijay Janapa Reddi
69a3e19f2f style: rebrand "Your TinyTorch" to "Your Tiny🔥Torch" across codebase
Consistent branding with flame emoji in all module ABOUTs,
milestone files, site documentation, and Python scripts.
2025-12-13 15:23:28 -05:00
Vijay Janapa Reddi
2e476722ae fix: improve PDF admonition colors and convert to native Sphinx directives
- Add sphinxsetup configuration with distinct colors for each admonition type
  (tip=green, warning=orange, note=blue, caution=red, etc.)
- Convert {admonition} with :class: attribute to native {tip}, {warning},
  {seealso} directives for proper Sphinx type detection in LaTeX output
- Remove unsupported emojis from site markdown files for LaTeX compatibility
- Update codespell ignore list for font names (Heros) and PDF options (FitH)
- Update 20 ABOUT.md files and 16 site/*.md files
2025-12-13 14:58:59 -05:00
Vijay Janapa Reddi
853eb03ee8 style: apply consistent whitespace and formatting across codebase 2025-12-13 14:05:34 -05:00
Vijay Janapa Reddi
efc577f53f fix: improve ABOUT.md accuracy and PDF-compatible Get Started sections
- Fix API signatures to match actual implementations across all modules
- Correct loop counts (Module 09: 6→7 nested loops)
- Fix import paths (Module 14, 18: perf.* not nn.*)
- Clarify vectorized vs explicit loop implementations (Modules 05, 12)
- Replace {grid} cards with admonition-based links for PDF rendering
- Add missing API documentation (Modules 06, 08, 10, 13, 16)
- Update about-generator agent with accuracy requirements
2025-12-13 14:03:15 -05:00
Vijay Janapa Reddi
5e26319fbc style: remove extra blank lines from markdown files
Clean up documentation by collapsing multiple consecutive blank lines
into single blank lines for consistency.
2025-12-13 13:32:53 -05:00
Vijay Janapa Reddi
d7ac86fa82 docs: generate standardized ABOUT.md files for all 19 modules (02-20)
Generated comprehensive ABOUT.md companion documents for all TinyTorch
modules following the canonical Module 01 template structure:

Foundation Tier (Modules 02-07):
- Activations: ReLU, Sigmoid, Tanh, GELU, Softmax with numerical stability
- Layers: Linear, Dropout, Sequential with Xavier/He initialization
- Losses: MSE, CrossEntropy with log-sum-exp stability
- Autograd: Computation graphs, chain rule, backward pass
- Optimizers: SGD, Adam, AdamW with momentum and weight decay
- Training: Training loops, schedulers, gradient clipping, checkpointing

Architecture Tier (Modules 08-13):
- DataLoader: Dataset abstraction, batching, shuffling, iterator protocol
- Spatial: Conv2d, MaxPool2d, AvgPool2d with explicit loops
- Tokenization: Character and BPE tokenizers with vocabulary building
- Embeddings: Embedding tables, positional encoding, lookup operations
- Attention: Scaled dot-product, multi-head attention, causal masking
- Transformers: LayerNorm, MLP, TransformerBlock, complete GPT model

Optimization Tier (Modules 14-20):
- Profiling: Timing, memory measurement, bottleneck identification
- Quantization: INT8 quantization, scale/zero-point, PTQ
- Compression: Magnitude pruning, structured pruning, knowledge distillation
- Memoization: KV cache, gradient checkpointing, cache invalidation
- Acceleration: Vectorization, BLAS, kernel fusion, tiled operations
- Benchmarking: Statistical validity, reproducibility, MLPerf methodology
- Capstone: Complete benchmarking system, submission generation

Each ABOUT.md includes:
- 13 standardized sections matching Module 01 structure
- Embedded code snippets from actual .py implementations
- Progressive terminology (only uses terms from prior modules)
- Mermaid architecture diagrams
- Quantitative questions with calculations
- Production context comparisons (TinyTorch vs PyTorch)
- Seminal paper references in Further Reading
- Tab-set code comparisons with "Your Tiny🔥Torch" branding
2025-12-13 11:24:45 -05:00
Vijay Janapa Reddi
7af42694fb refactor: consolidate ABOUT.md files using symlinks
- Replace site/modules/*_ABOUT.md files with symlinks to src/*/ABOUT.md
- src/ is now the single source of truth for module documentation
- Sync cleaned-up versions (emoji removal) from site/ back to src/
- Remove sync target from Makefile since symlinks handle everything
- Jupyter Book works with symlinks, no build changes needed
2025-12-12 16:43:16 -05:00
Vijay Janapa Reddi
a774c7e4bb refactor(tinytorch): standardize all module ABOUT.md structure
- Move 'Getting Started' section earlier (position 6, after Build → Use → Reflect)
- Add 'Common Pitfalls' section to all modules (3-5 pitfalls with code examples)
- Add 'Production Context' section to all modules (framework comparisons, real-world usage)
- Verify professional emoji usage (no emoji in section headers)
- Apply consistent structure across all 20 modules
2025-12-07 11:13:11 -08:00
Vijay Janapa Reddi
c602f97364 feat: integrate TinyTorch into MLSysBook repository
TinyTorch educational deep learning framework now lives at tinytorch/

Structure:
- tinytorch/src/         - Source modules (single source of truth)
- tinytorch/tito/        - CLI tool
- tinytorch/tests/       - Test suite
- tinytorch/site/        - Jupyter Book website
- tinytorch/milestones/  - Historical ML implementations
- tinytorch/datasets/    - Educational datasets (tinydigits, tinytalks)
- tinytorch/assignments/ - NBGrader assignments
- tinytorch/instructor/  - Teaching materials

Workflows (with tinytorch- prefix):
- tinytorch-ci.yml           - CI/CD pipeline
- tinytorch-publish-dev.yml  - Dev site deployment
- tinytorch-publish-live.yml - Live site deployment
- tinytorch-build-pdf.yml    - PDF generation
- tinytorch-release-check.yml - Release validation

Repository Variables added:
- TINYTORCH_ROOT  = tinytorch
- TINYTORCH_SRC   = tinytorch/src
- TINYTORCH_SITE  = tinytorch/site
- TINYTORCH_TESTS = tinytorch/tests

All workflows use \${{ vars.TINYTORCH_* }} for path configuration.

Note: tinytorch/site/_static/favicon.svg kept as SVG (valid for favicons)
2025-12-05 19:23:18 -08:00