Created standardized milestone documentation following the M06 pattern:
- M01 (1957 Perceptron): Forward pass vs trained model progression
- M02 (1969 XOR): Crisis demonstration and multi-layer solution
- M03 (1986 MLP): TinyDigits and MNIST hierarchical learning
- M04 (1998 CNN): Spatial operations on digits and CIFAR-10
- M05 (2017 Transformer): Q&A and dialogue generation with attention
Each README includes:
- Historical context and significance
- Required modules with clear dependencies
- Milestone structure explaining each script's purpose
- Expected results and performance metrics
- Key learning objectives and conceptual insights
- Running instructions with proper commands
- Further reading references
- Achievement unlocked summaries
This establishes single source of truth for milestone documentation
and provides students with comprehensive guides for each checkpoint.
- Added jupyter-book>=0.15.0,<1.0.0 dependency for documentation builds
- This dependency is referenced by GitHub Actions workflows
- Required for both HTML and PDF book generation
Cleaned up temporary AI-generated analysis files:
- modules/15_quantization/FIXES_APPLIED.md
- modules/15_quantization/FIXES_TO_APPLY.md
- modules/16_compression/FIXES_REQUIRED.md
- modules/17_memoization/FIXES_APPLIED.md
- Plus other untracked analysis files
These were temporary debugging/review artifacts. Now covered by
.gitignore patterns to prevent future accumulation.
Added comprehensive patterns to ignore AI-generated temporary reports:
- Module review reports (*_REPORT*.md)
- Analysis summaries (*_SUMMARY.md, *_ANALYSIS.md)
- Fix tracking (*_FIXES*.md, *_CHANGES*.md)
- Verification scripts (VERIFY_*.py)
- Other temporary docs (*_CHECKLIST.md, *_GUIDE.md, etc.)
These files are generated during module reviews and debugging sessions
but are not part of the permanent codebase documentation.
Added module.yaml for Module 20 (Competition & Validation):
- Module configuration and learning objectives
- Prerequisites and skill development tracking
- Test coverage and connection documentation
This module brings together all optimization techniques learned
in modules 14-18 for competition preparation.
Added all module development files to modules/XX_name/ directories:
Module notebooks and scripts:
- 18 modules with .ipynb and .py files (01-20, excluding some gaps)
- Moved from modules/source/ to direct module directories
- Includes tensor, autograd, layers, transformers, optimization modules
Module README files:
- Added README.md for modules with additional documentation
- Complements ABOUT.md files added earlier
This completes the module restructuring:
- Before: modules/source/XX_name/*_dev.{py,ipynb}
- After: modules/XX_name/*_dev.{py,ipynb}
All development happens directly in numbered module directories now.
Documentation updates across the codebase:
Root documentation:
- README.md: Updated references from book/ to site/
- CONTRIBUTING.md: Updated build and workflow instructions
- .shared-ai-rules.md: Updated AI assistant rules for new structure
GitHub configuration:
- Issue templates updated for new module locations
- Workflow references updated from book/ to site/
docs/ updates:
- STUDENT_QUICKSTART.md: New paths and structure
- module-rules.md: Updated module development guidelines
- NBGrader documentation: Updated for module restructuring
- Archive documentation: Updated references
Module documentation:
- modules/17_memoization/README.md: Updated after reordering
All documentation now correctly references:
- site/ instead of book/
- modules/XX_name/ instead of modules/source/
Re-exported all modules after restructuring:
- Updated _modidx.py with new module locations
- Removed outdated autogeneration headers
- Updated all core modules (tensor, autograd, layers, etc.)
- Updated optimization modules (quantization, compression, etc.)
- Updated TITO commands for new structure
Changes include:
- 24 tinytorch/ module files
- 24 tito/ command and core files
- Updated references from modules/source/ to modules/
All modules re-exported via nbdev from their new locations.
Completed restructuring: modules/source/XX_name/ → modules/XX_name/
All module development files moved to their numbered directories:
- modules/01_tensor/tensor_dev.{py,ipynb}
- modules/02_activations/activations_dev.{py,ipynb}
- ... (modules 03-20)
Removed obsolete source structure:
- modules/source/01_tensor/ through modules/source/20_capstone/
- modules/source/20_competition/ (legacy competition module)
- 43 files total (21 modules × 2 files each + 1 module.yaml)
This simplifies the module structure and makes development files
easier to find alongside their ABOUT.md and README.md files.
You were right - 150 samples was too small for decent accuracy.
Following Andrej Karpathy's "~1000 samples" educational dataset philosophy.
Results:
- Before (150 samples): 19% test accuracy (too small!)
- After (1000 samples): 79.5% test accuracy (decent!)
Changes:
- Increased training: 150 → 1000 samples (100 per digit class)
- Increased test: 47 → 200 samples (20 per digit class)
- Perfect class balance: 0.00 std deviation
- File size: 51 KB → 310 KB (still tiny for USB stick)
- Training time: ~3-5 sec → ~8-10 sec (still fast)
Updated:
- create_tinydigits.py: Load from sklearn, generate 1K samples
- train.pkl: 258 KB (1000 samples, perfectly balanced)
- test.pkl: 52 KB (200 samples, balanced)
- README.md: Updated all documentation with new sizes
- mlp_digits.py: Updated docstring to reflect 1K dataset
Dataset Philosophy:
"~1000 samples is the sweet spot for educational datasets"
- Small enough: Trains in seconds on CPU
- Large enough: Achieves decent accuracy (~80%)
- Balanced: Perfect stratification across all classes
- Reproducible: Fixed seed=42 for consistency
Still perfect for TinyTorch-on-a-stick vision:
- 310 KB fits on any USB drive
- Works on RasPi0
- No downloads needed
- Offline-first education
Replaces sklearn-sourced digits_8x8.npz with TinyTorch-branded dataset.
Changes:
- Created datasets/tinydigits/ (~51KB total)
- train.pkl: 150 samples (15 per digit class 0-9)
- test.pkl: 47 samples (balanced across digits)
- README.md: Full curation documentation
- LICENSE: BSD 3-Clause with sklearn attribution
- create_tinydigits.py: Reproducible generation script
- Updated milestones to use TinyDigits:
- mlp_digits.py: Now loads from datasets/tinydigits/
- cnn_digits.py: Now loads from datasets/tinydigits/
- Removed old data:
- datasets/tiny/ (67KB sklearn duplicate)
- milestones/03_1986_mlp/data/ (67KB old location)
Dataset Strategy:
TinyTorch now ships with only 2 curated datasets:
1. TinyDigits (51KB) - 8x8 digits for MLP/CNN milestones
2. TinyTalks (140KB) - Q&A pairs for transformer milestone
Total: 191KB shipped data (perfect for RasPi0 deployment)
Rationale:
- Self-contained: No downloads, works offline
- Citable: TinyTorch educational infrastructure for white paper
- Portable: Tiny footprint enables edge device deployment
- Fast: <5 sec training enables instant student feedback
Updated .gitignore to allow TinyTorch curated datasets while
still blocking downloaded large datasets.
Deleted 5 README/documentation files with stale information:
- 01_1957_perceptron/README.md
- 02_1969_xor/README.md
- 03_1986_mlp/README.md
- 04_1998_cnn/README.md
- 05_2017_transformer/PERFORMANCE_METRICS_DEMO.md
Issues with these files:
- Wrong file names (rosenblatt_perceptron.py, train_mlp.py, train_cnn.py)
- Old paths (examples/datasets/)
- Duplicate content (already in Python file docstrings)
- Could not be kept in sync with code
Documentation now lives exclusively in comprehensive Python docstrings
at the top of each milestone file, ensuring it stays accurate and
students see rich context when running files.
Deleted vaswani_shakespeare.py and get_shakespeare() from data_manager:
- 45-60 minute training time (too slow for educational demos)
- Required external download from Karpathy's char-rnn repo
- Replaced by faster TinyTalks ChatGPT milestone (3-5 min training)
Primary transformer milestone is now vaswani_chatgpt.py:
- Uses TinyTalks Q&A dataset (already in repo)
- Fast training with clear learning signal (Q&A format)
- Better pedagogical value (students see transformer learn to chat)
Removed achievement/gamification system that was unused:
- milestone_dashboard.py (620+ lines, only 1 file used it)
- .milestone_progress.json (progress tracking data)
- perceptron_trained_v2.py (only dashboard user, duplicate of perceptron_trained.py)
Rationale:
- Dashboard was used by only 1 of 15 milestone files
- Milestones are educational stories, not standardized tests
- Achievement badges felt gimmicky for ML systems learning
- Custom Rich UI in each file is clearer and more educational
- Reduces dependencies (removed psutil system monitoring)
- Remove overly broad patterns (*_ANALYSIS.md, *_AUDIT.md)
- Make report patterns more specific (MODULE_REVIEW_REPORT_*.md)
- Add clear comments explaining why directories are ignored
- Keep dataset ignores (data/, datasets/) as they are downloaded files
Create comprehensive guidelines for git commits, code quality, testing, and development workflow that apply to Cursor, Claude, and any other AI assistants
- Document two-tier testing approach (inline vs integration)
- Explain purpose and scope of each test type
- Provide test coverage matrix for all 20 modules
- Include testing workflow for students and instructors
- Add best practices and common patterns
- Show current status: 11/15 inline tests passing, all 20 modules have test infrastructure
- Add tests/16_quantization with run_all_tests.py and integration test
- Add tests/17_compression with run_all_tests.py and integration test
- Add tests/18_acceleration with run_all_tests.py and integration test
- Add tests/19_benchmarking with run_all_tests.py and integration test
- Add tests/20_capstone with run_all_tests.py and integration test
- All test files marked as pending implementation with TODO markers
- Completes test directory structure for all 20 modules
- Rename tests/14_kvcaching to tests/14_profiling
- Rename tests/15_profiling to tests/15_memoization
- Aligns test structure with optimization tier reorganization
- Import Module base class from core.layers
- Fix embeddings import path (text.embeddings not core.embeddings)
- Fix attention import (MultiHeadAttention not SelfAttention)
- Fix transformer import path (models.transformer not core.transformers)
- Handle missing functional module gracefully with try/except
- Update __all__ exports to match available components
