3bdfddca51
Clean, dependency-driven organization: - Part I (1-5): MLPs for XORNet - Part II (6-10): CNNs for CIFAR-10 - Part III (11-15): Transformers for TinyGPT Key improvements: - Dropped modules 16-17 (regularization/systems) to maintain scope - Moved normalization to module 13 (Part III where it's needed) - Created three CIFAR-10 examples: random, MLP, CNN - Each part introduces ONE major innovation (FC → Conv → Attention) CIFAR-10 now showcases progression: - test_random_baseline.py: ~10% (random chance) - train_mlp.py: ~55% (no convolutions) - train_cnn.py: ~60%+ (WITH Conv2D - shows why convolutions matter!) This follows actual ML history and each module is needed for its capstone.
TinyTorch Examples
Complete Applications Built with Your Framework
These examples demonstrate that the ML framework you built from scratch actually works! Each example is a real application that uses the components you created.
📁 Example Structure
Each example folder contains clearly named files:
train_*.py- Training scripts that teach the modeltest_*.py- Testing scripts that evaluate performancedemo_*.py- Interactive demonstrationsutils.py- Helper functions specific to that exampleREADME.md- Detailed documentation for students
🎯 The Three Capstone Examples
1. xornet/ - Neural Network Fundamentals
Proves: Your neural networks can learn non-linear functions
Files:
train_xor_network.py- Trains a network to solve XORvisualize_decision_boundary.py- Shows what the network learnedREADME.md- Explains why XOR is important
What students learn: XOR can't be solved linearly, but neural networks with hidden layers can solve it perfectly.
2. cifar10/ - Computer Vision
Proves: Your framework can handle real-world image classification
Files:
train_image_classifier.py- Trains CNN on CIFAR-10 imagestest_random_baseline.py- Shows random guessing gets ~10%evaluate_model.py- Tests your trained modelvisualize_predictions.py- Shows what the model seesREADME.md- Explains computer vision concepts
What students learn: How convolutions extract features and how real ML systems train on actual data.
3. tinygpt/ - Language Models
Proves: Your framework can build transformers and generate text
Files:
train_language_model.py- Trains GPT on text datagenerate_text.py- Interactive text generationtest_simple_patterns.py- Verifies the model can learntokenizer.py- Text processing utilitiesREADME.md- Explains language modeling
What students learn: How attention mechanisms enable language understanding and generation.
🚀 Running the Examples
Each example can be run immediately:
# XOR - Takes seconds, shows 100% accuracy
cd examples/xornet
python train_xor_network.py
# CIFAR-10 - Takes minutes, achieves 55%+ accuracy
cd examples/cifar10
python train_image_classifier.py
# TinyGPT - Takes minutes, generates text
cd examples/tinygpt
python train_language_model.py
python generate_text.py
📊 What Success Looks Like
- XORNet: 100% accuracy on XOR problem
- CIFAR-10: 55%+ accuracy (5.5x better than random)
- TinyGPT: Generates coherent character sequences
💡 For Students
These examples are the proof that you succeeded. You didn't just learn about neural networks - you built a framework capable of:
- Learning any function (XORNet)
- Classifying real images (CIFAR-10)
- Generating language (TinyGPT)
This is what ML engineers do in production!