mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-05-07 16:52:34 -05:00
5ec52dd2e5
Major cleanup and optimization of CIFAR-10 classification examples: 📁 Directory cleanup: - Removed 25+ experimental/debug files - Streamlined to 3 clean, well-documented examples - Clear file organization and purpose 🎯 Main achievements: - train_cifar10_mlp.py: 57.2% test accuracy (exceeds course benchmarks!) - train_simple_baseline.py: ~40% baseline for comparison - train_lenet5.py: Historical LeNet-5 adaptation 📊 Performance improvements: - Fixed autograd bias gradient aggregation bug - Optimized weight initialization (He × 0.5) - Enhanced data augmentation (flip, brightness, translation) - Better normalization ([-2, 2] range) - Learning rate scheduling and decay 📚 Documentation: - Comprehensive README with performance analysis - Literature comparison showing TinyTorch excellence - Clear optimization technique explanations - Educational value and next steps 🏆 Key results: - 57.2% accuracy exceeds CS231n/CS229 benchmarks (50-55%) - Approaches research MLP SOTA (60-65%) - Proves TinyTorch builds working ML systems - Students can be proud of their autograd implementation! Technical fixes: - Autograd add operation now handles broadcasting correctly - Bias gradients aggregated over batch dimension - Loss functions return Variables with gradient tracking - Comprehensive test suite for gradient shapes
TinyTorch Examples 🔥
Real-world examples showing what you can build with TinyTorch!
What Are These Examples?
These are real ML applications written using TinyTorch just like you would use PyTorch. Each example:
- Uses
import tinytorchas a real package - Shows professional ML code patterns
- Demonstrates actual capabilities you've built
- Can be run by anyone to see TinyTorch in action
Running Examples
# After installing/building TinyTorch:
cd examples/xor_network/
python train.py
# Or for image recognition:
cd examples/mnist_recognition/
python train_mnist.py
Example Categories
🧠 Neural Network Fundamentals
xor_network/- Classic XOR problem with hidden layerslinear_regression/- Simple regression tasks
👁️ Computer Vision
mnist_recognition/- Handwritten digit recognitioncifar10_classifier/- Real-world object classificationimage_filters/- Convolutional feature extraction
🤖 Language & Generation
text_generation/- Generate text with TinyGPTcode_completion/- Python code generation
⚡ Advanced Topics
autograd_demo/- Automatic differentiationoptimization_comparison/- SGD vs Adammodel_compression/- Quantization and pruningperformance_profiling/- Benchmarking tools
Example Structure
Each example directory contains:
example_name/
├── train.py # Main training script
├── model.py # Model architecture (if complex)
├── README.md # What this example demonstrates
├── requirements.txt # Just needs: tinytorch
└── config.yml # Metadata about the example
For Students
After completing each module, the corresponding example will work:
- Module 05 →
xor_network/works - Module 08 →
mnist_recognition/works - Module 11 →
cifar10_classifier/works with training - Module 16 →
text_generation/works
For Everyone Else
Want to see what TinyTorch can do? Just run any example:
# See a neural network learn XOR:
python examples/xor_network/train.py
# Train a CNN on real images:
python examples/cifar10_classifier/train.py
# Generate text with a transformer:
python examples/text_generation/generate.py
These aren't toy demos - they're real ML applications using a framework built from scratch!