b47c8ef259
��️ Major architectural improvement implementing clean separation of concerns: ✨ NEW: Activations Module - Complete activations module with ReLU, Sigmoid, Tanh implementations - Educational NBDev structure with student TODOs + instructor solutions - Comprehensive testing suite (24 tests) with mathematical correctness validation - Visual learning features with matplotlib plotting (disabled during testing) - Clean export to tinytorch.core.activations 🔧 REFACTOR: Layers Module - Removed duplicate activation function implementations - Clean import from activations module: 'from tinytorch.core.activations import ReLU, Sigmoid, Tanh' - Updated documentation to reflect modular architecture - Preserved all existing functionality while improving code organization 🧪 TESTING: Comprehensive Test Coverage - All 24 activations tests passing ✅ - All 17 layers tests passing ✅ - Integration tests verify clean architecture works end-to-end - CLI testing with 'tito test --module' works for both modules 📦 ARCHITECTURE: Clean Dependency Graph - activations (math functions) → layers (building blocks) → networks (applications) - Separation of concerns: pure math vs. neural network components - Reusable components across future modules - Single source of truth for activation implementations �� PEDAGOGY: Enhanced Learning Experience - Week-sized chunks: students master activations, then build layers - Clear progression from mathematical foundations to applications - Real-world software architecture patterns - Modular design principles in practice This establishes the foundation for scalable, maintainable ML systems education.
Tiny🔥Torch: Build ML Systems from Scratch
A hands-on systems course where you implement every component of a modern ML system
Disclaimer: TinyTorch is an educational framework developed independently and is not affiliated with or endorsed by Meta or the PyTorch project.
Tiny🔥Torch is a hands-on companion to Machine Learning Systems, providing practical coding exercises that complement the book's theoretical foundations. Rather than just learning about ML systems, you'll build one from scratch—implementing everything from tensors and autograd to hardware-aware optimization and deployment systems.
🎯 What You'll Build
By completing this course, you will have implemented a complete ML system:
Core Framework → Training Pipeline → Production System
- ✅ Tensors with automatic differentiation
- ✅ Neural network layers (MLP, CNN, Transformer)
- ✅ Training loops with optimizers (SGD, Adam)
- ✅ Data loading and preprocessing pipelines
- ✅ Model compression (pruning, quantization)
- ✅ Performance profiling and optimization
- ✅ Production deployment and monitoring
🚀 Quick Start
Ready to build? Choose your path:
🏃♂️ I want to start building now
→ QUICKSTART.md - Get coding in 10 minutes
📚 I want to understand the full course structure
→ PROJECT_GUIDE.md - Complete learning roadmap
🔍 I want to see the course in action
→ modules/setup/ - Browse the first module
🎓 Learning Approach
Module-First Development: Each module is self-contained with its own notebook, tests, and learning objectives. You'll work in Jupyter notebooks using the nbdev workflow to build a real Python package.
The Cycle: Write Code → Export → Test → Next Module
# The rhythm you'll use for every module
jupyter lab tensor_dev.ipynb # Write & test interactively
python bin/tito.py sync # Export to Python package
python bin/tito.py test # Verify implementation
📚 Course Structure
| Phase | Modules | What You'll Build |
|---|---|---|
| Foundation | Setup, Tensor, Autograd | Core mathematical engine |
| Neural Networks | MLP, CNN | Learning algorithms |
| Training Systems | Data, Training, Config | End-to-end pipelines |
| Production | Profiling, Compression, MLOps | Real-world deployment |
Total Time: 40-80 hours over several weeks • Prerequisites: Python basics
🛠️ Key Commands
python bin/tito.py info # Check progress
python bin/tito.py sync # Export notebooks
python bin/tito.py test --module [name] # Test implementation
🌟 Why Tiny🔥Torch?
Systems Engineering Principles: Learn to design ML systems from first principles
Hardware-Software Co-design: Understand how algorithms map to computational resources
Performance-Aware Development: Build systems optimized for real-world constraints
End-to-End Systems: From mathematical foundations to production deployment
📖 Educational Approach
Companion to Machine Learning Systems: This course provides hands-on implementation exercises that bring the book's concepts to life through code.
Learning by Building: Following the educational philosophy of Karpathy's micrograd, we learn complex systems by implementing them from scratch.
Real-World Systems: Drawing from production PyTorch and JAX architectures to understand industry-proven design patterns.
🤝 Contributing
We welcome contributions! Whether you're a student who found a bug or an instructor wanting to add modules, see our Contributing Guide.
📄 License
Apache License 2.0 - see the LICENSE file for details.
Ready to start building? → QUICKSTART.md 🚀