3acfc7c4af
Introduces a README with project overview, setup instructions, and course structure. Adds a VISION document outlining the project's goals, conventions, and architecture. Includes updates to the setup module's README to clarify module to package mapping.
4.3 KiB
Tiny🔥Torch: Build ML Systems from Scratch
A hands-on systems course where you implement every component of a modern ML system
Disclaimer: TinyTorch is an educational framework developed independently and is not affiliated with or endorsed by Meta or the PyTorch project.
Tiny🔥Torch is a hands-on companion to Machine Learning Systems, providing practical coding exercises that complement the book's theoretical foundations. Rather than just learning about ML systems, you'll build one from scratch—implementing everything from tensors and autograd to hardware-aware optimization and deployment systems.
🎯 What You'll Build
By completing this course, you will have implemented a complete ML system:
Core Framework → Training Pipeline → Production System
- ✅ Tensors with automatic differentiation
- ✅ Neural network layers (MLP, CNN, Transformer)
- ✅ Training loops with optimizers (SGD, Adam)
- ✅ Data loading and preprocessing pipelines
- ✅ Model compression (pruning, quantization)
- ✅ Performance profiling and optimization
- ✅ Production deployment and monitoring
🚀 Quick Start
Ready to build? Choose your path:
🏃♂️ I want to start building now
→ QUICKSTART.md - Get coding in 10 minutes
📚 I want to understand the full course structure
→ PROJECT_GUIDE.md - Complete learning roadmap
🔍 I want to see the course in action
→ modules/setup/ - Browse the first module
🎓 Learning Approach
Module-First Development: Each module is self-contained with its own notebook, tests, and learning objectives. You'll work in Jupyter notebooks using the nbdev workflow to build a real Python package.
The Cycle: Write Code → Export → Test → Next Module
# The rhythm you'll use for every module
jupyter lab tensor_dev.ipynb # Write & test interactively
python bin/tito.py sync # Export to Python package
python bin/tito.py test # Verify implementation
📚 Course Structure
| Phase | Modules | What You'll Build |
|---|---|---|
| Foundation | Setup, Tensor, Autograd | Core mathematical engine |
| Neural Networks | MLP, CNN | Learning algorithms |
| Training Systems | Data, Training, Config | End-to-end pipelines |
| Production | Profiling, Compression, MLOps | Real-world deployment |
Total Time: 40-80 hours over several weeks • Prerequisites: Python basics
🛠️ Key Commands
python bin/tito.py info # Check progress
python bin/tito.py sync # Export notebooks
python bin/tito.py test --module [name] # Test implementation
🌟 Why Tiny🔥Torch?
Systems Engineering Principles: Learn to design ML systems from first principles
Hardware-Software Co-design: Understand how algorithms map to computational resources
Performance-Aware Development: Build systems optimized for real-world constraints
End-to-End Systems: From mathematical foundations to production deployment
📖 Educational Approach
Companion to Machine Learning Systems: This course provides hands-on implementation exercises that bring the book's concepts to life through code.
Learning by Building: Following the educational philosophy of Karpathy's micrograd, we learn complex systems by implementing them from scratch.
Real-World Systems: Drawing from production PyTorch and JAX architectures to understand industry-proven design patterns.
🤝 Contributing
We welcome contributions! Whether you're a student who found a bug or an instructor wanting to add modules, see our Contributing Guide.
📄 License
Apache License 2.0 - see the LICENSE file for details.
Ready to start building? → QUICKSTART.md 🚀