0f6a19d280
- Fixed navigation by removing missing appendix references from _toc.yml - Moved complementary learning section up for better visibility (after astronaut hook) - Fixed duplicate rocket icons: 🎯 Capstone, 🛤️ Learning Path, ⚡ Ready to Start - Improved visual hierarchy with unique, meaningful icons for each section - Enhanced readability and scannability of landing page content
8.7 KiB
html_meta
| html_meta | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Tiny🔥Torch: Build your own Machine Learning framework from scratch.
Most ML education teaches you to use frameworks. TinyTorch teaches you to build them.
TinyTorch is a minimalist educational framework designed for learning by doing. Instead of relying on PyTorch or TensorFlow, you implement everything from scratch—tensors, autograd, optimizers, even MLOps tooling. This hands-on approach builds the deep systems intuition that sets ML engineers apart from ML users.
:class: tip
**A complete ML framework from scratch**: your own PyTorch style toolkit that can:
- ✅ Train neural networks on CIFAR-10 (real dataset!)
- ✅ Implement automatic differentiation (the "magic" behind PyTorch)
- ✅ Deploy production systems with 75% model compression
- ✅ Handle complete ML pipeline from data to monitoring
**Result:** You become the expert others ask about "how PyTorch actually works."
Everyone wants to be an astronaut. 🧑🚀 TinyTorch teaches you how to build the rocket ship. 🚀
:class: note
For comprehensive ML systems knowledge, we recommend [**Machine Learning Systems**](https://mlsysbook.ai) by [Prof. Vijay Janapa Reddi](https://profvjreddi.github.io/website/). While TinyTorch teaches you to **build** ML systems from scratch, that book provides the broader **systems context** and engineering principles for production AI.
💡 The Core Difference
Most ML courses focus on algorithms and theory. You learn what neural networks do and why they work, but you import everything:
Traditional ML Course: TinyTorch Approach:
├── import torch ├── class Tensor:
├── model = nn.Linear(10, 1) │ def __add__(self, other): ...
├── loss = nn.MSELoss() │ def backward(self): ...
└── optimizer.step() ├── class Linear:
│ def forward(self, x):
│ return x @ self.weight + self.bias
├── def mse_loss(pred, target):
│ return ((pred - target) ** 2).mean()
├── class SGD:
│ def step(self):
└── param.data -= lr * param.grad
Go from "How does this work?" 🤷 to "I implemented every line!" 💪
TinyTorch focuses on implementation and systems thinking. You learn how to build working systems with progressive scaffolding, production ready practices, and comprehensive course infrastructure that bridges the gap between learning and building.
🎓 Learning Philosophy: Build, Use, Reflect
Every component follows the same powerful learning cycle:
Example: Activation Functions
🔧 Build: Implement ReLU from scratch
def relu(x):
# YOU implement this function
return np.maximum(0, x) # Your solution
🚀 Use: Immediately use your own code
from tinytorch.core.activations import ReLU # YOUR implementation!
layer = ReLU()
output = layer.forward(input_tensor) # Your code working!
💡 Reflect: See it working in real networks
# Your ReLU is now part of a real neural network
model = Sequential([
Dense(784, 128),
ReLU(), # <-- Your implementation
Dense(128, 10)
])
This pattern repeats for every component: tensors, layers, optimizers, even MLOps systems. You build it, use it immediately, then reflect on how it fits into larger systems.
📚 Course Journey: 15 Modules
:class: note
**1. Setup** • **2. Tensors** • **3. Activations**
Understanding workflow, multi-dimensional arrays, and the mathematical functions that enable learning.
:class: note
**4. Layers** • **5. Networks** • **6. CNNs**
Dense layers, sequential architecture, and convolutional operations for computer vision.
:class: note
**7. DataLoader** • **8. Autograd** • **9. Optimizers** • **10. Training**
CIFAR-10 loading, automatic differentiation, SGD/Adam optimizers, and complete training orchestration.
:class: note
**11. Compression** • **12. Kernels** • **13. Benchmarking** • **14. MLOps**
Model optimization, high-performance operations, systematic evaluation, and production monitoring.
:class: note
**15. Capstone Project**
Choose your focus: performance engineering, algorithm extensions, systems optimization, framework analysis, or developer tools.
🔗 Complete System Integration
This isn't 14 separate exercises. Every component you build integrates into one fully functional ML framework:
:class: important
**Module 2: Your Tensor class** → **Module 3: Powers your activation functions** → **Module 4: Enables your layers** → **Module 5: Forms your networks** → **Module 8: Drives your autograd system** → **Module 9: Optimizes with your SGD/Adam** → **Module 10: Trains on real CIFAR-10 data**
**Result:** A complete, working ML framework that you built from scratch, capable of training real neural networks on real datasets.
🎯 Capstone: Optimize Your Framework
After completing the 14 core modules, you have a complete ML framework. Now make it better through systems engineering:
Choose Your Focus:
- ⚡ Performance Optimization: GPU kernels, vectorization, memory-efficient operations
- 🧠 Algorithm Extensions: Transformer layers, BatchNorm, Dropout, advanced optimizers
- 🔧 Systems Engineering: Multi-GPU training, distributed computing, memory profiling
- 📊 Benchmarking Deep Dive: Compare your framework to PyTorch, identify bottlenecks
- 🛠️ Developer Experience: Better debugging tools, visualization, error messages
The Challenge: Use only your TinyTorch implementation as the base. No copying from PyTorch. This proves you understand the engineering trade-offs and can optimize real ML systems.
🛤️ Choose Your Learning Path
:class: important
### **🔬 [Quick Exploration](usage-paths/quick-exploration.md)** *(5 minutes)*
*"I want to see what this is about"*
- Click and run code immediately in your browser (Binder)
- No installation or setup required
- Implement ReLU, tensors, neural networks interactively
- Perfect for getting a feel for the course
### **🏗️ [Serious Development](usage-paths/serious-development.md)** *(8+ weeks)*
*"I want to build this myself"*
- Fork the repo and work locally with full development environment
- Build complete ML framework from scratch with `tito` CLI
- 14 progressive assignments from setup to production MLOps
- Professional development workflow with automated testing
### **👨🏫 [Classroom Use](usage-paths/classroom-use.md)** *(Instructors)*
*"I want to teach this course"*
- Complete course infrastructure with NBGrader integration
- Automated grading for comprehensive testing
- Flexible pacing (8-16 weeks) with proven pedagogical structure
- Turn-key solution for ML systems education
⚡ Ready to Start?
Quick Taste: Try Module 1 Right Now
Want to see what TinyTorch feels like? Launch the Setup chapter in Binder and implement your first TinyTorch function in 2 minutes!
🙏 Acknowledgments
TinyTorch originated from CS249r: Tiny Machine Learning Systems at Harvard University. We're inspired by projects like tinygrad, micrograd, and MiniTorch that demonstrate the power of minimal implementations.