377967185c
- Live publish now merges dev into main before building - Removed auto-trigger on push to main (manual only) - All build jobs depend on sync completing first - Ensures public site always reflects tested dev state - Updated README with improved badges and formatting
7.3 KiB
🔥 TinyTorch
Build Your Own ML Framework From Scratch
Most ML courses teach you to use frameworks. TinyTorch teaches you to build them.
🧪 Early Access - TinyTorch is available for early adopters! All 20 modules are implemented with complete solutions and the
titoCLI for streamlined workflows.🎯 Spring 2025: Full public release with community features and classroom deployment resources.
Why TinyTorch?
"Most ML education teaches you to use frameworks. TinyTorch teaches you to build them."
The difference between ML users and ML engineers determines who drives innovation versus who merely consumes it. TinyTorch bridges this gap by teaching you to build every component of modern AI systems from scratch.
A Harvard University course that transforms you from framework user to systems engineer, giving you the deep understanding needed to optimize, debug, and innovate at the foundation of AI.
What You'll Build
A complete ML framework capable of:
🎯 North Star Achievement: Train CNNs for image classification
- Real computer vision on standard benchmark datasets
- Built entirely from scratch using only NumPy
- Competitive performance with modern frameworks
Additional Capabilities:
- GPT-style language models with attention mechanisms
- Modern optimizers (Adam, SGD) with learning rate scheduling
- Performance profiling, optimization, and competitive benchmarking
No dependencies on PyTorch or TensorFlow - everything is YOUR code!
Quick Start
# Clone the repository
git clone https://github.com/harvard-edge/cs249r_book.git
cd cs249r_book/tinytorch
# Install in editable mode
pip install -e .
# Verify installation
tito doctor
# Start building!
tito module start 01
Note
: TinyTorch is part of the ML Systems Book project. Installing from the book repository keeps everything together and lets you easily update with
git pull.
20 Progressive Modules
Build your framework through four progressive parts:
| Part | Modules | What You Build |
|---|---|---|
| I. Foundations | 01-07 | Tensors, activations, layers, losses, autograd, optimizers, training |
| II. Vision | 08-09 | DataLoaders, Conv2d, CNNs for image classification |
| III. Language | 10-13 | Tokenization, embeddings, attention, transformers |
| IV. Optimization | 14-20 | Profiling, quantization, compression, acceleration, benchmarking, capstone |
Each module asks: "Can I build this capability from scratch?"
📖 Full curriculum and module details →
Historical Milestones
As you progress, unlock recreations of landmark ML achievements:
| Year | Milestone | Your Achievement |
|---|---|---|
| 1957 | Perceptron | Binary classification with gradient descent |
| 1969 | XOR Crisis | Multi-layer networks solve non-linear problems |
| 1986 | Backpropagation | Multi-layer network training |
| 1998 | CNN Revolution | Image classification with convolutions |
| 2017 | Transformer Era | Language generation with self-attention |
| 2018+ | MLPerf | Production-ready optimization |
These aren't toy demos - they're historically significant ML achievements rebuilt with YOUR framework!
Learning Philosophy
# Traditional Course:
import torch
model.fit(X, y) # Magic happens
# TinyTorch:
# You implement every component
# You measure memory usage
# You optimize performance
# You understand the systems
Why Build Your Own Framework?
- Deep Understanding - Know exactly what
loss.backward()does - Systems Thinking - Understand memory, compute, and scaling
- Debugging Skills - Fix problems at any level of the stack
- Production Ready - Learn patterns used in real ML systems
Documentation
| Audience | Resources |
|---|---|
| Students | Course Website ・ Quick Start ・ FAQ |
| Instructors | Instructor Guide ・ NBGrader Setup ・ TA Guide |
| Contributors | Contributing Guide ・ Module Development |
Repository Structure
tinytorch/
├── src/ # Source files (contributors edit here)
├── modules/ # Generated notebooks (learners work here)
├── tinytorch/ # Generated package (import from here)
├── milestones/ # Historical ML achievements
├── tests/ # Comprehensive test suite
├── site/ # Documentation website
└── tito/ # CLI tool
Key workflow: src/*.py → modules/*.ipynb → tinytorch/*.py
Join the Community
TinyTorch is part of the ML Systems Book ecosystem. We're building an open community of learners and educators passionate about ML systems.
Ways to get involved:
- ⭐ Star this repo to show support
- 💬 Join Discussions to ask questions
- 🐛 Report issues or suggest improvements
- 🤝 Contribute modules, fixes, or documentation
See CONTRIBUTING.md for guidelines.
Related Projects
"TinyTorch" is a popular name for educational ML frameworks. We acknowledge excellent projects with similar names:
- tinygrad - George Hotz's minimalist framework
- micrograd - Andrej Karpathy's tiny autograd
- MiniTorch - Cornell's educational framework
Our TinyTorch distinguishes itself through its 20-module curriculum, NBGrader integration, ML systems focus, and connection to the ML Systems Book ecosystem.
Acknowledgments
Created by Prof. Vijay Janapa Reddi at Harvard University.
Special thanks to students and contributors who helped build this framework.
License
MIT License - see LICENSE for details.
📖 Full Documentation ・ 💬 Discussions ・ 🌐 ML Systems Book
Start Small. Go Deep. Build ML Systems.