d203fba8b8
Updated all references to reflect new module order: - Module 05: DataLoader (was 08) - Module 06: Autograd (was 05) - Module 07: Optimizers (was 06) - Module 08: Training (was 07) Changes include: - paper/paper.tex: 20+ references, tier descriptions, milestones - src/: Export commands, dependency diagrams, docstrings - tests/: Dependency chains, integration tests, README - tito/: export_utils.py path mappings - tinytorch/: Auto-generated package file headers Foundation Tier is now Modules 01-08 Architecture Tier is now Modules 09-13
3.4 KiB
Learning Resources
TinyTorch teaches you to build ML systems. These resources help you understand the why behind what you're building.
Companion Textbook
Machine Learning Systems
mlsysbook.ai by Prof. Vijay Janapa Reddi (Harvard University)
TinyTorch began as hands-on labs for this textbook. While TinyTorch can be used standalone, the ML Systems book provides the theoretical depth and production context behind every module you build.
What it teaches: Systems engineering for production ML—memory hierarchies, performance optimization, deployment strategies, and the engineering decisions behind modern ML frameworks.
How it connects to TinyTorch:
- TinyTorch modules directly implement concepts from the book's chapters
- The book explains why PyTorch, TensorFlow, and JAX make certain design decisions
- Together, they provide both hands-on implementation and theoretical understanding
When to use it: Read in parallel with TinyTorch. When you implement Module 06 (Autograd), read the book's chapter on automatic differentiation to understand the systems engineering behind your code.
Related Academic Courses
-
CS 329S: Machine Learning Systems Design (Stanford) Production ML systems and deployment
-
TinyML and Efficient Deep Learning (MIT 6.5940) Edge computing, model compression, and efficient ML
-
CS 249r: Tiny Machine Learning (Harvard) TinyML systems and resource-constrained ML
-
CS 231n: Convolutional Neural Networks (Stanford) Computer vision - complements TinyTorch Modules 08-09
-
CS 224n: Natural Language Processing (Stanford) Transformers and NLP - complements TinyTorch Modules 10-13
Other Textbooks
-
Deep Learning by Goodfellow, Bengio, Courville Mathematical foundations behind what you implement in TinyTorch
-
Hands-On Machine Learning by Aurélien Géron Practical implementations using established frameworks
Minimal Frameworks
Alternative approaches to building ML from scratch:
-
micrograd by Andrej Karpathy Autograd in 100 lines. Perfect 2-hour intro before TinyTorch.
-
nanoGPT by Andrej Karpathy Minimalist GPT implementation. Complements TinyTorch Modules 12-13.
-
tinygrad by George Hotz Performance-focused educational framework with GPU acceleration.
Production Framework Internals
-
PyTorch Internals by Edward Yang How PyTorch actually works under the hood
-
PyTorch: Extending PyTorch Custom operators and autograd functions
Ready to start? See the Quick Start for a 15-minute hands-on introduction.