Learning Resources

TinyTorch teaches you to build ML systems. These resources help you understand the why behind what you’re building.

---

Companion Textbook

Machine Learning Systems

mlsysbook.ai by Prof. Vijay Janapa Reddi (Harvard University)

TinyTorch began as hands-on labs for this textbook. While TinyTorch can be used standalone, the ML Systems book provides the theoretical depth and production context behind every module you build.

What it teaches: Systems engineering for production ML—memory hierarchies, performance optimization, deployment strategies, and the engineering decisions behind modern ML frameworks.

How it connects to TinyTorch:

• TinyTorch modules directly implement concepts from the book’s chapters

• The book explains why PyTorch, TensorFlow, and JAX make certain design decisions

• Together, they provide both hands-on implementation and theoretical understanding

When to use it: Read in parallel with TinyTorch. When you implement Module 05 (Autograd), read the book’s chapter on automatic differentiation to understand the systems engineering behind your code.

---

Related Academic Courses

• CS 329S: Machine Learning Systems Design (Stanford) Production ML systems and deployment

• TinyML and Efficient Deep Learning (MIT 6.5940) Edge computing, model compression, and efficient ML

• CS 249r: Tiny Machine Learning (Harvard) TinyML systems and resource-constrained ML

• CS 231n: Convolutional Neural Networks (Stanford) Computer vision - complements TinyTorch Modules 08-09

• CS 224n: Natural Language Processing (Stanford) Transformers and NLP - complements TinyTorch Modules 10-13

---

Other Textbooks

• Deep Learning by Goodfellow, Bengio, Courville Mathematical foundations behind what you implement in TinyTorch

• Hands-On Machine Learning by Aurélien Géron Practical implementations using established frameworks

---

Minimal Frameworks

Alternative approaches to building ML from scratch:

• micrograd by Andrej Karpathy Autograd in 100 lines. Perfect 2-hour intro before TinyTorch.

• nanoGPT by Andrej Karpathy Minimalist GPT implementation. Complements TinyTorch Modules 12-13.

• tinygrad by George Hotz Performance-focused educational framework with GPU acceleration.

---

Production Framework Internals

• PyTorch Internals by Edward Yang How PyTorch actually works under the hood

• PyTorch: Extending PyTorch Custom operators and autograd functions

---

Ready to start? See the Quick Start Guide for a 15-minute hands-on introduction.