Collabs

Understanding the Interplay Between Algorithms and Systems

Status: Coming Summer 2026

What Are Collabs?

Collabs are hands-on Google Colab simulations that bridge the gap between reading about ML systems (the textbook) and building them from scratch (TinyTorch).

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│                 │     │                 │     │                 │
│    Textbook     │────▶│     Collabs     │────▶│    TinyTorch    │
│                 │     │                 │     │                 │
│  Concepts &     │     │  Experiment &   │     │  Build from     │
│  Theory         │     │  Explore        │     │  Scratch        │
│                 │     │                 │     │                 │
└─────────────────┘     └─────────────────┘     └─────────────────┘
      READ                    EXPLORE                  BUILD

The Learning Journey

Phase	Resource	What You Do
Understand	Textbook	Learn concepts, theory, and system design principles
Experiment	Collabs	Explore tradeoffs, tweak parameters, see how decisions ripple through systems
Build	TinyTorch	Implement everything from scratch, own every line of code

Why Collabs?

ML systems are where algorithms meet hardware. A model that works perfectly in theory can fail in practice due to memory limits, latency constraints, or numerical precision. Collabs help you develop intuition for these algorithm-system interactions.

See the tradeoffs — How does batch size affect memory? How does quantization affect accuracy?
Explore interactively — Adjust parameters and watch how changes ripple through the system
Build intuition — Understand why systems behave the way they do, not just what they do
Zero setup — Run directly in your browser via Google Colab

Example Topics (Planned)

Memory vs. Compute Tradeoffs — Watch how batch size affects memory footprint and training speed
Quantization Effects — See accuracy degradation as you reduce precision from FP32 → INT8 → INT4
Attention Visualization — Explore what transformer attention heads actually learn
Optimization Landscapes — Navigate loss surfaces with different optimizers
Pruning Strategies — Compare structured vs. unstructured pruning on real models

Stay Updated

Collabs are under active development. To be notified when they launch:

Resource	Description
Textbook	ML Systems principles and practices
TinyTorch	Build your own ML framework from scratch
Discussions	Ask questions, share feedback

Read. Explore. Build. (Collabs coming soon)