cs249r_book/tinytorch/site/intro.md

title, og:title

title	og:title
Don't import torch. Build it.	Don't import torch. Build it.

Don't import torch. Build it.

<p style="text-align: center; font-size: 2.5rem; margin: 1rem 0 0.5rem 0; font-weight: 700;">
Build Your Own ML Framework
</p>
<p style="text-align: center; margin: 0 0 1rem 0;">
  <span style="background: #fef3c7; border: 1px solid #f59e0b; border-radius: 1rem; padding: 0.25rem 0.75rem; font-size: 0.75rem; color: #92400e; font-weight: 600;">🚧 Preview · Classroom ready 2026</span>
</p>

Don't import it. Build it.

From tensors to systems. An educational framework for building and optimizing ML—understand how PyTorch, TensorFlow, and JAX really work.

Start Building →

⭐ ... learners · every ⭐ helps support free ML education

<style>
.approach-box {
  max-width: 720px;
  margin: 0 auto 2rem auto;
  background: linear-gradient(135deg, #1e293b 0%, #0f172a 100%);
  border-radius: 12px;
  padding: 2rem;
  border: 1px solid rgba(249, 115, 22, 0.3);
  box-shadow: 0 8px 24px rgba(0,0,0,0.2);
}
.approach-title {
  color: #f97316;
  font-size: 1.1rem;
  font-weight: 600;
  margin: 0 0 1rem 0;
  text-align: center;
}
.approach-grid {
  display: grid;
  grid-template-columns: repeat(2, 1fr);
  gap: 1rem;
}
.approach-item {
  display: flex;
  align-items: flex-start;
  gap: 0.75rem;
}
.approach-icon {
  font-size: 1.25rem;
  flex-shrink: 0;
}
.approach-text {
  color: #e2e8f0;
  font-size: 0.9rem;
  line-height: 1.5;
}
.approach-text strong {
  color: #fbbf24;
}
@media (max-width: 600px) {
  .approach-grid { grid-template-columns: 1fr; }
}
</style>

<div class="approach-box">
  <div class="approach-grid">
    <div class="approach-item">
      <span class="approach-icon">🔧</span>
      <span class="approach-text"><strong>Build each piece</strong> — Tensors, autograd, attention. No magic imports.</span>
    </div>
    <div class="approach-item">
      <span class="approach-icon">📚</span>
      <span class="approach-text"><strong>Recreate history</strong> — Perceptron → CNN → Transformers → MLPerf.</span>
    </div>
    <div class="approach-item">
      <span class="approach-icon">⚡</span>
      <span class="approach-text"><strong>Understand systems</strong> — Memory, compute, optimization trade-offs.</span>
    </div>
    <div class="approach-item">
      <span class="approach-icon">🎯</span>
      <span class="approach-text"><strong>Debug anything</strong> — OOM, NaN, slow training—because you built it.</span>
    </div>
  </div>
</div>

Recreate ML History

Walk through ML history by rebuilding its greatest breakthroughs with YOUR TinyTorch implementations. Click each milestone to see what you'll build and how it shaped modern AI.

<div class="ml-timeline-container">
    <div class="ml-timeline-line"></div>

    <div class="ml-timeline-item left perceptron">
        <div class="ml-timeline-dot"></div>
        <div class="ml-timeline-content">
            <div class="ml-timeline-year">1958</div>
            <div class="ml-timeline-title">The Perceptron</div>
            <div class="ml-timeline-desc">The first trainable neural network</div>
            <div class="ml-timeline-tech">Input → Linear → Sigmoid → Output</div>
        </div>
    </div>

    <div class="ml-timeline-item right xor">
        <div class="ml-timeline-dot"></div>
        <div class="ml-timeline-content">
            <div class="ml-timeline-year">1969</div>
            <div class="ml-timeline-title">XOR Crisis</div>
            <div class="ml-timeline-desc">Minsky & Papert expose limits of single-layer networks</div>
            <div class="ml-timeline-tech">Input → Linear → Sigmoid → FAIL!</div>
        </div>
    </div>

    <div class="ml-timeline-item left mlp">
        <div class="ml-timeline-dot"></div>
        <div class="ml-timeline-content">
            <div class="ml-timeline-year">1986</div>
            <div class="ml-timeline-title">MLP Revival</div>
            <div class="ml-timeline-desc">Backpropagation enables deep learning (95%+ MNIST)</div>
            <div class="ml-timeline-tech">Images → Flatten → Linear → ... → Classes</div>
        </div>
    </div>

    <div class="ml-timeline-item right cnn">
        <div class="ml-timeline-dot"></div>
        <div class="ml-timeline-content">
            <div class="ml-timeline-year">1998</div>
            <div class="ml-timeline-title">CNN Revolution 🎯</div>
            <div class="ml-timeline-desc">Spatial intelligence unlocks computer vision (75%+ CIFAR-10)</div>
            <div class="ml-timeline-tech">Images → Conv → Pool → ... → Classes</div>
        </div>
    </div>

    <div class="ml-timeline-item left transformer">
        <div class="ml-timeline-dot"></div>
        <div class="ml-timeline-content">
            <div class="ml-timeline-year">2017</div>
            <div class="ml-timeline-title">Transformer Era</div>
            <div class="ml-timeline-desc">Attention launches the LLM revolution</div>
            <div class="ml-timeline-tech">Tokens → Attention → FFN → Output</div>
        </div>
    </div>

    <div class="ml-timeline-item right olympics">
        <div class="ml-timeline-dot"></div>
        <div class="ml-timeline-content">
            <div class="ml-timeline-year">2018–Present</div>
            <div class="ml-timeline-title">MLPerf Benchmarks</div>
            <div class="ml-timeline-desc">Production optimization (8-16× smaller, 12-40× faster)</div>
            <div class="ml-timeline-tech">Profile → Compress → Accelerate</div>
        </div>
    </div>
</div>

Why Build Instead of Use?

"Building systems creates irreversible understanding."

Traditional ML Education

import torch
model = torch.nn.Linear(784, 10)
output = model(input)
# When this breaks, you're stuck

Problem: You can't debug what you don't understand.

TinyTorch: Build → Use → Reflect

# BUILD it yourself
class Linear:
    def forward(self, x):
        return x @ self.weight + self.bias

# USE it on real data
loss.backward()  # YOUR autograd

Advantage: You can debug it because you built it.

Learning Path

Four progressive tiers take you from foundations to production systems:

Foundation (01-08)

Tensors, autograd, layers, training loops

Architecture (09-13)

CNNs, attention, transformers, GPT

Optimization (14-19)

Profiling, quantization, acceleration

Torch Olympics (20)

Competition-ready capstone project

The Big Picture • Getting Started • Preface

Is This For You?

🎓 Students

Taking ML courses, want to understand what's behind import torch

👩‍🏫 Instructors

Teaching ML systems with ready-made hands-on labs

🚀 Self-learners

Career changers or hobbyists going deeper than tutorials

Prerequisites: Python + basic linear algebra. No ML experience required.

Join the Community

See learners building ML systems worldwide

Add yourself to the map • Share your progress • Connect with builders

Part of the MLSysBook project — every ⭐ helps support free ML education

🌍 Join the Community ⭐ Star on GitHub ... 💬 Discuss on GitHub 📬 Get Updates

<script> async function fetchGitHubStars() { const starElement = document.getElementById('star-count'); const starElementHero = document.getElementById('star-count-hero'); try { const response = await fetch('https://api.github.com/repos/harvard-edge/cs249r_book'); const data = await response.json(); const starCount = data.stargazers_count; const formattedCount = starCount.toLocaleString(); if (starElement) starElement.textContent = formattedCount; if (starElementHero) starElementHero.textContent = formattedCount; } catch (error) { console.error('Failed to fetch GitHub stars:', error); if (starElement) starElement.textContent = '10k+'; if (starElementHero) starElementHero.textContent = '10k+'; } } document.addEventListener('DOMContentLoaded', fetchGitHubStars); </script>

Next Steps: Quick Start (15 min) • The Big Picture • Community