---
pagetitle: "TinyTorch — Don't import torch. Build it."
---

<!--
  Hero hierarchy:
    h1   = "Don't import it. Build it."   (visual hero, quotable, matches pagetitle)
    h2/p = "Build your own ML framework."  (subtitle — describes what h1 means)
  The earlier version had the subtitle rendered bigger than the slogan, which
  inverted the semantic hierarchy: screen readers and search engines saw the
  subtitle as the page's most important content. Fixing that here.
-->
```{=html}
<h1 class="hero-gradient-text">Don't import it. Build it.</h1>

<p style="text-align: center; font-size: 1.25rem; margin: 0.25rem 0 0.5rem 0; font-weight: 500; color: #475569;">
Build your own ML framework — from tensors to systems.

</p>

<p style="text-align: center; margin: 0 0 1rem 0;">
<span class="preview-badge">Preview · Classroom ready Fall 2026</span>
</p>

<p style="text-align: center; font-size: 1.1rem; margin: 0 auto 1.5rem auto; max-width: 750px; color: #374151;">
An educational framework for building and optimizing ML — understand how PyTorch, TensorFlow, and JAX really work.

</p>

<p style="text-align: center; font-size: 0.95rem; margin: -0.75rem auto 1.5rem auto; max-width: 760px; color: #64748b;">
TinyTorch is usable today for self-paced learning and active course pilots. APIs, instructor packaging, and classroom workflows will continue to stabilize through the Fall 2026 classroom release.
</p>

<div style="text-align: center; margin: 0 0 1rem 0;">
<a href="getting-started.qmd" class="btn-tinytorch">Start Building →</a>
</div>
<div style="text-align: center; margin: 0 0 2.5rem 0;">
<a href="https://github.com/harvard-edge/cs249r_book?tab=readme-ov-file#support-this-work" target="_blank" class="star-link">
<span>⭐</span>

<span id="star-count-hero" style="font-weight: 600;">...</span>
<span>stars on GitHub — <strong>add yours</strong> and support free ML education</span>
</a>

</div>

<div class="approach-box">
<div class="approach-grid">
<div class="approach-item">
<span class="approach-icon">🔧</span>
<span class="approach-text"><strong>Build each piece</strong> — Tensors, autograd, attention. No magic imports.</span>
</div>
<div class="approach-item">
<span class="approach-icon">📚</span>
<span class="approach-text"><strong>Recreate history</strong> — Perceptron → CNN → Transformers → MLPerf.</span>
</div>
<div class="approach-item">
<span class="approach-icon">⚡</span>
<span class="approach-text"><strong>Understand systems</strong> — Memory, compute, optimization trade-offs.</span>
</div>
<div class="approach-item">
<span class="approach-icon">🎯</span>
<span class="approach-text"><strong>Debug anything</strong> — OOM, NaN, slow training—because you built it.</span>
</div>
</div>
</div>

```
## Recreate ML History

Walk through ML history by rebuilding its greatest breakthroughs with YOUR TinyTorch implementations. Click each milestone to see what you'll build and how it shaped modern AI.

```{=html}
<div class="ml-timeline-container">
<div class="ml-timeline-line"></div>
<div class="ml-timeline-item left perceptron">
<div class="ml-timeline-dot"></div>
<div class="ml-timeline-content">
<div class="ml-timeline-year">1958</div>
<div class="ml-timeline-title">The Perceptron</div>
<div class="ml-timeline-desc">The first trainable neural network</div>
<div class="ml-timeline-tech">Input → Linear → Sigmoid → Output</div>
</div>
</div>
<div class="ml-timeline-item right xor">
<div class="ml-timeline-dot"></div>
<div class="ml-timeline-content">
<div class="ml-timeline-year">1969</div>
<div class="ml-timeline-title">XOR Crisis</div>
<div class="ml-timeline-desc">Minsky & Papert expose limits of single-layer networks</div>
<div class="ml-timeline-tech">Input → Linear → Sigmoid → FAIL!</div>
</div>
</div>
<div class="ml-timeline-item left mlp">
<div class="ml-timeline-dot"></div>
<div class="ml-timeline-content">
<div class="ml-timeline-year">1986</div>
<div class="ml-timeline-title">MLP Revival</div>
<div class="ml-timeline-desc">Backpropagation enables deep learning (95%+ MNIST)</div>
<div class="ml-timeline-tech">Images → Flatten → Linear → ... → Classes</div>
</div>
</div>
<div class="ml-timeline-item right cnn">
<div class="ml-timeline-dot"></div>
<div class="ml-timeline-content">
<div class="ml-timeline-year">1998</div>
<div class="ml-timeline-title">CNN Revolution 🎯</div>
<div class="ml-timeline-desc">Spatial intelligence unlocks computer vision (75%+ CIFAR-10)</div>
<div class="ml-timeline-tech">Images → Conv → Pool → ... → Classes</div>
</div>
</div>
<div class="ml-timeline-item left transformer">
<div class="ml-timeline-dot"></div>
<div class="ml-timeline-content">
<div class="ml-timeline-year">2017</div>
<div class="ml-timeline-title">Transformer Era</div>
<div class="ml-timeline-desc">Attention launches the LLM revolution</div>
<div class="ml-timeline-tech">Tokens → Attention → FFN → Output</div>
</div>
</div>
<div class="ml-timeline-item right olympics">
<div class="ml-timeline-dot"></div>
<div class="ml-timeline-content">
<div class="ml-timeline-year">2018–Present</div>
<div class="ml-timeline-title">MLPerf Benchmarks</div>
<div class="ml-timeline-desc">Production optimization (8-16× smaller, 12-40× faster)</div>
<div class="ml-timeline-tech">Profile → Compress → Accelerate</div>
</div>
</div>
</div>

```
## Why Build Instead of Use?

```{=html}
<p style="text-align: center; font-size: 1.3rem; font-style: italic; color: #475569; margin: 0 0 1.5rem 0; max-width: 600px; margin-left: auto; margin-right: auto;">
"Building systems creates irreversible understanding."

</p>

```
:::: {.comparison-grid}

::: {.comparison-bad}
[Traditional ML Education]{.comparison-title}

```python
import torch
model = torch.nn.Linear(784, 10)
output = model(input)
# When this breaks, you're stuck
```

**Problem**: You can't debug what you don't understand.
:::

::: {.comparison-good}
[TinyTorch: Build → Use → Reflect]{.comparison-title}

```python
# BUILD it yourself
class Linear:
    def forward(self, x):
        return x @ self.weight + self.bias

# USE it on real data
loss.backward()  # YOUR autograd
```

**Advantage**: You can debug it because you built it.
:::

::::

## Learning Path

Four progressive tiers take you from foundations to production systems:

:::: {.tier-grid}

::: {.tier-card .tier-foundation}
[**Foundation (01-08)**<br>Tensors, autograd, layers, training loops](tiers/foundation.qmd)
:::

::: {.tier-card .tier-architecture}
[**Architecture (09-13)**<br>CNNs, attention, transformers, GPT](tiers/architecture.qmd)
:::

::: {.tier-card .tier-optimization}
[**Optimization (14-19)**<br>Profiling, quantization, acceleration](tiers/optimization.qmd)
:::

::: {.tier-card .tier-olympics}
[**Torch Olympics (20)**<br>Competition-ready capstone project](tiers/olympics.qmd)
:::

::::

[**The Big Picture**](big-picture.qmd) | [**Getting Started**](getting-started.qmd) | [**Welcome**](preface.qmd)

## Is This For You?

:::: {.audience-grid}

::: {.audience-card}
[🎓 **Students**]{.audience-title}

[Taking ML courses, want to understand what's behind `import torch`]{.audience-desc}
:::

::: {.audience-card}
[👩‍🏫 **Instructors**]{.audience-title}

[Teaching ML systems with ready-made hands-on labs]{.audience-desc}
:::

::: {.audience-card}
[🚀 **Self-learners**]{.audience-title}

[Career changers or hobbyists going deeper than tutorials]{.audience-desc}
:::

::::

**Prerequisites**: Python + basic linear algebra. No ML experience required.


## Join the Community

```{=html}
<div class="community-section">
<p style="color: #f1f5f9; font-size: 1.25rem; margin: 0 0 0.5rem 0; font-weight: 600;">
See learners building ML systems worldwide

</p>
<p style="color: #94a3b8; margin: 0 0 0.75rem 0;">
Add yourself to the map · Share your progress · Connect with builders

</p>
<p style="color: #fbbf24; margin: 0 0 1.5rem 0; font-size: 0.9rem;">
Part of the <a href="https://github.com/harvard-edge/cs249r_book?tab=readme-ov-file#support-this-work" target="_blank" style="color: #fbbf24; text-decoration: underline;">MLSysBook</a> project — every ⭐ helps support free ML education

</p>
<div style="display: flex; gap: 1rem; justify-content: center; flex-wrap: wrap; align-items: center;">
<a href="community.qmd" class="btn-tinytorch">🌍 Join the Community</a>
<a href="https://github.com/harvard-edge/cs249r_book" target="_blank" style="display: inline-flex; align-items: center; gap: 0.5rem; background: rgba(255,255,255,0.15); border: 1px solid rgba(255,255,255,0.3); color: #ffffff; padding: 0.75rem 1.5rem; border-radius: 0.5rem; text-decoration: none; font-weight: 600; font-size: 1rem;">⭐ Star on GitHub <span id="star-count" style="background: rgba(249,115,22,0.3); padding: 0.2rem 0.6rem; border-radius: 1rem; font-size: 0.85rem; color: #fbbf24;">...</span></a>
<a href="https://github.com/harvard-edge/cs249r_book/discussions/1076" target="_blank" style="display: inline-block; background: rgba(255,255,255,0.15); border: 1px solid rgba(255,255,255,0.3); color: #ffffff; padding: 0.75rem 2rem; border-radius: 0.5rem; text-decoration: none; font-weight: 600; font-size: 1rem;">💬 Discuss on GitHub</a>
<a href="#" onclick="event.preventDefault(); if(window.openSubscribeModal) openSubscribeModal();" style="display: inline-block; background: rgba(255,255,255,0.15); border: 1px solid rgba(255,255,255,0.3); color: #ffffff; padding: 0.75rem 2rem; border-radius: 0.5rem; text-decoration: none; font-weight: 600; font-size: 1rem;">📬 Get Updates</a>
</div>
</div>

<script>
async function fetchGitHubStars() {
  const starElement = document.getElementById('star-count');
  const starElementHero = document.getElementById('star-count-hero');
  try {
    const response = await fetch('https://api.github.com/repos/harvard-edge/cs249r_book');
    const data = await response.json();
    const formattedCount = data.stargazers_count.toLocaleString();
    if (starElement) starElement.textContent = formattedCount;
    if (starElementHero) starElementHero.textContent = formattedCount;
  } catch (error) {
    if (starElement) starElement.textContent = '10k+';
    if (starElementHero) starElementHero.textContent = '10k+';
  }
}
document.addEventListener('DOMContentLoaded', fetchGitHubStars);

</script>
```

**Next Steps**: [**Quick Start**](getting-started.qmd) (15 min) | [**The Big Picture**](big-picture.qmd) | [**Community**](community.qmd)
