github-starred/TinyTorch
2
0
Fork 0
mirror of https://github.com/MLSysBook/TinyTorch.git synced 2026-06-05 23:07:53 -05:00
Code Issues 7 Packages Projects Releases Wiki Activity
Files
c8dc692a094e555bd293572279542e26daf810fa
TinyTorch/book/_build/html/intro.html
Vijay Janapa Reddi c8dc692a09 FEAT: Add interactive learning timeline and clean up website presentation 
- Create comprehensive learning timeline page showing 60+ years of ML evolution
- Visual progress timeline from Perceptron (1957) to TinyMLPerf (2025)
- Module progression map with historical context and achievements
- Capability checkpoints tracking system integration
- Clean up emoji usage in TOC for professional presentation
- Add timeline as first item in Getting Started section
- Show students exactly what they'll build at each milestone
- Connect each module to real historical breakthroughs
- Emphasize progression from foundation to production systems
2025-09-26 14:57:44 -04:00

946 lines
51 KiB
HTML
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<!DOCTYPE html>
<html lang="en" data-content_root="./" >
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<meta content="TinyTorch: Build your own ML framework from scratch" property="og:title" />
<meta content="Learn ML systems by building them. From computer vision to language models. Comprehensive educational framework for understanding ML systems engineering." property="og:description" />
<meta content="https://mlsysbook.github.io/TinyTorch/" property="og:url" />
<meta content="website" property="og:type" />
<meta content="https://mlsysbook.github.io/TinyTorch/logo.png" property="og:image" />
<meta content="TinyTorch Course" property="og:site_name" />
<meta content="summary_large_image" name="twitter:card" />
<meta content="TinyTorch: Build your own ML framework" name="twitter:title" />
<meta content="TinyTorch is a minimalist framework for building machine learning systems from scratch—from vision to language." name="twitter:description" />
<meta content="https://mlsysbook.github.io/TinyTorch/logo.png" name="twitter:image" />
<title>TinyTorch: Build Your Own ML Framework from First Principles &#8212; TinyTorch</title>
<script data-cfasync="false">
document.documentElement.dataset.mode = localStorage.getItem("mode") || "";
document.documentElement.dataset.theme = localStorage.getItem("theme") || "";
</script>
<!-- Loaded before other Sphinx assets -->
<link href="_static/styles/theme.css?digest=dfe6caa3a7d634c4db9b" rel="stylesheet" />
<link href="_static/styles/bootstrap.css?digest=dfe6caa3a7d634c4db9b" rel="stylesheet" />
<link href="_static/styles/pydata-sphinx-theme.css?digest=dfe6caa3a7d634c4db9b" rel="stylesheet" />
<link href="_static/vendor/fontawesome/6.5.2/css/all.min.css?digest=dfe6caa3a7d634c4db9b" rel="stylesheet" />
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-solid-900.woff2" />
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-brands-400.woff2" />
<link rel="preload" as="font" type="font/woff2" crossorigin href="_static/vendor/fontawesome/6.5.2/webfonts/fa-regular-400.woff2" />
<link rel="stylesheet" type="text/css" href="_static/pygments.css?v=03e43079" />
<link rel="stylesheet" type="text/css" href="_static/styles/sphinx-book-theme.css?v=eba8b062" />
<link rel="stylesheet" type="text/css" href="_static/togglebutton.css?v=13237357" />
<link rel="stylesheet" type="text/css" href="_static/copybutton.css?v=76b2166b" />
<link rel="stylesheet" type="text/css" href="_static/mystnb.8ecb98da25f57f5357bf6f572d296f466b2cfe2517ffebfabe82451661e28f02.css?v=6644e6bb" />
<link rel="stylesheet" type="text/css" href="_static/sphinx-thebe.css?v=4fa983c6" />
<link rel="stylesheet" type="text/css" href="_static/sphinx-design.min.css?v=95c83b7e" />
<link rel="stylesheet" type="text/css" href="_static/custom.css?v=5a689c52" />
<!-- Pre-loaded scripts that we'll load fully later -->
<link rel="preload" as="script" href="_static/scripts/bootstrap.js?digest=dfe6caa3a7d634c4db9b" />
<link rel="preload" as="script" href="_static/scripts/pydata-sphinx-theme.js?digest=dfe6caa3a7d634c4db9b" />
<script src="_static/vendor/fontawesome/6.5.2/js/all.min.js?digest=dfe6caa3a7d634c4db9b"></script>
<script src="_static/documentation_options.js?v=9eb32ce0"></script>
<script src="_static/doctools.js?v=9a2dae69"></script>
<script src="_static/sphinx_highlight.js?v=dc90522c"></script>
<script src="_static/clipboard.min.js?v=a7894cd8"></script>
<script src="_static/copybutton.js?v=f281be69"></script>
<script src="_static/scripts/sphinx-book-theme.js?v=887ef09a"></script>
<script>let toggleHintShow = 'Click to show';</script>
<script>let toggleHintHide = 'Click to hide';</script>
<script>let toggleOpenOnPrint = 'true';</script>
<script src="_static/togglebutton.js?v=4a39c7ea"></script>
<script>var togglebuttonSelector = '.toggle, .admonition.dropdown';</script>
<script src="_static/design-tabs.js?v=f930bc37"></script>
<script>const THEBE_JS_URL = "https://unpkg.com/thebe@0.8.2/lib/index.js"; const thebe_selector = ".thebe,.cell"; const thebe_selector_input = "pre"; const thebe_selector_output = ".output, .cell_output"</script>
<script async="async" src="_static/sphinx-thebe.js?v=c100c467"></script>
<script>var togglebuttonSelector = '.toggle, .admonition.dropdown';</script>
<script>const THEBE_JS_URL = "https://unpkg.com/thebe@0.8.2/lib/index.js"; const thebe_selector = ".thebe,.cell"; const thebe_selector_input = "pre"; const thebe_selector_output = ".output, .cell_output"</script>
<script type="module" src="https://cdn.jsdelivr.net/npm/mermaid@10.6.1/dist/mermaid.esm.min.mjs"></script>
<script type="module" src="https://cdn.jsdelivr.net/npm/@mermaid-js/layout-elk@0.1.4/dist/mermaid-layout-elk.esm.min.mjs"></script>
<script type="module">import mermaid from "https://cdn.jsdelivr.net/npm/mermaid@10.6.1/dist/mermaid.esm.min.mjs";import elkLayouts from "https://cdn.jsdelivr.net/npm/@mermaid-js/layout-elk@0.1.4/dist/mermaid-layout-elk.esm.min.mjs";mermaid.registerLayoutLoaders(elkLayouts);mermaid.initialize({startOnLoad:false});</script>
<script src="https://cdn.jsdelivr.net/npm/d3@7.9.0/dist/d3.min.js"></script>
<script type="module">
import mermaid from "https://cdn.jsdelivr.net/npm/mermaid@10.6.1/dist/mermaid.esm.min.mjs";
window.addEventListener("load", () => mermaid.run());
</script>
<script>DOCUMENTATION_OPTIONS.pagename = 'intro';</script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Your TinyTorch Journey: From First Neural Network to Modern AI" href="learning-timeline.html" />
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<meta name="docsearch:language" content="en"/>
</head>
<body data-bs-spy="scroll" data-bs-target=".bd-toc-nav" data-offset="180" data-bs-root-margin="0px 0px -60%" data-default-mode="">
<div id="pst-skip-link" class="skip-link d-print-none"><a href="#main-content">Skip to main content</a></div>
<div id="pst-scroll-pixel-helper"></div>
<button type="button" class="btn rounded-pill" id="pst-back-to-top">
<i class="fa-solid fa-arrow-up"></i>Back to top</button>
<input type="checkbox"
class="sidebar-toggle"
id="pst-primary-sidebar-checkbox"/>
<label class="overlay overlay-primary" for="pst-primary-sidebar-checkbox"></label>
<input type="checkbox"
class="sidebar-toggle"
id="pst-secondary-sidebar-checkbox"/>
<label class="overlay overlay-secondary" for="pst-secondary-sidebar-checkbox"></label>
<div class="search-button__wrapper">
<div class="search-button__overlay"></div>
<div class="search-button__search-container">
<form class="bd-search d-flex align-items-center"
action="search.html"
method="get">
<i class="fa-solid fa-magnifying-glass"></i>
<input type="search"
class="form-control"
name="q"
id="search-input"
placeholder="Search this book..."
aria-label="Search this book..."
autocomplete="off"
autocorrect="off"
autocapitalize="off"
spellcheck="false"/>
<span class="search-button__kbd-shortcut"><kbd class="kbd-shortcut__modifier">Ctrl</kbd>+<kbd>K</kbd></span>
</form></div>
</div>
<div class="pst-async-banner-revealer d-none">
<aside id="bd-header-version-warning" class="d-none d-print-none" aria-label="Version warning"></aside>
</div>
<header class="bd-header navbar navbar-expand-lg bd-navbar d-print-none">
</header>
<div class="bd-container">
<div class="bd-container__inner bd-page-width">
<div class="bd-sidebar-primary bd-sidebar">
<div class="sidebar-header-items sidebar-primary__section">
</div>
<div class="sidebar-primary-items__start sidebar-primary__section">
<div class="sidebar-primary-item">
<a class="navbar-brand logo" href="#">
<img src="_static/logo.png" class="logo__image only-light" alt="TinyTorch - Home"/>
<script>document.write(`<img src="_static/logo.png" class="logo__image only-dark" alt="TinyTorch - Home"/>`);</script>
</a></div>
<div class="sidebar-primary-item">
<script>
document.write(`
<button class="btn search-button-field search-button__button" title="Search" aria-label="Search" data-bs-placement="bottom" data-bs-toggle="tooltip">
<i class="fa-solid fa-magnifying-glass"></i>
<span class="search-button__default-text">Search</span>
<span class="search-button__kbd-shortcut"><kbd class="kbd-shortcut__modifier">Ctrl</kbd>+<kbd class="kbd-shortcut__modifier">K</kbd></span>
</button>
`);
</script></div>
<div class="sidebar-primary-item"><nav class="bd-links bd-docs-nav" aria-label="Main">
<div class="bd-toc-item navbar-nav active">
<ul class="nav bd-sidenav bd-sidenav__home-link">
<li class="toctree-l1 current active">
<a class="reference internal" href="#">
TinyTorch: Build Your Own ML Framework from First Principles
</a>
</li>
</ul>
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Getting Started</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="learning-timeline.html">Your Learning Journey</a></li>
<li class="toctree-l1"><a class="reference internal" href="usage-paths/quick-exploration.html">Quick Exploration</a></li>
<li class="toctree-l1"><a class="reference internal" href="usage-paths/serious-development.html">Serious Development</a></li>
<li class="toctree-l1"><a class="reference internal" href="usage-paths/classroom-use.html">Classroom Use</a></li>
<li class="toctree-l1"><a class="reference internal" href="instructor-guide.html">Instructor Guide</a></li>
</ul>
<p aria-level="2" class="caption" role="heading"><span class="caption-text">MLP Era Foundation (1980s)</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="chapters/01-setup.html">1. Setup</a></li>
<li class="toctree-l1"><a class="reference internal" href="chapters/02-tensor.html">2. Tensors</a></li>
<li class="toctree-l1"><a class="reference internal" href="chapters/03-activations.html">3. Activations</a></li>
<li class="toctree-l1"><a class="reference internal" href="chapters/04-layers.html">4. Layers</a></li>
</ul>
<p aria-level="2" class="caption" role="heading"><span class="caption-text">MLP Intelligence (1980s)</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="chapters/05-dense.html">5. Dense Networks</a></li>
<li class="toctree-l1"><a class="reference internal" href="chapters/11-training.html">6. Training</a></li>
</ul>
<p aria-level="2" class="caption" role="heading"><span class="caption-text">CNN Revolution (1989-1998)</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="chapters/06-spatial.html">7. Spatial Convolutions</a></li>
<li class="toctree-l1"><a class="reference internal" href="chapters/08-dataloader.html">8. DataLoader</a></li>
</ul>
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Modern Training Systems</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="chapters/09-autograd.html">9. Autograd</a></li>
<li class="toctree-l1"><a class="reference internal" href="chapters/10-optimizers.html">10. Optimizers</a></li>
<li class="toctree-l1"><a class="reference internal" href="chapters/07-attention.html">11. Attention Mechanisms</a></li>
</ul>
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Production Systems</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="chapters/12-compression.html">12. Compression</a></li>
<li class="toctree-l1"><a class="reference internal" href="chapters/13-kernels.html">13. Kernels</a></li>
<li class="toctree-l1"><a class="reference internal" href="chapters/14-benchmarking.html">14. Benchmarking</a></li>
<li class="toctree-l1"><a class="reference internal" href="chapters/15-mlops.html">15. MLOps</a></li>
</ul>
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Universal Intelligence (2017+)</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="chapters/16-tinygpt.html">16. TinyGPT</a></li>
</ul>
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Additional Resources</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="vision.html">TinyTorch Vision</a></li>
<li class="toctree-l1"><a class="reference internal" href="kiss-principle.html">KISS Principle</a></li>
<li class="toctree-l1"><a class="reference internal" href="checkpoint-system.html">Checkpoint System</a></li>
<li class="toctree-l1"><a class="reference internal" href="testing-framework.html">Testing Framework</a></li>
<li class="toctree-l1"><a class="reference internal" href="resources.html">Learning Resources</a></li>
</ul>
</div>
</nav></div>
</div>
<div class="sidebar-primary-items__end sidebar-primary__section">
</div>
<div id="rtd-footer-container"></div>
</div>
<main id="main-content" class="bd-main" role="main">
<div class="sbt-scroll-pixel-helper"></div>
<div class="bd-content">
<div class="bd-article-container">
<div class="bd-header-article d-print-none">
<div class="header-article-items header-article__inner">
<div class="header-article-items__start">
<div class="header-article-item"><button class="sidebar-toggle primary-toggle btn btn-sm" title="Toggle primary sidebar" data-bs-placement="bottom" data-bs-toggle="tooltip">
<span class="fa-solid fa-bars"></span>
</button></div>
</div>
<div class="header-article-items__end">
<div class="header-article-item">
<div class="article-header-buttons">
<div class="dropdown dropdown-source-buttons">
<button class="btn dropdown-toggle" type="button" data-bs-toggle="dropdown" aria-expanded="false" aria-label="Source repositories">
<i class="fab fa-github"></i>
</button>
<ul class="dropdown-menu">
<li><a href="https://github.com/mlsysbook/TinyTorch" target="_blank"
class="btn btn-sm btn-source-repository-button dropdown-item"
title="Source repository"
data-bs-placement="left" data-bs-toggle="tooltip"
>
<span class="btn__icon-container">
<i class="fab fa-github"></i>
</span>
<span class="btn__text-container">Repository</span>
</a>
</li>
<li><a href="https://github.com/mlsysbook/TinyTorch/edit/main/book/intro.md" target="_blank"
class="btn btn-sm btn-source-edit-button dropdown-item"
title="Suggest edit"
data-bs-placement="left" data-bs-toggle="tooltip"
>
<span class="btn__icon-container">
<i class="fas fa-pencil-alt"></i>
</span>
<span class="btn__text-container">Suggest edit</span>
</a>
</li>
<li><a href="https://github.com/mlsysbook/TinyTorch/issues/new?title=Issue%20on%20page%20%2Fintro.html&body=Your%20issue%20content%20here." target="_blank"
class="btn btn-sm btn-source-issues-button dropdown-item"
title="Open an issue"
data-bs-placement="left" data-bs-toggle="tooltip"
>
<span class="btn__icon-container">
<i class="fas fa-lightbulb"></i>
</span>
<span class="btn__text-container">Open issue</span>
</a>
</li>
</ul>
</div>
<div class="dropdown dropdown-download-buttons">
<button class="btn dropdown-toggle" type="button" data-bs-toggle="dropdown" aria-expanded="false" aria-label="Download this page">
<i class="fas fa-download"></i>
</button>
<ul class="dropdown-menu">
<li><a href="_sources/intro.md" target="_blank"
class="btn btn-sm btn-download-source-button dropdown-item"
title="Download source file"
data-bs-placement="left" data-bs-toggle="tooltip"
>
<span class="btn__icon-container">
<i class="fas fa-file"></i>
</span>
<span class="btn__text-container">.md</span>
</a>
</li>
<li>
<button onclick="window.print()"
class="btn btn-sm btn-download-pdf-button dropdown-item"
title="Print to PDF"
data-bs-placement="left" data-bs-toggle="tooltip"
>
<span class="btn__icon-container">
<i class="fas fa-file-pdf"></i>
</span>
<span class="btn__text-container">.pdf</span>
</button>
</li>
</ul>
</div>
<button onclick="toggleFullScreen()"
class="btn btn-sm btn-fullscreen-button"
title="Fullscreen mode"
data-bs-placement="bottom" data-bs-toggle="tooltip"
>
<span class="btn__icon-container">
<i class="fas fa-expand"></i>
</span>
</button>
<script>
document.write(`
<button class="btn btn-sm nav-link pst-navbar-icon theme-switch-button" title="light/dark" aria-label="light/dark" data-bs-placement="bottom" data-bs-toggle="tooltip">
<i class="theme-switch fa-solid fa-sun fa-lg" data-mode="light"></i>
<i class="theme-switch fa-solid fa-moon fa-lg" data-mode="dark"></i>
<i class="theme-switch fa-solid fa-circle-half-stroke fa-lg" data-mode="auto"></i>
</button>
`);
</script>
<script>
document.write(`
<button class="btn btn-sm pst-navbar-icon search-button search-button__button" title="Search" aria-label="Search" data-bs-placement="bottom" data-bs-toggle="tooltip">
<i class="fa-solid fa-magnifying-glass fa-lg"></i>
</button>
`);
</script>
<button class="sidebar-toggle secondary-toggle btn btn-sm" title="Toggle secondary sidebar" data-bs-placement="bottom" data-bs-toggle="tooltip">
<span class="fa-solid fa-list"></span>
</button>
</div></div>
</div>
</div>
</div>
<div id="jb-print-docs-body" class="onlyprint">
<h1>TinyTorch: Build Your Own ML Framework from First Principles</h1>
<!-- Table of contents -->
<div id="print-main-content">
<div id="jb-print-toc">
<div>
<h2> Contents </h2>
</div>
<nav aria-label="Page">
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#the-historic-journey-from-mlps-to-modern-ai">The Historic Journey: From MLPs to Modern AI</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#learning-philosophy-build-use-reflect">Learning Philosophy: Build, Use, Reflect</a><ul class="nav section-nav flex-column">
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#example-activation-functions">Example: Activation Functions</a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#track-your-capabilities">Track Your Capabilities</a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#beyond-code-systems-intuition">Beyond Code: Systems Intuition</a></li>
</ul>
</li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#who-this-is-for">Who This Is For</a><ul class="nav section-nav flex-column">
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#perfect-for">Perfect For:</a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#prerequisites">Prerequisites:</a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#career-impact">Career Impact:</a></li>
</ul>
</li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#progressive-learning-journey-20-modules">Progressive Learning Journey: 20 Modules</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#complete-system-integration">Complete System Integration</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#ready-to-start">Ready to Start?</a><ul class="nav section-nav flex-column">
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#quick-taste-try-module-1-right-now">Quick Taste: Try Module 1 Right Now</a></li>
</ul>
</li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#acknowledgments">Acknowledgments</a></li>
</ul>
</nav>
</div>
</div>
</div>
<div id="searchbox"></div>
<article class="bd-article">
<section id="tinytorch-build-your-own-ml-framework-from-first-principles">
<h1>TinyTorch: Build Your Own ML Framework from First Principles<a class="headerlink" href="#tinytorch-build-your-own-ml-framework-from-first-principles" title="Link to this heading">#</a></h1>
<p><strong>Most ML education teaches you to <em>use</em> frameworks. TinyTorch teaches you to <em>build</em> them.</strong></p>
<p>Tiny🔥Torch is a minimalist framework for building machine learning systems from scratch—from tensors to systems. Instead of relying on PyTorch or TensorFlow, you implement everything yourself—tensors, autograd, optimizers, even MLOps tooling.</p>
<p><strong>The Vision: Train ML Systems Engineers, Not Just ML Users</strong></p>
<p>This hands-on approach builds the deep systems intuition that separates ML engineers from ML users. You’ll understand not just <em>what</em> neural networks do, but <em>how</em> they work under the hood, <em>why</em> certain design choices matter in production, and <em>when</em> to make trade-offs between memory, speed, and accuracy.</p>
<div class="tip admonition">
<p class="admonition-title">What You’ll Build: The Complete ML Evolution Story</p>
<p><strong>A complete ML framework from scratch</strong> that recreates the history of ML breakthroughs:</p>
<p><strong>MLP Era (1980s): The Foundation</strong></p>
<ul class="simple">
<li><p><strong>Train MLPs to 95%+ accuracy on MNIST</strong> (handwritten digit recognition)</p></li>
<li><p>Implement automatic differentiation from first principles</p></li>
<li><p>Master gradient-based optimization with SGD and Adam</p></li>
</ul>
<p><strong>CNN Revolution (2012): Spatial Intelligence</strong></p>
<ul class="simple">
<li><p><strong>Modern CNNs</strong>: Build convolutional networks from scratch</p></li>
<li><p><strong>CIFAR-10 Achievement</strong>: 75%+ accuracy on real images (our north star goal)</p></li>
<li><p><strong>Spatial Operations</strong>: Conv2d, MaxPool2d, and efficient data pipelines</p></li>
</ul>
<p><strong>Transformer Era (2017-present): Language &amp; Beyond</strong></p>
<ul class="simple">
<li><p><strong>TinyGPT</strong>: Complete language models using your vision framework</p></li>
<li><p><strong>Universal Architecture</strong>: 95% component reuse from vision to language</p></li>
<li><p><strong>Modern ML Systems</strong>: Full pipeline from data loading to deployment</p></li>
</ul>
<p><strong>Result:</strong> You experience firsthand how ML evolved from simple perceptrons to modern AI systems, implementing every breakthrough yourself. All 20 modules pass comprehensive tests with 100% health status.</p>
</div>
<p><em>Understanding how to build ML systems makes you a more effective ML engineer.</em></p>
<div class="note admonition">
<p class="admonition-title">The Perfect Learning Combination</p>
<p>TinyTorch was designed as the hands-on lab companion to <a class="reference external" href="https://mlsysbook.ai"><strong>Machine Learning Systems</strong></a> by <a class="reference external" href="https://vijay.seas.harvard.edu">Prof. Vijay Janapa Reddi</a> (Harvard). The book teaches you ML systems <strong>theory and principles</strong> - TinyTorch lets you <strong>implement and experience</strong> those concepts firsthand. Together, they provide complete ML systems mastery.</p>
</div>
<hr class="docutils" />
<section id="the-historic-journey-from-mlps-to-modern-ai">
<h2>The Historic Journey: From MLPs to Modern AI<a class="headerlink" href="#the-historic-journey-from-mlps-to-modern-ai" title="Link to this heading">#</a></h2>
<p>TinyTorch recreates the actual progression of machine learning breakthroughs. You don’t just learn modern AI - you <strong>experience the evolution that created it</strong>:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="n">MLP</span> <span class="n">Era</span> <span class="p">(</span><span class="mi">1980</span><span class="n">s</span><span class="p">):</span> <span class="n">CNN</span> <span class="n">Revolution</span> <span class="p">(</span><span class="mi">2012</span><span class="p">):</span> <span class="n">Transformer</span> <span class="n">Era</span> <span class="p">(</span><span class="mi">2017</span><span class="p">):</span>
<span class="err">├──</span> <span class="k">class</span><span class="w"> </span><span class="nc">MLP</span><span class="p">:</span> <span class="err">├──</span> <span class="k">class</span><span class="w"> </span><span class="nc">CNN</span><span class="p">:</span> <span class="err">├──</span> <span class="k">class</span><span class="w"> </span><span class="nc">TinyGPT</span><span class="p">:</span>
<span class="err">│</span> <span class="k">def</span><span class="w"> </span><span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span> <span class="err">│</span> <span class="k">def</span><span class="w"> </span><span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span> <span class="err">│</span> <span class="k">def</span><span class="w"> </span><span class="nf">forward</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">x</span><span class="p">):</span>
<span class="err">│</span> <span class="n">h</span> <span class="o">=</span> <span class="n">x</span><span class="o">.</span><span class="n">reshape</span><span class="p">(</span><span class="n">batch</span><span class="p">,</span><span class="o">-</span><span class="mi">1</span><span class="p">)</span> <span class="err">│</span> <span class="n">h</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">conv1</span><span class="p">(</span><span class="n">x</span><span class="p">)</span> <span class="err">│</span> <span class="n">h</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">embed</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
<span class="err">│</span> <span class="n">h</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">fc1</span><span class="p">(</span><span class="n">h</span><span class="p">)</span> <span class="err">│</span> <span class="n">h</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">pool</span><span class="p">(</span><span class="n">h</span><span class="p">)</span> <span class="err">│</span> <span class="n">h</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">attention</span><span class="p">(</span><span class="n">h</span><span class="p">)</span>
<span class="err">│</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">fc2</span><span class="p">(</span><span class="n">h</span><span class="p">)</span> <span class="err">│</span> <span class="n">h</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">conv2</span><span class="p">(</span><span class="n">h</span><span class="p">)</span> <span class="err">│</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">lm_head</span><span class="p">(</span><span class="n">h</span><span class="p">)</span>
<span class="err">│</span> <span class="err">│</span> <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">fc</span><span class="p">(</span><span class="n">h</span><span class="o">.</span><span class="n">flat</span><span class="p">())</span> <span class="err">│</span>
<span class="err">│</span> <span class="n">Result</span><span class="p">:</span> <span class="mi">95</span><span class="o">%+</span> <span class="n">MNIST</span> <span class="err">│</span> <span class="n">Result</span><span class="p">:</span> <span class="mi">75</span><span class="o">%+</span> <span class="n">CIFAR</span><span class="o">-</span><span class="mi">10</span> <span class="err">│</span> <span class="n">Result</span><span class="p">:</span> <span class="n">Language</span> <span class="n">generation</span>
<span class="err">└──</span> <span class="s2">&quot;Foundation for all └── &quot;</span><span class="n">Spatial</span> <span class="n">features</span> <span class="n">excel</span><span class="s2">&quot; └── &quot;</span><span class="n">Universal</span> <span class="n">architecture</span><span class="s2">&quot;</span>
<span class="n">neural</span> <span class="n">networks</span><span class="s2">&quot;</span>
<span class="n">The</span> <span class="n">SAME</span> <span class="n">tensor</span> <span class="n">operations</span> <span class="n">power</span> <span class="nb">all</span> <span class="n">three</span> <span class="n">eras</span> <span class="o">-</span> <span class="n">you</span> <span class="n">build</span> <span class="n">them</span> <span class="n">once</span><span class="p">,</span> <span class="n">use</span> <span class="n">everywhere</span><span class="o">.</span>
</pre></div>
</div>
<p><strong>The ML Evolution Story:</strong></p>
<ul class="simple">
<li><p><strong>1980s</strong>: MLPs could learn, but struggled with complex patterns</p></li>
<li><p><strong>1989</strong>: LeNet-1 proved convolutions extract spatial features</p></li>
<li><p><strong>1998</strong>: LeNet-5 established CNNs as the vision standard</p></li>
<li><p><strong>2012</strong>: AlexNet showed deep CNNs dominate computer vision</p></li>
<li><p><strong>2017</strong>: Transformers unified vision AND language processing</p></li>
<li><p><strong>Today</strong>: Same mathematical foundations power all AI systems</p></li>
</ul>
<p>TinyTorch focuses on implementation and systems thinking. You learn <em>how</em> to build working systems with progressive scaffolding, production ready practices, and comprehensive course infrastructure that bridges the gap between learning and building.</p>
<p><strong>What Makes This Different: Systems-First Thinking</strong></p>
<p>Traditional ML courses teach algorithms. TinyTorch teaches <strong>ML systems engineering</strong>:</p>
<ul class="simple">
<li><p><strong>Memory Management</strong>: Why Adam uses 3× more memory than SGD and when that matters</p></li>
<li><p><strong>Performance Analysis</strong>: How attention mechanisms scale O(N²) and limit context length</p></li>
<li><p><strong>Production Trade-offs</strong>: When to use gradient accumulation vs larger GPUs</p></li>
<li><p><strong>Hardware Awareness</strong>: How cache misses make naive convolution 100× slower</p></li>
<li><p><strong>System Design</strong>: How autograd graphs consume memory and enable gradient checkpointing</p></li>
</ul>
<p><strong>Result</strong>: You become the engineer who designs ML systems, not just uses them.</p>
</section>
<hr class="docutils" />
<section id="learning-philosophy-build-use-reflect">
<h2>Learning Philosophy: Build, Use, Reflect<a class="headerlink" href="#learning-philosophy-build-use-reflect" title="Link to this heading">#</a></h2>
<p>Every component follows the same powerful learning cycle:</p>
<section id="example-activation-functions">
<h3>Example: Activation Functions<a class="headerlink" href="#example-activation-functions" title="Link to this heading">#</a></h3>
<p><strong>Build:</strong> Implement ReLU from scratch</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">relu</span><span class="p">(</span><span class="n">x</span><span class="p">):</span>
<span class="c1"># YOU implement this function</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">maximum</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="n">x</span><span class="p">)</span> <span class="c1"># Your solution</span>
</pre></div>
</div>
<p><strong>Use:</strong> Immediately use your own code</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span><span class="w"> </span><span class="nn">tinytorch.core.activations</span><span class="w"> </span><span class="kn">import</span> <span class="n">ReLU</span> <span class="c1"># YOUR implementation!</span>
<span class="n">layer</span> <span class="o">=</span> <span class="n">ReLU</span><span class="p">()</span>
<span class="n">output</span> <span class="o">=</span> <span class="n">layer</span><span class="o">.</span><span class="n">forward</span><span class="p">(</span><span class="n">input_tensor</span><span class="p">)</span> <span class="c1"># Your code working!</span>
</pre></div>
</div>
<p><strong>Reflect:</strong> See it working in real networks</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="c1"># Your ReLU is now part of a real neural network</span>
<span class="n">model</span> <span class="o">=</span> <span class="n">Sequential</span><span class="p">([</span>
<span class="n">Dense</span><span class="p">(</span><span class="mi">784</span><span class="p">,</span> <span class="mi">128</span><span class="p">),</span>
<span class="n">ReLU</span><span class="p">(),</span> <span class="c1"># &lt;-- Your implementation</span>
<span class="n">Dense</span><span class="p">(</span><span class="mi">128</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span>
<span class="p">])</span>
</pre></div>
</div>
<p>This pattern repeats for every component: tensors, layers, optimizers, even MLOps systems. You build it, use it immediately, then reflect on how it fits into larger systems.</p>
</section>
<section id="track-your-capabilities">
<h3>Track Your Capabilities<a class="headerlink" href="#track-your-capabilities" title="Link to this heading">#</a></h3>
<p>TinyTorch uses a <a class="reference internal" href="checkpoint-system.html"><span class="std std-doc">checkpoint system</span></a> to track your progress through <strong>ML systems engineering capabilities</strong>:</p>
<ul class="simple">
<li><p><strong>Foundation</strong> → Core ML primitives and setup</p></li>
<li><p><strong>Architecture</strong> → Neural network building</p></li>
<li><p><strong>Training</strong> → Model training pipeline</p></li>
<li><p><strong>Inference</strong> → Deployment and optimization</p></li>
<li><p><strong>Serving</strong> → Complete system integration</p></li>
</ul>
<p>Use <code class="docutils literal notranslate"><span class="pre">tito</span> <span class="pre">checkpoint</span> <span class="pre">status</span></code> to see your progress anytime!</p>
</section>
<section id="beyond-code-systems-intuition">
<h3>Beyond Code: Systems Intuition<a class="headerlink" href="#beyond-code-systems-intuition" title="Link to this heading">#</a></h3>
<p>Each module includes <strong>ML Systems Thinking</strong> sections that connect your implementations to production reality:</p>
<ul class="simple">
<li><p><em>“How does your tensor implementation compare to PyTorch’s memory management?”</em></p></li>
<li><p><em>“When would you choose SGD over Adam in production training?”</em></p></li>
<li><p><em>“How do frameworks handle the quadratic memory scaling of attention?”</em></p></li>
<li><p><em>“What happens to your autograd implementation under distributed training?”</em></p></li>
</ul>
<p>These aren’t just academic questions - they’re the system-level challenges that ML engineers solve every day.</p>
</section>
</section>
<hr class="docutils" />
<section id="who-this-is-for">
<h2>Who This Is For<a class="headerlink" href="#who-this-is-for" title="Link to this heading">#</a></h2>
<section id="perfect-for">
<h3>Perfect For:<a class="headerlink" href="#perfect-for" title="Link to this heading">#</a></h3>
<ul class="simple">
<li><p><strong>CS students</strong> who want to understand ML systems beyond high-level APIs</p></li>
<li><p><strong>Software engineers</strong> transitioning to ML engineering roles</p></li>
<li><p><strong>ML practitioners</strong> who want to optimize and debug production systems</p></li>
<li><p><strong>Researchers</strong> who need to implement custom operations and architectures</p></li>
<li><p><strong>Anyone curious</strong> about how PyTorch/TensorFlow actually work under the hood</p></li>
</ul>
</section>
<section id="prerequisites">
<h3>Prerequisites:<a class="headerlink" href="#prerequisites" title="Link to this heading">#</a></h3>
<ul class="simple">
<li><p><strong>Python programming</strong> (comfortable with classes, functions, basic NumPy)</p></li>
<li><p><strong>Linear algebra basics</strong> (matrix multiplication, gradients)</p></li>
<li><p><strong>Learning mindset</strong> - we’ll teach you everything else!</p></li>
</ul>
</section>
<section id="career-impact">
<h3>Career Impact:<a class="headerlink" href="#career-impact" title="Link to this heading">#</a></h3>
<p>After TinyTorch, you’ll be the person your team asks:</p>
<ul class="simple">
<li><p><em>“Why is this training so slow?”</em> (You’ll know how to profile and optimize)</p></li>
<li><p><em>“Can we fit this model in GPU memory?”</em> (You’ll understand memory trade-offs)</p></li>
<li><p><em>“What’s the best optimizer for this problem?”</em> (You’ll know the system implications)</p></li>
</ul>
</section>
</section>
<hr class="docutils" />
<section id="progressive-learning-journey-20-modules">
<h2>Progressive Learning Journey: 20 Modules<a class="headerlink" href="#progressive-learning-journey-20-modules" title="Link to this heading">#</a></h2>
<div class="important admonition">
<p class="admonition-title">ML Systems Engineering Focus</p>
<p><strong>20 modules</strong> that build complete ML systems understanding through implementation:
From tensors to transformers, profiling to production deployment!</p>
</div>
<div class="note admonition">
<p class="admonition-title">Neural Network Foundations (Modules 1-8)</p>
<p><strong>1. Setup</strong> • <strong>2. Tensor</strong> • <strong>3. Activations</strong> • <strong>4. Layers</strong> • <strong>5. Losses</strong> • <strong>6. Autograd</strong> • <strong>7. Optimizers</strong> • <strong>8. Training</strong></p>
<p><strong>Foundation</strong>: Complete neural network training capability with gradient computation!</p>
<ul class="simple">
<li><p><strong>Module 2</strong>: Tensor operations and autograd from the start</p></li>
<li><p><strong>Module 6</strong>: Computational graphs and automatic differentiation</p></li>
<li><p><strong>Module 8</strong>: <strong>Train XOR and MNIST with complete framework!</strong></p></li>
</ul>
</div>
<div class="note admonition">
<p class="admonition-title">Computer Vision (Modules 9-10)</p>
<p><strong>9. Spatial</strong> • <strong>10. DataLoader</strong></p>
<p>Add convolutional intelligence: Conv2d, MaxPool2d, and efficient data loading.
<strong>Result</strong>: Train CNNs on CIFAR-10 to 75%+ accuracy!</p>
</div>
<div class="note admonition">
<p class="admonition-title">Language Models (Modules 11-14)</p>
<p><strong>11. Tokenization</strong> • <strong>12. Embeddings</strong> • <strong>13. Attention</strong> • <strong>14. Transformers</strong></p>
<p>Universal intelligence: Build GPT-style language models using your vision infrastructure.
<strong>Result</strong>: Complete TinyGPT using 95% of your vision components!</p>
</div>
<div class="note admonition">
<p class="admonition-title">Systems Optimization (Modules 15-20)</p>
<p><strong>15. Profiling</strong> • <strong>16. Acceleration</strong> • <strong>17. Quantization</strong> • <strong>18. Compression</strong> • <strong>19. Caching</strong> • <strong>20. Benchmarking</strong></p>
<p>Production systems: Optimize, profile, and compete in ML systems performance.
<strong>Result</strong>: TinyMLPerf competition framework for optimization excellence!</p>
</div>
</section>
<hr class="docutils" />
<section id="complete-system-integration">
<h2>Complete System Integration<a class="headerlink" href="#complete-system-integration" title="Link to this heading">#</a></h2>
<p><strong>This isn’t 20 separate exercises.</strong> Every component you build integrates into one fully functional ML framework with universal foundations:</p>
<div class="important admonition">
<p class="admonition-title">How It All Connects</p>
<pre class="mermaid">
flowchart TD
A[01_setup&lt;br/&gt;Environment &amp; CLI] --&gt; B[02_tensor&lt;br/&gt;Tensor + Autograd&lt;br/&gt;GRADIENTS FROM START]
B --&gt; C[03_activations&lt;br/&gt;ReLU + Softmax&lt;br/&gt;Nonlinearity Functions]
C --&gt; D[04_layers&lt;br/&gt;Linear + Module&lt;br/&gt;Building Blocks]
D --&gt; E[05_losses&lt;br/&gt;MSE + CrossEntropy&lt;br/&gt;Loss Functions]
E --&gt; F[06_autograd&lt;br/&gt;Computation Graphs&lt;br/&gt;Automatic Differentiation]
F --&gt; G[07_optimizers&lt;br/&gt;SGD + Adam&lt;br/&gt;Gradient Descent]
G --&gt; H[08_training&lt;br/&gt;Complete Training&lt;br/&gt;TRAIN NETWORKS NOW]
H --&gt; I[09_spatial&lt;br/&gt;Conv2d + MaxPool2d&lt;br/&gt;Spatial Operations]
H --&gt; J[10_dataloader&lt;br/&gt;CIFAR10 + DataLoader&lt;br/&gt;Data Pipeline]
I --&gt; J
J --&gt; K[CIFAR-10 CNNs&lt;br/&gt;75%+ Accuracy Achievement]
H --&gt; L[11_tokenization&lt;br/&gt;Text Processing]
L --&gt; M[12_embeddings&lt;br/&gt;Token Embeddings]
M --&gt; N[13_attention&lt;br/&gt;Multi-Head Attention]
N --&gt; O[14_transformers&lt;br/&gt;TinyGPT&lt;br/&gt;Language Models]
K --&gt; P[15_profiling&lt;br/&gt;Performance Analysis]
O --&gt; P
P --&gt; Q[16_acceleration&lt;br/&gt;Hardware Optimization]
Q --&gt; R[17_quantization&lt;br/&gt;Model Compression]
R --&gt; S[18_compression&lt;br/&gt;Pruning &amp; Distillation]
S --&gt; T[19_caching&lt;br/&gt;Memory Optimization]
T --&gt; U[20_benchmarking&lt;br/&gt;TinyMLPerf Competition]
style H fill:#ff6b6b,stroke:#333,stroke-width:3px,color:#fff
style K fill:#4ecdc4,stroke:#333,stroke-width:3px,color:#fff
style O fill:#45b7d1,stroke:#333,stroke-width:3px,color:#fff
style U fill:#ffd93d,stroke:#333,stroke-width:3px,color:#333
</pre></div>
<p><strong>Result:</strong> Every component you build converges into TinyGPT - proving your framework is complete and production-ready.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>
### TinyGPT: Proving Framework Universality
TinyGPT is your **capstone achievement** - demonstrating that the same foundations power all modern AI:
**The Historical Proof:**
- **1980s MLP components** → **1989 CNN revolution** → **2017 Transformer era**
- **95% component reuse**: Your tensors, layers, and training systems work across all three eras
- **Universal mathematics**: The same operations that power MLPs (52.7%) and CNNs (LeNet-5: 47.5%) also power language models
**What TinyGPT Proves:**
- **Framework Universality**: Vision and language use identical mathematical foundations
- **Component Integration**: All 20 modules work together seamlessly across domains
- **Systems Mastery**: You understand how modern AI builds on historical breakthroughs
- **Career Readiness**: You can implement any architecture from any era
**The Achievement:** Build GPT using components you designed for computer vision. This proves you didn&#39;t just learn isolated techniques - you built a complete, universal ML framework capable of any task.
---
## Choose Your Learning Path
```{admonition} Three Ways to Engage with TinyTorch
:class: important
### [Quick Exploration](usage-paths/quick-exploration.md) *(5 minutes)*
*&quot;I want to see what this is about&quot;*
- Click and run code immediately in your browser (Binder)
- No installation or setup required
- Implement ReLU, tensors, neural networks interactively
- Perfect for getting a feel for the course
### [Serious Development](usage-paths/serious-development.md) *(8+ weeks)*
*&quot;I want to build this myself&quot;*
- Fork the repo and work locally with full development environment
- Build complete ML framework from scratch with `tito` CLI
- 20 progressive assignments from setup to system optimization
- Professional development workflow with automated testing
### [Classroom Use](usage-paths/classroom-use.md) *(Instructors)*
*&quot;I want to teach this course&quot;*
- Complete course infrastructure with NBGrader integration
- Automated grading for comprehensive testing
- Flexible pacing (8-16 weeks) with proven pedagogical structure
- Turn-key solution for ML systems education
</pre></div>
</div>
</section>
<hr class="docutils" />
<section id="ready-to-start">
<h2>Ready to Start?<a class="headerlink" href="#ready-to-start" title="Link to this heading">#</a></h2>
<section id="quick-taste-try-module-1-right-now">
<h3>Quick Taste: Try Module 1 Right Now<a class="headerlink" href="#quick-taste-try-module-1-right-now" title="Link to this heading">#</a></h3>
<p>Want to see what TinyTorch feels like? <strong><a class="reference internal" href="chapters/01-setup.html"><span class="std std-doc">Launch the Setup chapter</span></a></strong> in Binder and implement your first TinyTorch function in 2 minutes!</p>
</section>
</section>
<hr class="docutils" />
<section id="acknowledgments">
<h2>Acknowledgments<a class="headerlink" href="#acknowledgments" title="Link to this heading">#</a></h2>
<p>TinyTorch originated from CS249r: Tiny Machine Learning Systems at Harvard University. We’re inspired by projects like <a class="reference external" href="https://github.com/geohot/tinygrad">tinygrad</a>, <a class="reference external" href="https://github.com/karpathy/micrograd">micrograd</a>, and <a class="reference external" href="https://minitorch.github.io/">MiniTorch</a> that demonstrate the power of minimal implementations.</p>
</section>
<div class="toctree-wrapper compound">
</div>
<div class="toctree-wrapper compound">
</div>
<div class="toctree-wrapper compound">
</div>
<div class="toctree-wrapper compound">
</div>
<div class="toctree-wrapper compound">
</div>
<div class="toctree-wrapper compound">
</div>
<div class="toctree-wrapper compound">
</div>
<div class="toctree-wrapper compound">
</div>
<div class="toctree-wrapper compound">
</div>
</section>
<script type="text/x-thebe-config">
{
requestKernel: true,
binderOptions: {
repo: "binder-examples/jupyter-stacks-datascience",
ref: "master",
},
codeMirrorConfig: {
theme: "abcdef",
mode: "python"
},
kernelOptions: {
name: "python3",
path: "./."
},
predefinedOutput: true
}
</script>
<script>kernelName = 'python3'</script>
</article>
<footer class="prev-next-footer d-print-none">
<div class="prev-next-area">
<a class="right-next"
href="learning-timeline.html"
title="next page">
<div class="prev-next-info">
<p class="prev-next-subtitle">next</p>
<p class="prev-next-title">Your TinyTorch Journey: From First Neural Network to Modern AI</p>
</div>
<i class="fa-solid fa-angle-right"></i>
</a>
</div>
</footer>
</div>
<div class="bd-sidebar-secondary bd-toc"><div class="sidebar-secondary-items sidebar-secondary__inner">
<div class="sidebar-secondary-item">
<div class="page-toc tocsection onthispage">
<i class="fa-solid fa-list"></i> Contents
</div>
<nav class="bd-toc-nav page-toc">
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#the-historic-journey-from-mlps-to-modern-ai">The Historic Journey: From MLPs to Modern AI</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#learning-philosophy-build-use-reflect">Learning Philosophy: Build, Use, Reflect</a><ul class="nav section-nav flex-column">
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#example-activation-functions">Example: Activation Functions</a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#track-your-capabilities">Track Your Capabilities</a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#beyond-code-systems-intuition">Beyond Code: Systems Intuition</a></li>
</ul>
</li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#who-this-is-for">Who This Is For</a><ul class="nav section-nav flex-column">
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#perfect-for">Perfect For:</a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#prerequisites">Prerequisites:</a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#career-impact">Career Impact:</a></li>
</ul>
</li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#progressive-learning-journey-20-modules">Progressive Learning Journey: 20 Modules</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#complete-system-integration">Complete System Integration</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#ready-to-start">Ready to Start?</a><ul class="nav section-nav flex-column">
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#quick-taste-try-module-1-right-now">Quick Taste: Try Module 1 Right Now</a></li>
</ul>
</li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#acknowledgments">Acknowledgments</a></li>
</ul>
</nav></div>
</div></div>
</div>
<footer class="bd-footer-content">
<div class="bd-footer-content__inner container">
<div class="footer-item">
<p class="component-author">
By Prof. Vijay Janapa Reddi (Harvard University)
</p>
</div>
<div class="footer-item">
<p class="copyright">
© Copyright 2025.
<br/>
</p>
</div>
<div class="footer-item">
</div>
<div class="footer-item">
</div>
</div>
</footer>
</main>
</div>
</div>
<!-- Scripts loaded after <body> so the DOM is not blocked -->
<script src="_static/scripts/bootstrap.js?digest=dfe6caa3a7d634c4db9b"></script>
<script src="_static/scripts/pydata-sphinx-theme.js?digest=dfe6caa3a7d634c4db9b"></script>
<footer class="bd-footer">
</footer>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink