diff --git a/book/_build/.doctrees/intro.doctree b/book/_build/.doctrees/intro.doctree index fe2481e4..ec891086 100644 Binary files a/book/_build/.doctrees/intro.doctree and b/book/_build/.doctrees/intro.doctree differ diff --git a/book/_build/html/_sources/intro.md b/book/_build/html/_sources/intro.md index 62379d3b..dfdefd7e 100644 --- a/book/_build/html/_sources/intro.md +++ b/book/_build/html/_sources/intro.md @@ -51,7 +51,7 @@ TinyTorch grew out of the CS249r: Tiny Machine Learning Systems course at Harvar ## 🎯 What You'll Build -### **Progressive Complexity - Each Module Builds on Previous Work** +### **Progressive Complexity - Each Module Builds on Prior Work** ```{admonition} 🏗️ Foundation (Modules 0-2) :class: note @@ -121,7 +121,7 @@ model = Sequential([ --- -## 📊 Proven Student Outcomes +## 📊 Expected Student Outcomes ```{admonition} Real Results from Real Students :class: success @@ -142,11 +142,10 @@ model = Sequential([ ## 🌟 What Makes This Different -### **🔬 Real Data, Real Systems** -- Work with **CIFAR-10** (not toy datasets) -- **Production-style** code organization -- **Performance considerations** and engineering practices -- **Professional development** workflow with automated testing +### **🔬 Engineering Principles +- **Production-style** code organization throughout every module +- **Performance-focused** engineering and optimization practices +- **Professional development** workflow with automated testing and CI ### **🚀 Immediate Feedback** - Code works **immediately** after implementation @@ -188,19 +187,16 @@ Traditional ML Course: TinyTorch Approach: ├── import torch ├── class Tensor: ├── model = nn.Linear(10, 1) │ def __add__(self, other): ... ├── loss = nn.MSELoss() │ def backward(self): ... -├── optimizer.step() ├── class Linear: -└── "How does this work?" 🤷 │ def forward(self, x): +└── optimizer.step() ├── class Linear: + │ def forward(self, x): │ return x @ self.weight + self.bias ├── def mse_loss(pred, target): │ return ((pred - target) ** 2).mean() ├── class SGD: │ def step(self): - │ param.data -= lr * param.grad - └── "I implemented every line!" 💪 + └── param.data -= lr * param.grad + +Transform your curiosity "How does this work?" 🤷 into confidence: "I built every part myself!" 💪 ``` -**Result:** You become the person others come to when they need to understand "how PyTorch actually works under the hood." - ---- - -*Built with ❤️ for hands-on ML systems education. Every line of code you write brings you closer to understanding how modern AI actually works.* +**Result:** You become the person others come to when they need to understand "how PyTorch actually works under the hood." Every line of code you write brings you closer to understanding how modern AI works. diff --git a/book/_build/html/intro.html b/book/_build/html/intro.html index 5625af5e..9392cc15 100644 --- a/book/_build/html/intro.html +++ b/book/_build/html/intro.html @@ -426,16 +426,16 @@ document.write(`
  • 📚 Educational Foundation
  • 🚀 Choose Your Learning Path
  • 🎯 What You’ll Build
  • 🎓 Learning Philosophy: Build → Use → Understand
    • -
  • 📊 Proven Student Outcomes
    • +
  • 📊 Expected Student Outcomes
  • 🌟 What Makes This Different @@ -508,8 +508,8 @@ document.write(`

    🎯 What You’ll Build#

    -
    -

    Progressive Complexity - Each Module Builds on Previous Work#

    +
    +

    Progressive Complexity - Each Module Builds on Prior Work#

    🏗️ Foundation (Modules 0-2)

    Week 1-3: Core Infrastructure

    @@ -580,8 +580,8 @@ document.write(`
    -
    -

    📊 Proven Student Outcomes#

    +
    +

    📊 Expected Student Outcomes#

    Real Results from Real Students

    After completing TinyTorch, students consistently:

    @@ -597,13 +597,12 @@ document.write(`

    🌟 What Makes This Different#

    -
    -

    🔬 Real Data, Real Systems#

    +
    +

    **🔬 Engineering Principles#

      -

    • Work with CIFAR-10 (not toy datasets)

      • -

    • Production-style code organization

      • -

    • Performance considerations and engineering practices

      • -

    • Professional development workflow with automated testing

      • +

    • Production-style code organization throughout every module

      • +

    • Performance-focused engineering and optimization practices

      • +

    • Professional development workflow with automated testing and CI

    @@ -647,20 +646,19 @@ document.write(` ├── import torch ├── class Tensor: ├── model = nn.Linear(10, 1) │ def __add__(self, other): ... ├── loss = nn.MSELoss() │ def backward(self): ... -├── optimizer.step() ├── class Linear: -└── "How does this work?" 🤷 │ def forward(self, x): +└── optimizer.step() ├── class Linear: + │ def forward(self, x): │ return x @ self.weight + self.bias ├── def mse_loss(pred, target): │ return ((pred - target) ** 2).mean() ├── class SGD: │ def step(self): - │ param.data -= lr * param.grad - └── "I implemented every line!" 💪 + └── param.data -= lr * param.grad + +Transform your curiosity "How does this work?" 🤷 into confidence: "I built every part myself!" 💪
    -

    Result: You become the person others come to when they need to understand “how PyTorch actually works under the hood.”

    -
    -

    Built with ❤️ for hands-on ML systems education. Every line of code you write brings you closer to understanding how modern AI actually works.

    +

    Result: You become the person others come to when they need to understand “how PyTorch actually works under the hood.” Every line of code you write brings you closer to understanding how modern AI works.

    @@ -734,16 +732,16 @@ document.write(`
  • 📚 Educational Foundation
  • 🚀 Choose Your Learning Path
  • 🎯 What You’ll Build
  • 🎓 Learning Philosophy: Build → Use → Understand
    • -
  • 📊 Proven Student Outcomes
    • +
  • 📊 Expected Student Outcomes
  • 🌟 What Makes This Different diff --git a/book/_build/html/searchindex.js b/book/_build/html/searchindex.js index 38f84c16..f6bbe3b9 100644 --- a/book/_build/html/searchindex.js +++ b/book/_build/html/searchindex.js @@ -1 +1 @@ -Search.setIndex({"alltitles": {"1. Review Course Materials": [[15, "review-course-materials"]], "1. System Awareness": [[0, "system-awareness"]], "2. Reproducibility": [[0, "reproducibility"]], "2. Setup Your Course": [[15, "setup-your-course"]], "3. Customize for Your Needs": [[15, "customize-for-your-needs"]], "3. Professional Development": [[0, "professional-development"]], "4. Launch Your Course!": [[15, "launch-your-course"]], "4. ML Systems Context": [[0, "ml-systems-context"]], "Activations - Nonlinearity in Neural Networks": [[2, null]], "Assessment Variations": [[15, "assessment-variations"]], "Assignment Management": [[15, "assignment-management"]], "Autograd - Automatic Differentiation Engine": [[7, null]], "Automated Grading System": [[15, "automated-grading-system"]], "Benchmarking - Systematic ML Performance Evaluation": [[12, null]], "Build Up: Chapter 4 - Layers": [[16, "build-up-chapter-4-layers"]], "Build \u2192 Use \u2192 Analyze": [[7, "build-use-analyze"], [8, "build-use-analyze"], [12, "build-use-analyze"]], "Build \u2192 Use \u2192 Deploy": [[13, "build-use-deploy"]], "Build \u2192 Use \u2192 Optimize": [[9, "build-use-optimize"], [10, "build-use-optimize"], [11, "build-use-optimize"]], "Build \u2192 Use \u2192 Reflect": [[4, "build-use-reflect"], [5, "build-use-reflect"], [6, "build-use-reflect"]], "Build \u2192 Use \u2192 Understand": [[1, "build-use-understand"], [2, "build-use-understand"], [3, "build-use-understand"], [17, "build-use-understand"]], "CNN - Convolutional Neural Networks": [[5, null]], "Choose Your Adventure": [[14, null]], "Choose Your Module": [[17, "choose-your-module"]], "Compression & Optimization - Making AI Models Efficient": [[10, null]], "Course Status Monitoring": [[15, "course-status-monitoring"]], "DataLoader - Data Loading and Preprocessing": [[6, null]], "Example: How You\u2019ll Learn Activation Functions": [[14, "example-how-you-ll-learn-activation-functions"]], "Flexible Pacing": [[15, "flexible-pacing"]], "Get Help": [[17, "get-help"]], "Grading & Feedback": [[15, "grading-feedback"]], "Grading Rubric": [[15, "grading-rubric"]], "Immediate Feedback": [[17, "immediate-feedback"]], "Kernels - Hardware-Optimized ML Operations": [[11, null]], "Layers - Building Blocks of Neural Networks": [[3, null]], "MLOps - Production ML Systems": [[13, null]], "Networks - Neural Network Architectures": [[4, null]], "Optimizers - Gradient-Based Parameter Updates": [[8, null]], "Point Distribution (Suggested)": [[15, "point-distribution-suggested"]], "Prerequisite Adjustments": [[15, "prerequisite-adjustments"]], "Progressive Complexity - Each Module Builds on Previous Work": [[14, "progressive-complexity-each-module-builds-on-previous-work"]], "Quality Assurance": [[17, "quality-assurance"]], "Quick Taste: Try Chapter 1 Right Now": [[14, "quick-taste-try-chapter-1-right-now"]], "Real Data, Real Systems": [[17, "real-data-real-systems"]], "Real Results": [[17, null]], "Real Results from Real Students": [[14, null]], "Setup - TinyTorch System Configuration": [[0, null]], "Start Here: Chapter 1 - Setup": [[16, "start-here-chapter-1-setup"]], "Step 1: Clone and Setup": [[15, "step-1-clone-and-setup"]], "Step 1: Get the Code": [[17, "step-1-get-the-code"]], "Step 2: Initialize NBGrader": [[15, "step-2-initialize-nbgrader"]], "Step 2: Setup Environment": [[17, "step-2-setup-environment"]], "Step 3: Generate First Assignment": [[15, "step-3-generate-first-assignment"]], "Step 3: Start Building": [[17, "step-3-start-building"]], "Step 4: Build \u2192 Test \u2192 Export \u2192 Use": [[17, "step-4-build-test-export-use"]], "Step 4: Test Workflow": [[15, "step-4-test-workflow"]], "Student Learning Results": [[15, null]], "Tensor - Core Data Structure": [[1, null]], "The Big Picture: Why Configuration Matters in ML Systems": [[0, "the-big-picture-why-configuration-matters-in-ml-systems"]], "The tito CLI System": [[17, "the-tito-cli-system"]], "Then Try: Chapter 3 - Activations": [[16, "then-try-chapter-3-activations"]], "Three Ways to Engage with TinyTorch": [[14, null]], "Tiny\ud83d\udd25Torch: Build your own Machine Learning framework from scratch.": [[14, null]], "Training - Complete Neural Network Training Pipeline": [[9, null]], "What You\u2019ll Experience": [[16, null]], "What You\u2019ll Learn": [[4, "what-you-ll-learn"], [5, "what-you-ll-learn"], [6, "what-you-ll-learn"]], "\u26a0\ufe0f Important Limitations": [[16, "important-limitations"]], "\u26a1 Production & Performance (Modules 10-13)": [[14, null]], "\u26a1 Production & Performance (Weeks 11-14)": [[15, "production-performance-weeks-11-14"]], "\u26a1 Production & Performance (Weeks 7-8)": [[17, "production-performance-weeks-7-8"]], "\ud83c\udf1f What Makes This Different": [[14, "what-makes-this-different"]], "\ud83c\udf93 Learning Path (Progressive Complexity)": [[17, "learning-path-progressive-complexity"]], "\ud83c\udf93 Learning Philosophy: Build \u2192 Use \u2192 Understand": [[14, "learning-philosophy-build-use-understand"]], "\ud83c\udf93 Proven Pedagogical Outcomes": [[15, "proven-pedagogical-outcomes"]], "\ud83c\udfaf Complete Course Infrastructure": [[15, "complete-course-infrastructure"]], "\ud83c\udfaf Customization Options": [[15, "customization-options"]], "\ud83c\udfaf Learning Goals": [[0, null], [1, null], [2, null], [3, null], [4, null], [5, null], [6, null], [7, null], [8, null], [9, null], [10, null], [11, null], [12, null], [13, null]], "\ud83c\udfaf Progressive Complexity": [[14, "progressive-complexity"]], "\ud83c\udfaf Training Systems (Modules 6-9)": [[14, null]], "\ud83c\udfaf Training Systems (Weeks 5-6)": [[17, "training-systems-weeks-5-6"]], "\ud83c\udfaf Training Systems (Weeks 7-10)": [[15, "training-systems-weeks-7-10"]], "\ud83c\udfaf What You\u2019ll Achieve": [[14, null]], "\ud83c\udfaf What You\u2019ll Build": [[14, "what-you-ll-build"], [17, "what-you-ll-build"]], "\ud83c\udfaf What You\u2019ll Understand": [[16, "what-you-ll-understand"]], "\ud83c\udfaf Why This Approach Works": [[17, "why-this-approach-works"]], "\ud83c\udfd7\ufe0f Big Picture: Why Build from Scratch?": [[14, "big-picture-why-build-from-scratch"]], "\ud83c\udfd7\ufe0f Foundation (Modules 0-2)": [[14, null]], "\ud83c\udfd7\ufe0f Foundation (Weeks 1-2)": [[17, "foundation-weeks-1-2"]], "\ud83c\udfd7\ufe0f Foundation Block (Weeks 1-3)": [[15, "foundation-block-weeks-1-3"]], "\ud83c\udfd7\ufe0f Serious Development Path": [[17, null]], "\ud83d\udc68\u200d\ud83c\udfeb Classroom Use Guide": [[15, null]], "\ud83d\udcbe Save Your Progress": [[0, null], [1, null], [2, null], [3, null], [4, null], [5, null], [6, null], [7, null], [8, null], [9, null], [10, null], [11, null], [12, null], [13, null]], "\ud83d\udcca Assessment & Grading": [[15, "assessment-grading"]], "\ud83d\udcca Proven Student Outcomes": [[14, "proven-student-outcomes"], [17, "proven-student-outcomes"]], "\ud83d\udcda Course Structure & Pacing": [[15, "course-structure-pacing"]], "\ud83d\udcda Educational Foundation": [[14, "educational-foundation"]], "\ud83d\udcda Recommended Exploration Path": [[16, "recommended-exploration-path"]], "\ud83d\udcde Instructor Support": [[15, "instructor-support"]], "\ud83d\udd04 Next Steps": [[16, "next-steps"]], "\ud83d\udd2c Quick Exploration Path": [[16, null]], "\ud83d\udd2c Real Data, Real Systems": [[14, "real-data-real-systems"]], "\ud83d\ude80 Choose Your Learning Path": [[14, "choose-your-learning-path"]], "\ud83d\ude80 Getting Started": [[15, "getting-started"]], "\ud83d\ude80 Immediate Feedback": [[14, "immediate-feedback"]], "\ud83d\ude80 Interactive Learning": [[0, "interactive-learning"], [1, "interactive-learning"], [2, "interactive-learning"], [3, "interactive-learning"], [4, "interactive-learning"], [5, "interactive-learning"], [6, "interactive-learning"], [7, "interactive-learning"], [8, "interactive-learning"], [9, "interactive-learning"], [10, "interactive-learning"], [11, "interactive-learning"], [12, "interactive-learning"], [13, "interactive-learning"]], "\ud83d\ude80 Launch Immediately (0 Setup Required)": [[16, "launch-immediately-0-setup-required"]], "\ud83d\ude80 Quick Instructor Setup": [[15, "quick-instructor-setup"]], "\ud83d\ude80 Quick Start (5 minutes)": [[17, "quick-start-5-minutes"]], "\ud83d\ude80 Ready to Start?": [[14, "ready-to-start"], [17, "ready-to-start"]], "\ud83d\udee0\ufe0f Development Workflow": [[17, "development-workflow"]], "\ud83d\udee0\ufe0f Instructor Workflow": [[15, "instructor-workflow"]], "\ud83e\uddf1 Building Blocks (Modules 3-5)": [[14, null]], "\ud83e\uddf1 Building Blocks (Weeks 3-4)": [[17, "building-blocks-weeks-3-4"]], "\ud83e\uddf1 Building Blocks (Weeks 4-6)": [[15, "building-blocks-weeks-4-6"]]}, "docnames": ["chapters/01-setup", "chapters/02-tensor", "chapters/03-activations", "chapters/04-layers", "chapters/05-networks", "chapters/06-cnn", "chapters/07-dataloader", "chapters/08-autograd", "chapters/09-optimizers", "chapters/10-training", "chapters/11-compression", "chapters/12-kernels", "chapters/13-benchmarking", "chapters/14-mlops", "intro", "usage-paths/classroom-use", "usage-paths/quick-exploration", "usage-paths/serious-development"], "envversion": {"sphinx": 62, "sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.intersphinx": 1, "sphinxcontrib.bibtex": 9}, "filenames": ["chapters/01-setup.md", "chapters/02-tensor.md", "chapters/03-activations.md", "chapters/04-layers.md", "chapters/05-networks.md", "chapters/06-cnn.md", "chapters/07-dataloader.md", "chapters/08-autograd.md", "chapters/09-optimizers.md", "chapters/10-training.md", "chapters/11-compression.md", "chapters/12-kernels.md", "chapters/13-benchmarking.md", "chapters/14-mlops.md", "intro.md", "usage-paths/classroom-use.md", "usage-paths/quick-exploration.md", "usage-paths/serious-development.md"], "indexentries": {}, "objects": {}, "objnames": {}, "objtypes": {}, "terms": {"": [0, 14], "00": [], "00_setup": [], "01": [15, 17], "01_setup": [0, 15, 17], "01_tensor": 15, "02": 17, "02_activ": 15, "02_tensor": [1, 15], "03": 17, "03_activ": [2, 15], "03_layer": 15, "04": [15, 17], "04_layer": [3, 15], "04_network": 15, "05": 17, "05_cnn": 15, "05_network": 4, "06": 17, "06_cnn": 5, "06_dataload": 15, "07": 17, "07_autograd": 15, "07_dataload": 6, "08": 17, "08_autograd": 7, "08_optim": 15, "09": 17, "09_optim": 8, "09_train": 15, "10": 17, "100": 17, "10_compress": 15, "10_train": 9, "11": [14, 17], "11_compress": 10, "11_kernel": 15, "12": [15, 17], "128": 14, "12_benchmark": 15, "12_kernel": 11, "13": [15, 17], "135": 15, "13_benchmark": 12, "13_mlop": 15, "14": 14, "145": 15, "14_mlop": 13, "15": 14, "16": [14, 15], "2": 16, "20": [14, 15, 16], "200": [14, 15, 17], "25": 15, "30": [14, 15], "35": 15, "4": 14, "40": 15, "445": 15, "5": [15, 16], "60": 15, "7": 14, "75": [10, 14, 15, 17], "784": 14, "8": [14, 15], "80": [14, 15], "85": [14, 15], "9": 15, "90": [14, 15], "95": [14, 15], "A": 17, "By": [4, 5, 6, 14], "For": 16, "In": 0, "No": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "Or": 15, "The": [4, 5, 6, 16], "__add__": 14, "abil": 16, "about": [14, 16], "abstract": [6, 14], "academ": [14, 15], "acceler": 8, "access": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "account": 16, "across": [14, 15, 17], "activ": [3, 5, 15, 17], "activations_dev": 2, "actual": 14, "adam": [8, 14, 15, 17], "adapt": 8, "add": [2, 15], "advanc": [14, 15], "affect": [4, 6, 10], "after": [14, 15, 16, 17], "aha": 14, "ai": [14, 16], "algorithm": [8, 11, 15], "all": [1, 3, 6, 13, 14, 15, 16, 17], "analysi": 15, "analyt": 15, "analyz": 9, "ani": [0, 4, 16], "anyth": 16, "api": 16, "appli": [8, 10, 11, 12], "approach": [14, 16], "approxim": 4, "ar": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "architectur": [10, 12, 14, 15, 17], "arithmet": 1, "arrai": [1, 14, 15, 17], "assign": [14, 17], "assur": 15, "attribut": 0, "author": [], "auto": [15, 17], "autograd": [14, 15, 17], "autograd_dev": 7, "autom": [0, 13, 14, 17], "automat": [1, 13, 14, 15, 17], "avail": 0, "avoid": 12, "awai": 17, "awar": 17, "backpropag": [7, 14, 15, 17], "backward": [7, 14], "base": [10, 15], "basic": 15, "batch": [6, 14, 15, 17], "becom": [0, 7, 14, 16], "befor": 14, "begin": 1, "beginn": 15, "behavior": [4, 8], "behind": [7, 14], "benchmark": [14, 15, 17], "benchmarking_dev": 12, "better": [11, 14, 15, 17], "between": [6, 16], "beyond": 11, "bia": 14, "bin": [15, 17], "binarycrossentropi": 9, "binder": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "blob": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "block": [0, 5, 16], "book": 14, "bring": [9, 14], "broad": 14, "broader": 0, "brows": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15], "browser": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "bug": 17, "build": 0, "built": [14, 15, 17], "button": 16, "c": 17, "cach": 11, "can": [0, 5, 14, 15, 16], "capabl": 4, "carefulli": 14, "cd": [15, 17], "chain": 14, "challeng": 15, "check": [15, 17], "checkpoint": [9, 17], "choos": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "cifar": [14, 15, 17], "class": [1, 6, 7, 14, 15, 16, 17], "classif": [9, 16, 17], "classifi": 17, "classroom": [14, 16], "clear": 15, "cli": [14, 15], "click": [14, 16], "clone": 17, "close": 13, "closer": 14, "cloud": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "cnn": [14, 17], "cnn_dev": 5, "code": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16], "codebas": [14, 15], "colab": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "collabor": 0, "collect": 15, "com": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17], "combin": [4, 10, 15, 16], "come": 14, "comment": 15, "commit": 16, "common": [4, 12], "commun": 15, "compact": 10, "compar": [8, 10], "comparison": 13, "compat": 0, "competit": 15, "complet": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 16, 17], "complex": [2, 4, 7], "compon": [3, 12, 13, 14, 15, 16, 17], "compos": [3, 4, 5], "composit": [4, 15, 17], "comprehens": [1, 2, 14, 15, 17], "compress": [11, 14, 15, 17], "compression_dev": 10, "comput": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17], "concept": 7, "confid": 12, "config": 0, "configur": 16, "connect": [0, 6], "consider": [14, 17], "consist": [6, 14, 17], "constraint": 0, "construct": 15, "contact": 17, "continu": [14, 15, 17], "conv2d": [5, 15], "converg": [8, 15, 17], "convolut": [14, 15, 17], "core": [0, 5, 8, 14, 15, 17], "correct": [15, 17], "correctli": 15, "cours": 14, "cover": 14, "coverag": 17, "cpu": [0, 11], "creat": [0, 1, 3, 4, 6, 7, 9, 10, 11, 12, 13, 15, 16, 17], "creation": 16, "credit": 0, "cross": [15, 17], "crossentropi": 9, "crucial": [6, 16], "cs249r": 14, "custom": [0, 11, 14, 17], "data": [2, 3, 5, 9, 15, 16], "dataload": [14, 17], "dataloader_dev": 6, "dataset": [6, 9, 14, 15, 17], "debug": 0, "deep": [3, 4, 14, 15, 17], "deepli": [14, 15, 17], "def": 14, "degrad": 13, "dens": [3, 5, 14, 15, 16, 17], "depend": [0, 17], "deploi": [14, 17], "deploy": [0, 9, 10, 13, 14, 15, 17], "descent": [8, 17], "design": 4, "detail": 15, "detect": [5, 13], "dev": 17, "develop": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16], "differ": [0, 4, 6, 12], "differenti": [1, 14, 15, 17], "dimension": [1, 14, 15, 17], "direct": [15, 17], "discord": 15, "discuss": 17, "distil": 10, "dive": 15, "do": 15, "docstr": 15, "doctor": [15, 17], "document": [0, 15], "doe": [14, 15], "done": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "download": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "drift": 13, "durat": 15, "dynam": [9, 17], "earli": 15, "educ": 16, "effect": 8, "effici": [5, 6, 11, 17], "emerg": 5, "enabl": [0, 2, 3, 5, 6, 8, 14, 17], "end": [4, 5, 6, 14, 16, 17], "enforc": 15, "engag": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "engin": [0, 6, 14, 15, 17], "ensur": [14, 15, 17], "entir": 9, "environ": [0, 14, 15, 16], "error": 17, "essenti": [2, 9], "evalu": [9, 14, 15, 17], "ever": 15, "everi": [14, 17], "everyth": [9, 15, 17], "exactli": 0, "expect": 15, "experi": [14, 17], "explicit": 5, "explor": 14, "export": [0, 15], "express": 7, "extend": [0, 15], "extra": 15, "f": 4, "familiar": 16, "faster": [10, 11], "featur": [3, 5, 17], "feed": 6, "feel": 14, "filepath": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "first": [14, 16, 17], "fit": [14, 16], "flatten": 5, "flexibl": 14, "flow": 7, "focu": 15, "follow": 17, "fork": [14, 16], "form": [1, 3], "forward": [4, 14, 15, 17], "foundat": [0, 1, 3, 6, 7], "four": [2, 10, 12], "framework": [6, 12, 15, 16, 17], "friendli": 15, "from": [3, 5, 6, 10, 15, 16, 17], "full": [14, 15], "function": [0, 2, 4, 9, 11, 15, 16, 17], "fundament": [1, 3, 4], "gain": [10, 16], "gap": 14, "gener": [12, 17], "get": [2, 14, 16], "gh": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "git": [15, 17], "github": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17], "give": 14, "go": 14, "goal": 15, "googl": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "gpu": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "grad": 14, "grade": [0, 14, 17], "gradebook": 15, "gradient": [7, 15, 17], "graduat": [14, 15], "graph": [7, 15, 17], "great": 16, "grew": 14, "ground": 14, "guid": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "guidanc": 17, "hand": [14, 16], "handl": [1, 6, 14, 17], "happen": 14, "hardwar": [0, 15, 17], "harvard": 14, "have": [14, 16, 17], "health": 17, "hello_tinytorch": 17, "hello_world": 14, "help": 0, "here": 5, "hidden": 0, "hierarch": 5, "high": [14, 15, 16, 17], "hood": [14, 16], "hour": 15, "how": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 16, 17], "http": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17], "i": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17], "imag": [5, 14, 15, 16, 17], "immedi": [2, 15], "implement": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], "import": [2, 3, 5, 14, 17], "importantli": 14, "improv": [8, 9, 11, 17], "inact": 16, "includ": 17, "indic": [14, 17], "industri": [12, 14, 15], "infer": 4, "info": [0, 15, 17], "inform": 0, "infrastructur": [13, 14, 17], "init": 15, "initi": [3, 17], "inlin": 17, "input_tensor": 14, "insight": 16, "inspir": 12, "instal": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], "institut": 15, "instructor": 14, "integr": [14, 15, 17], "intellig": 8, "intens": 15, "interact": 14, "intern": 17, "invari": 5, "ipynb": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "issu": [0, 13, 17], "its": 5, "janapa": [], "jump": 17, "jupyt": [16, 17], "just": [14, 16], "kei": [14, 15], "kernel": [14, 17], "kernels_dev": 11, "know": 14, "knowledg": 10, "l": 15, "lab": 17, "larger": [6, 14, 15], "latenc": 12, "launch": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14], "layer": [2, 4, 5, 14, 15, 17], "layer_1": 4, "layer_2": 4, "layer_n": 4, "layers_dev": 3, "layout": 1, "leaderboard": 15, "learn": 16, "learnabl": 5, "lectur": 15, "length": 15, "let": 0, "level": 16, "librari": [0, 11, 15, 17], "lifecycl": 13, "like": [4, 5, 14], "line": 14, "linear": [3, 14, 15, 17], "linux": 0, "live": 16, "ll": [0, 1, 7, 10, 11], "load": [14, 15, 17], "loader": 6, "local": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "loop": [5, 9, 13, 15, 17], "loss": [9, 14, 15, 17], "lr": 14, "machin": 6, "maco": 0, "magic": [14, 17], "magnitud": 10, "main": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "maintain": [0, 13], "mainten": [0, 13], "make": [5, 7, 15, 17], "manag": [0, 1, 6, 13, 17], "map": 5, "master": [0, 1, 2, 4, 6, 7, 8, 9, 10, 11], "math": [14, 16], "mathemat": [7, 15, 17], "matrix": [3, 14, 15, 16, 17], "matter": [10, 11, 16], "matur": 0, "maximum": 10, "mean": 14, "meaning": 12, "measur": [9, 11, 14, 15, 17], "mechan": 5, "memori": [0, 1, 6, 10, 11, 15, 17], "messag": [14, 17], "methodologi": 12, "metric": [0, 9, 14, 15, 17], "minut": [14, 16], "ml": [1, 6, 14, 15, 16, 17], "mlop": [14, 15, 17], "mlops_dev": 13, "mlp": [4, 15], "mlperf": [12, 15, 17], "mlsysbook": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "model": [0, 6, 9, 11, 12, 13, 14, 15, 17], "modern": [5, 11, 14], "modifi": [15, 16], "modul": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15], "moment": 14, "momentum": 8, "monitor": [0, 13, 14, 17], "more": [10, 16], "most": [2, 14, 15, 17], "move": 11, "mse": 9, "mse_loss": 14, "mseloss": 14, "multi": [4, 14, 15, 17], "multipl": [3, 14, 15, 16, 17], "my": 17, "mybind": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "myself": [14, 17], "n": 1, "naiv": 3, "name": 17, "nbgrader": [0, 1, 2, 14, 17], "nbgrader_config": 15, "need": [0, 14, 16], "network": [1, 6, 7, 8, 10, 14, 15, 16, 17], "networks_dev": 4, "neural": [1, 6, 7, 8, 10, 14, 15, 16, 17], "new": 17, "next": 14, "nn": 14, "nonlinear": [15, 16, 17], "note": 15, "notebook": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17], "novel": 15, "now": [16, 17], "number": 4, "numpi": [11, 14], "o": 6, "off": 10, "offic": 15, "offlin": 12, "onc": 15, "ongo": [15, 16], "open": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17], "oper": [1, 5, 7, 14, 15, 16, 17], "optim": [0, 6, 14, 15, 17], "optimizers_dev": 8, "orchestr": [9, 14, 15], "org": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17], "organ": [14, 17], "other": [0, 5, 14], "our": [14, 15], "out": 14, "output": 14, "over": 13, "overal": 15, "overview": 15, "own": [16, 17], "pace": 14, "packag": [0, 17], "parallel": 11, "param": 14, "paramet": 5, "part": [0, 14], "pass": [4, 15], "pattern": [2, 4, 5, 6, 8, 11, 14, 15, 17], "pedagog": 14, "per": 15, "perceptron": [4, 17], "perfect": [5, 14, 15, 16, 17], "perform": [0, 1, 6, 7, 9, 11, 13], "persist": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17], "person": [0, 14], "pipelin": [5, 6, 13, 14, 15, 17], "pitfal": 12, "platform": 0, "possibl": [7, 14, 17], "power": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "practic": [14, 15, 17], "pre": 16, "precis": 0, "pred": 14, "prefer": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "preprocess": [14, 15, 17], "principl": [11, 14], "privat": 15, "process": [6, 9, 11, 15, 17], "product": 0, "profession": [12, 14, 15, 17], "profil": [11, 15], "progress": 15, "project": [12, 15, 16], "propag": [7, 15, 17], "proper": [0, 12], "properli": 0, "provid": [15, 17], "prune": [10, 14, 15, 17], "pursu": [14, 15], "py": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17], "python": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17], "pytorch": [14, 17], "qualiti": 15, "quantiz": [10, 14, 15, 17], "queri": 0, "question": 17, "rang": 15, "rate": [8, 14, 15, 17], "readi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16], "real": [0, 3, 6, 9, 10, 11, 15, 16], "realli": 1, "reddi": [], "reduc": [10, 17], "reduct": [14, 15], "regress": 9, "relationship": 16, "releas": [15, 17], "reliabl": 17, "relu": [2, 14, 16, 17], "remov": 10, "repeat": 14, "repo": [14, 16], "report": [12, 15, 17], "requir": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], "research": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15], "resourc": [], "respond": 13, "result": [2, 12, 16], "retrain": 13, "return": 14, "reusabl": 6, "role": [14, 15], "run": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "satisfi": 16, "save": 16, "scaffold": 14, "scalabl": 6, "scale": 0, "scenario": 12, "schedul": [8, 14, 15, 17], "scratch": [3, 5, 6, 10, 15, 16, 17], "second": 14, "see": [2, 3, 5, 8, 13, 14, 16, 17], "self": [14, 17], "semest": 15, "sequenti": [4, 14, 15, 17], "seriou": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "server": 0, "session": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16], "setup": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14], "setup_dev": [0, 17], "sgd": [8, 14, 15, 17], "sh": [15, 17], "shallow": 4, "shape": 1, "share": 5, "should": 14, "show": [0, 17], "shuffl": [6, 17], "sigmoid": [2, 14, 17], "simd": 11, "simpl": [4, 14, 16], "simul": 15, "size": [10, 14, 15, 17], "skill": 0, "skip": 15, "slack": 15, "slide": [5, 15], "small": 14, "smaller": 10, "softmax": [2, 17], "softwar": 0, "solut": [0, 14], "sourc": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17], "spatial": [5, 15, 17], "specif": [1, 15, 17], "stack": [3, 17], "standard": [11, 12, 15], "statist": [12, 15, 17], "statu": 17, "step": 14, "strategi": [3, 8, 17], "structur": 10, "style": [11, 14, 15, 17], "submiss": [15, 17], "success": [14, 17], "successfulli": 14, "suggest": 17, "support": [0, 17], "switch": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "system": [1, 6, 7, 12, 16], "systemat": [11, 14, 15, 17], "take": 15, "tanh": [2, 14, 17], "target": [14, 15], "teach": [0, 14, 15, 16], "team": 17, "techniqu": 10, "temporari": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "tensor": [2, 14, 15, 16, 17], "tensor_dev": 1, "test": [0, 1, 2, 3, 14], "than": [6, 11, 14, 15, 17], "thei": [9, 14], "them": [6, 14], "themselv": 13, "theori": [], "thi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "think": 6, "through": [2, 4, 7, 8, 15], "throughput": 12, "time": 13, "timeout": 16, "tinytorch": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17], "tito": [14, 15], "togeth": [3, 9, 13, 16, 17], "toi": [14, 17], "tool": [11, 14, 15], "total": 15, "track": [0, 7, 17], "trade": 10, "tradit": 14, "train": [0, 7, 8, 10, 13], "training_dev": 9, "transform": [1, 2, 3, 5, 15, 16], "translat": 5, "trigger": 13, "troubleshoot": 0, "truli": 7, "try": 17, "turn": [14, 15], "tutori": 15, "type": [1, 6, 17], "under": [14, 16], "understand": [0, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15], "unimport": 10, "unit": 15, "univers": 14, "up": 14, "updat": [0, 17], "us": 0, "usag": 10, "user": [14, 15], "util": 11, "v": [14, 16], "v2": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "valid": [12, 14, 15, 17], "valuabl": 16, "variabl": 7, "variant": 17, "ve": 16, "vector": 11, "verbos": 15, "verifi": [15, 17], "version": [0, 15], "view": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "vijai": [], "virtual": [15, 17], "vision": [5, 14, 15, 17], "visual": [2, 4, 14, 15, 16, 17], "wa": 0, "wai": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "want": [14, 16, 17], "we": [3, 4, 9, 11, 12, 13], "week": 14, "weekli": 15, "weight": [3, 10, 14, 15, 17], "welcom": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "what": 0, "when": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "where": [1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13], "while": 14, "why": [2, 5, 6, 10, 11, 13, 16], "wide": 4, "window": [0, 5], "without": [13, 16], "work": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16], "workflow": [0, 1, 2, 13, 14, 16], "workload": 11, "world": [9, 10, 14, 15, 17], "write": 14, "x": [4, 14], "you": [0, 1, 7, 10, 11], "your": 16, "yourself": 14}, "titles": ["Setup - TinyTorch System Configuration", "Tensor - Core Data Structure", "Activations - Nonlinearity in Neural Networks", "Layers - Building Blocks of Neural Networks", "Networks - Neural Network Architectures", "CNN - Convolutional Neural Networks", "DataLoader - Data Loading and Preprocessing", "Autograd - Automatic Differentiation Engine", "Optimizers - Gradient-Based Parameter Updates", "Training - Complete Neural Network Training Pipeline", "Compression & Optimization - Making AI Models Efficient", "Kernels - Hardware-Optimized ML Operations", "Benchmarking - Systematic ML Performance Evaluation", "MLOps - Production ML Systems", "Tiny\ud83d\udd25Torch: Build your own Machine Learning framework from scratch.", "\ud83d\udc68\u200d\ud83c\udfeb Classroom Use Guide", "\ud83d\udd2c Quick Exploration Path", "\ud83c\udfd7\ufe0f Serious Development Path"], "titleterms": {"0": [14, 16], "1": [0, 14, 15, 16, 17], "10": [14, 15], "11": 15, "13": 14, "14": 15, "2": [0, 14, 15, 17], "3": [0, 14, 15, 16, 17], "4": [0, 15, 16, 17], "5": [14, 17], "6": [14, 15, 17], "7": [15, 17], "8": 17, "9": 14, "The": [0, 17], "Then": 16, "achiev": 14, "activ": [2, 14, 16], "adjust": 15, "adventur": 14, "ai": 10, "analyz": [7, 8, 12], "approach": 17, "architectur": 4, "assess": 15, "assign": 15, "assur": 17, "autograd": 7, "autom": 15, "automat": 7, "awar": 0, "base": 8, "benchmark": 12, "big": [0, 14], "block": [3, 14, 15, 17], "build": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], "chapter": [14, 16], "choos": [14, 17], "classroom": 15, "cli": 17, "clone": 15, "cnn": 5, "code": 17, "complet": [9, 15], "complex": [14, 17], "compress": 10, "configur": 0, "context": 0, "convolut": 5, "core": 1, "cours": 15, "custom": 15, "data": [1, 6, 14, 17], "dataload": 6, "deep": [], "deploi": 13, "develop": [0, 17], "differ": 14, "differenti": 7, "distribut": 15, "each": 14, "educ": 14, "effici": 10, "engag": 14, "engin": 7, "environ": 17, "evalu": 12, "exampl": 14, "experi": 16, "explor": 16, "export": 17, "feedback": [14, 15, 17], "first": 15, "flexibl": 15, "foundat": [14, 15, 17], "framework": 14, "from": 14, "function": 14, "gener": 15, "get": [15, 17], "go": [], "goal": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "grade": 15, "gradient": 8, "guid": 15, "hardwar": 11, "help": 17, "here": 16, "how": 14, "immedi": [14, 16, 17], "import": 16, "infrastructur": 15, "initi": 15, "instructor": 15, "interact": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "kernel": 11, "launch": [15, 16], "layer": [3, 16], "learn": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17], "limit": 16, "ll": [4, 5, 6, 14, 16, 17], "load": 6, "machin": 14, "make": [10, 14], "manag": 15, "materi": 15, "matter": 0, "minut": 17, "ml": [0, 11, 12, 13], "mlop": 13, "model": 10, "modul": [14, 17], "monitor": 15, "nbgrader": 15, "need": 15, "network": [2, 3, 4, 5, 9], "neural": [2, 3, 4, 5, 9], "next": 16, "nonlinear": 2, "now": 14, "oper": 11, "optim": [8, 9, 10, 11], "option": 15, "outcom": [14, 15, 17], "own": 14, "pace": 15, "paramet": 8, "path": [14, 16, 17], "pedagog": 15, "perform": [12, 14, 15, 17], "philosophi": 14, "pictur": [0, 14], "pipelin": 9, "point": 15, "preprocess": 6, "prerequisit": 15, "previou": 14, "product": [13, 14, 15, 17], "profession": 0, "progress": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17], "proven": [14, 15, 17], "qualiti": 17, "quick": [14, 15, 16, 17], "readi": [14, 17], "real": [14, 17], "recommend": 16, "reflect": [4, 5, 6], "reproduc": 0, "requir": 16, "result": [14, 15, 17], "review": 15, "right": 14, "rubric": 15, "save": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "scratch": 14, "seriou": 17, "setup": [0, 15, 16, 17], "small": [], "start": [14, 15, 16, 17], "statu": 15, "step": [15, 16, 17], "structur": [1, 15], "student": [14, 15, 17], "suggest": 15, "support": 15, "system": [0, 13, 14, 15, 17], "systemat": 12, "tast": 14, "tensor": 1, "test": [15, 17], "thi": [14, 17], "three": 14, "tini": 14, "tinytorch": [0, 14], "tito": 17, "torch": 14, "train": [9, 14, 15, 17], "try": [14, 16], "understand": [1, 2, 3, 14, 16, 17], "up": 16, "updat": 8, "us": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17], "variat": 15, "wai": 14, "week": [15, 17], "what": [4, 5, 6, 14, 16, 17], "why": [0, 14, 17], "work": [14, 17], "workflow": [15, 17], "you": [4, 5, 6, 14, 16, 17], "your": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17]}}) \ No newline at end of file +Search.setIndex({"alltitles": {"**\ud83d\udd2c Engineering Principles": [[14, "engineering-principles"]], "1. Review Course Materials": [[15, "review-course-materials"]], "1. System Awareness": [[0, "system-awareness"]], "2. Reproducibility": [[0, "reproducibility"]], "2. Setup Your Course": [[15, "setup-your-course"]], "3. Customize for Your Needs": [[15, "customize-for-your-needs"]], "3. Professional Development": [[0, "professional-development"]], "4. Launch Your Course!": [[15, "launch-your-course"]], "4. ML Systems Context": [[0, "ml-systems-context"]], "Activations - Nonlinearity in Neural Networks": [[2, null]], "Assessment Variations": [[15, "assessment-variations"]], "Assignment Management": [[15, "assignment-management"]], "Autograd - Automatic Differentiation Engine": [[7, null]], "Automated Grading System": [[15, "automated-grading-system"]], "Benchmarking - Systematic ML Performance Evaluation": [[12, null]], "Build Up: Chapter 4 - Layers": [[16, "build-up-chapter-4-layers"]], "Build \u2192 Use \u2192 Analyze": [[7, "build-use-analyze"], [8, "build-use-analyze"], [12, "build-use-analyze"]], "Build \u2192 Use \u2192 Deploy": [[13, "build-use-deploy"]], "Build \u2192 Use \u2192 Optimize": [[9, "build-use-optimize"], [10, "build-use-optimize"], [11, "build-use-optimize"]], "Build \u2192 Use \u2192 Reflect": [[4, "build-use-reflect"], [5, "build-use-reflect"], [6, "build-use-reflect"]], "Build \u2192 Use \u2192 Understand": [[1, "build-use-understand"], [2, "build-use-understand"], [3, "build-use-understand"], [17, "build-use-understand"]], "CNN - Convolutional Neural Networks": [[5, null]], "Choose Your Adventure": [[14, null]], "Choose Your Module": [[17, "choose-your-module"]], "Compression & Optimization - Making AI Models Efficient": [[10, null]], "Course Status Monitoring": [[15, "course-status-monitoring"]], "DataLoader - Data Loading and Preprocessing": [[6, null]], "Example: How You\u2019ll Learn Activation Functions": [[14, "example-how-you-ll-learn-activation-functions"]], "Flexible Pacing": [[15, "flexible-pacing"]], "Get Help": [[17, "get-help"]], "Grading & Feedback": [[15, "grading-feedback"]], "Grading Rubric": [[15, "grading-rubric"]], "Immediate Feedback": [[17, "immediate-feedback"]], "Kernels - Hardware-Optimized ML Operations": [[11, null]], "Layers - Building Blocks of Neural Networks": [[3, null]], "MLOps - Production ML Systems": [[13, null]], "Networks - Neural Network Architectures": [[4, null]], "Optimizers - Gradient-Based Parameter Updates": [[8, null]], "Point Distribution (Suggested)": [[15, "point-distribution-suggested"]], "Prerequisite Adjustments": [[15, "prerequisite-adjustments"]], "Progressive Complexity - Each Module Builds on Prior Work": [[14, "progressive-complexity-each-module-builds-on-prior-work"]], "Quality Assurance": [[17, "quality-assurance"]], "Quick Taste: Try Chapter 1 Right Now": [[14, "quick-taste-try-chapter-1-right-now"]], "Real Data, Real Systems": [[17, "real-data-real-systems"]], "Real Results": [[17, null]], "Real Results from Real Students": [[14, null]], "Setup - TinyTorch System Configuration": [[0, null]], "Start Here: Chapter 1 - Setup": [[16, "start-here-chapter-1-setup"]], "Step 1: Clone and Setup": [[15, "step-1-clone-and-setup"]], "Step 1: Get the Code": [[17, "step-1-get-the-code"]], "Step 2: Initialize NBGrader": [[15, "step-2-initialize-nbgrader"]], "Step 2: Setup Environment": [[17, "step-2-setup-environment"]], "Step 3: Generate First Assignment": [[15, "step-3-generate-first-assignment"]], "Step 3: Start Building": [[17, "step-3-start-building"]], "Step 4: Build \u2192 Test \u2192 Export \u2192 Use": [[17, "step-4-build-test-export-use"]], "Step 4: Test Workflow": [[15, "step-4-test-workflow"]], "Student Learning Results": [[15, null]], "Tensor - Core Data Structure": [[1, null]], "The Big Picture: Why Configuration Matters in ML Systems": [[0, "the-big-picture-why-configuration-matters-in-ml-systems"]], "The tito CLI System": [[17, "the-tito-cli-system"]], "Then Try: Chapter 3 - Activations": [[16, "then-try-chapter-3-activations"]], "Three Ways to Engage with TinyTorch": [[14, null]], "Tiny\ud83d\udd25Torch: Build your own Machine Learning framework from scratch.": [[14, null]], "Training - Complete Neural Network Training Pipeline": [[9, null]], "What You\u2019ll Experience": [[16, null]], "What You\u2019ll Learn": [[4, "what-you-ll-learn"], [5, "what-you-ll-learn"], [6, "what-you-ll-learn"]], "\u26a0\ufe0f Important Limitations": [[16, "important-limitations"]], "\u26a1 Production & Performance (Modules 10-13)": [[14, null]], "\u26a1 Production & Performance (Weeks 11-14)": [[15, "production-performance-weeks-11-14"]], "\u26a1 Production & Performance (Weeks 7-8)": [[17, "production-performance-weeks-7-8"]], "\ud83c\udf1f What Makes This Different": [[14, "what-makes-this-different"]], "\ud83c\udf93 Learning Path (Progressive Complexity)": [[17, "learning-path-progressive-complexity"]], "\ud83c\udf93 Learning Philosophy: Build \u2192 Use \u2192 Understand": [[14, "learning-philosophy-build-use-understand"]], "\ud83c\udf93 Proven Pedagogical Outcomes": [[15, "proven-pedagogical-outcomes"]], "\ud83c\udfaf Complete Course Infrastructure": [[15, "complete-course-infrastructure"]], "\ud83c\udfaf Customization Options": [[15, "customization-options"]], "\ud83c\udfaf Learning Goals": [[0, null], [1, null], [2, null], [3, null], [4, null], [5, null], [6, null], [7, null], [8, null], [9, null], [10, null], [11, null], [12, null], [13, null]], "\ud83c\udfaf Progressive Complexity": [[14, "progressive-complexity"]], "\ud83c\udfaf Training Systems (Modules 6-9)": [[14, null]], "\ud83c\udfaf Training Systems (Weeks 5-6)": [[17, "training-systems-weeks-5-6"]], "\ud83c\udfaf Training Systems (Weeks 7-10)": [[15, "training-systems-weeks-7-10"]], "\ud83c\udfaf What You\u2019ll Achieve": [[14, null]], "\ud83c\udfaf What You\u2019ll Build": [[14, "what-you-ll-build"], [17, "what-you-ll-build"]], "\ud83c\udfaf What You\u2019ll Understand": [[16, "what-you-ll-understand"]], "\ud83c\udfaf Why This Approach Works": [[17, "why-this-approach-works"]], "\ud83c\udfd7\ufe0f Big Picture: Why Build from Scratch?": [[14, "big-picture-why-build-from-scratch"]], "\ud83c\udfd7\ufe0f Foundation (Modules 0-2)": [[14, null]], "\ud83c\udfd7\ufe0f Foundation (Weeks 1-2)": [[17, "foundation-weeks-1-2"]], "\ud83c\udfd7\ufe0f Foundation Block (Weeks 1-3)": [[15, "foundation-block-weeks-1-3"]], "\ud83c\udfd7\ufe0f Serious Development Path": [[17, null]], "\ud83d\udc68\u200d\ud83c\udfeb Classroom Use Guide": [[15, null]], "\ud83d\udcbe Save Your Progress": [[0, null], [1, null], [2, null], [3, null], [4, null], [5, null], [6, null], [7, null], [8, null], [9, null], [10, null], [11, null], [12, null], [13, null]], "\ud83d\udcca Assessment & Grading": [[15, "assessment-grading"]], "\ud83d\udcca Expected Student Outcomes": [[14, "expected-student-outcomes"]], "\ud83d\udcca Proven Student Outcomes": [[17, "proven-student-outcomes"]], "\ud83d\udcda Course Structure & Pacing": [[15, "course-structure-pacing"]], "\ud83d\udcda Educational Foundation": [[14, "educational-foundation"]], "\ud83d\udcda Recommended Exploration Path": [[16, "recommended-exploration-path"]], "\ud83d\udcde Instructor Support": [[15, "instructor-support"]], "\ud83d\udd04 Next Steps": [[16, "next-steps"]], "\ud83d\udd2c Quick Exploration Path": [[16, null]], "\ud83d\ude80 Choose Your Learning Path": [[14, "choose-your-learning-path"]], "\ud83d\ude80 Getting Started": [[15, "getting-started"]], "\ud83d\ude80 Immediate Feedback": [[14, "immediate-feedback"]], "\ud83d\ude80 Interactive Learning": [[0, "interactive-learning"], [1, "interactive-learning"], [2, "interactive-learning"], [3, "interactive-learning"], [4, "interactive-learning"], [5, "interactive-learning"], [6, "interactive-learning"], [7, "interactive-learning"], [8, "interactive-learning"], [9, "interactive-learning"], [10, "interactive-learning"], [11, "interactive-learning"], [12, "interactive-learning"], [13, "interactive-learning"]], "\ud83d\ude80 Launch Immediately (0 Setup Required)": [[16, "launch-immediately-0-setup-required"]], "\ud83d\ude80 Quick Instructor Setup": [[15, "quick-instructor-setup"]], "\ud83d\ude80 Quick Start (5 minutes)": [[17, "quick-start-5-minutes"]], "\ud83d\ude80 Ready to Start?": [[14, "ready-to-start"], [17, "ready-to-start"]], "\ud83d\udee0\ufe0f Development Workflow": [[17, "development-workflow"]], "\ud83d\udee0\ufe0f Instructor Workflow": [[15, "instructor-workflow"]], "\ud83e\uddf1 Building Blocks (Modules 3-5)": [[14, null]], "\ud83e\uddf1 Building Blocks (Weeks 3-4)": [[17, "building-blocks-weeks-3-4"]], "\ud83e\uddf1 Building Blocks (Weeks 4-6)": [[15, "building-blocks-weeks-4-6"]]}, "docnames": ["chapters/01-setup", "chapters/02-tensor", "chapters/03-activations", "chapters/04-layers", "chapters/05-networks", "chapters/06-cnn", "chapters/07-dataloader", "chapters/08-autograd", "chapters/09-optimizers", "chapters/10-training", "chapters/11-compression", "chapters/12-kernels", "chapters/13-benchmarking", "chapters/14-mlops", "intro", "usage-paths/classroom-use", "usage-paths/quick-exploration", "usage-paths/serious-development"], "envversion": {"sphinx": 62, "sphinx.domains.c": 3, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 9, "sphinx.domains.index": 1, "sphinx.domains.javascript": 3, "sphinx.domains.math": 2, "sphinx.domains.python": 4, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.intersphinx": 1, "sphinxcontrib.bibtex": 9}, "filenames": ["chapters/01-setup.md", "chapters/02-tensor.md", "chapters/03-activations.md", "chapters/04-layers.md", "chapters/05-networks.md", "chapters/06-cnn.md", "chapters/07-dataloader.md", "chapters/08-autograd.md", "chapters/09-optimizers.md", "chapters/10-training.md", "chapters/11-compression.md", "chapters/12-kernels.md", "chapters/13-benchmarking.md", "chapters/14-mlops.md", "intro.md", "usage-paths/classroom-use.md", "usage-paths/quick-exploration.md", "usage-paths/serious-development.md"], "indexentries": {}, "objects": {}, "objnames": {}, "objtypes": {}, "terms": {"": [0, 14], "00": [], "00_setup": [], "01": [15, 17], "01_setup": [0, 15, 17], "01_tensor": 15, "02": 17, "02_activ": 15, "02_tensor": [1, 15], "03": 17, "03_activ": [2, 15], "03_layer": 15, "04": [15, 17], "04_layer": [3, 15], "04_network": 15, "05": 17, "05_cnn": 15, "05_network": 4, "06": 17, "06_cnn": 5, "06_dataload": 15, "07": 17, "07_autograd": 15, "07_dataload": 6, "08": 17, "08_autograd": 7, "08_optim": 15, "09": 17, "09_optim": 8, "09_train": 15, "10": 17, "100": 17, "10_compress": 15, "10_train": 9, "11": [14, 17], "11_compress": 10, "11_kernel": 15, "12": [15, 17], "128": 14, "12_benchmark": 15, "12_kernel": 11, "13": [15, 17], "135": 15, "13_benchmark": 12, "13_mlop": 15, "14": 14, "145": 15, "14_mlop": 13, "15": 14, "16": [14, 15], "2": 16, "20": [14, 15, 16], "200": [14, 15, 17], "25": 15, "30": [14, 15], "35": 15, "4": 14, "40": 15, "445": 15, "5": [15, 16], "60": 15, "7": 14, "75": [10, 14, 15, 17], "784": 14, "8": [14, 15], "80": [14, 15], "85": [14, 15], "9": 15, "90": [14, 15], "95": [14, 15], "A": 17, "By": [4, 5, 6, 14], "For": 16, "In": 0, "No": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "Or": 15, "The": [4, 5, 6, 16], "__add__": 14, "abil": 16, "about": [14, 16], "abstract": [6, 14], "academ": [14, 15], "acceler": 8, "access": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "account": 16, "across": [14, 15, 17], "activ": [3, 5, 15, 17], "activations_dev": 2, "actual": 14, "adam": [8, 14, 15, 17], "adapt": 8, "add": [2, 15], "advanc": [14, 15], "affect": [4, 6, 10], "after": [14, 15, 16, 17], "aha": 14, "ai": [14, 16], "algorithm": [8, 11, 15], "all": [1, 3, 6, 13, 14, 15, 16, 17], "analysi": 15, "analyt": 15, "analyz": 9, "ani": [0, 4, 16], "anyth": 16, "api": 16, "appli": [8, 10, 11, 12], "approach": [14, 16], "approxim": 4, "ar": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "architectur": [10, 12, 14, 15, 17], "arithmet": 1, "arrai": [1, 14, 15, 17], "assign": [14, 17], "assur": 15, "attribut": 0, "author": [], "auto": [15, 17], "autograd": [14, 15, 17], "autograd_dev": 7, "autom": [0, 13, 14, 17], "automat": [1, 13, 14, 15, 17], "avail": 0, "avoid": 12, "awai": 17, "awar": 17, "backpropag": [7, 14, 15, 17], "backward": [7, 14], "base": [10, 15], "basic": 15, "batch": [6, 14, 15, 17], "becom": [0, 7, 14, 16], "befor": 14, "begin": 1, "beginn": 15, "behavior": [4, 8], "behind": [7, 14], "benchmark": [14, 15, 17], "benchmarking_dev": 12, "better": [11, 14, 15, 17], "between": [6, 16], "beyond": 11, "bia": 14, "bin": [15, 17], "binarycrossentropi": 9, "binder": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "blob": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "block": [0, 5, 16], "book": 14, "bring": [9, 14], "broad": 14, "broader": 0, "brows": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15], "browser": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "bug": 17, "build": 0, "built": [14, 15, 17], "button": 16, "c": 17, "cach": 11, "can": [0, 5, 14, 15, 16], "capabl": 4, "carefulli": 14, "cd": [15, 17], "chain": 14, "challeng": 15, "check": [15, 17], "checkpoint": [9, 17], "choos": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "ci": 14, "cifar": [14, 15, 17], "class": [1, 6, 7, 14, 15, 16, 17], "classif": [9, 16, 17], "classifi": 17, "classroom": [14, 16], "clear": 15, "cli": [14, 15], "click": [14, 16], "clone": 17, "close": 13, "closer": 14, "cloud": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "cnn": [14, 17], "cnn_dev": 5, "code": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16], "codebas": [14, 15], "colab": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "collabor": 0, "collect": 15, "com": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17], "combin": [4, 10, 15, 16], "come": 14, "comment": 15, "commit": 16, "common": [4, 12], "commun": 15, "compact": 10, "compar": [8, 10], "comparison": 13, "compat": 0, "competit": 15, "complet": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 16, 17], "complex": [2, 4, 7], "compon": [3, 12, 13, 14, 15, 16, 17], "compos": [3, 4, 5], "composit": [4, 15, 17], "comprehens": [1, 2, 14, 15, 17], "compress": [11, 14, 15, 17], "compression_dev": 10, "comput": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17], "concept": 7, "confid": [12, 14], "config": 0, "configur": 16, "connect": [0, 6], "consider": 17, "consist": [6, 14, 17], "constraint": 0, "construct": 15, "contact": 17, "continu": [14, 15, 17], "conv2d": [5, 15], "converg": [8, 15, 17], "convolut": [14, 15, 17], "core": [0, 5, 8, 14, 15, 17], "correct": [15, 17], "correctli": 15, "cours": 14, "cover": 14, "coverag": 17, "cpu": [0, 11], "creat": [0, 1, 3, 4, 6, 7, 9, 10, 11, 12, 13, 15, 16, 17], "creation": 16, "credit": 0, "cross": [15, 17], "crossentropi": 9, "crucial": [6, 16], "cs249r": 14, "curios": 14, "custom": [0, 11, 14, 17], "data": [2, 3, 5, 9, 14, 15, 16], "dataload": [14, 17], "dataloader_dev": 6, "dataset": [6, 9, 14, 15, 17], "debug": 0, "deep": [3, 4, 14, 15, 17], "deepli": [14, 15, 17], "def": 14, "degrad": 13, "dens": [3, 5, 14, 15, 16, 17], "depend": [0, 17], "deploi": [14, 17], "deploy": [0, 9, 10, 13, 14, 15, 17], "descent": [8, 17], "design": 4, "detail": 15, "detect": [5, 13], "dev": 17, "develop": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16], "differ": [0, 4, 6, 12], "differenti": [1, 14, 15, 17], "dimension": [1, 14, 15, 17], "direct": [15, 17], "discord": 15, "discuss": 17, "distil": 10, "dive": 15, "do": 15, "docstr": 15, "doctor": [15, 17], "document": [0, 15], "doe": [14, 15], "done": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "download": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "drift": 13, "durat": 15, "dynam": [9, 17], "earli": 15, "educ": 16, "effect": 8, "effici": [5, 6, 11, 17], "emerg": 5, "enabl": [0, 2, 3, 5, 6, 8, 14, 17], "end": [4, 5, 6, 14, 16, 17], "enforc": 15, "engag": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "engin": [0, 6, 15, 17], "ensur": [14, 15, 17], "entir": 9, "environ": [0, 14, 15, 16], "error": 17, "essenti": [2, 9], "evalu": [9, 14, 15, 17], "ever": 15, "everi": [14, 17], "everyth": [9, 15, 17], "exactli": 0, "expect": 15, "experi": [14, 17], "explicit": 5, "explor": 14, "export": [0, 15], "express": 7, "extend": [0, 15], "extra": 15, "f": 4, "familiar": 16, "faster": [10, 11], "featur": [3, 5, 17], "feed": 6, "feel": 14, "filepath": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "first": [14, 16, 17], "fit": [14, 16], "flatten": 5, "flexibl": 14, "flow": 7, "focu": 15, "focus": 14, "follow": 17, "fork": [14, 16], "form": [1, 3], "forward": [4, 14, 15, 17], "foundat": [0, 1, 3, 6, 7], "four": [2, 10, 12], "framework": [6, 12, 15, 16, 17], "friendli": 15, "from": [3, 5, 6, 10, 15, 16, 17], "full": [14, 15], "function": [0, 2, 4, 9, 11, 15, 16, 17], "fundament": [1, 3, 4], "gain": [10, 16], "gap": 14, "gener": [12, 17], "get": [2, 14, 16], "gh": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "git": [15, 17], "github": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17], "give": 14, "go": 14, "goal": 15, "googl": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "gpu": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "grad": 14, "grade": [0, 14, 17], "gradebook": 15, "gradient": [7, 15, 17], "graduat": [14, 15], "graph": [7, 15, 17], "great": 16, "grew": 14, "ground": 14, "guid": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "guidanc": 17, "hand": [14, 16], "handl": [1, 6, 14, 17], "happen": 14, "hardwar": [0, 15, 17], "harvard": 14, "have": [14, 16, 17], "health": 17, "hello_tinytorch": 17, "hello_world": 14, "help": 0, "here": 5, "hidden": 0, "hierarch": 5, "high": [14, 15, 16, 17], "hood": [14, 16], "hour": 15, "how": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 16, 17], "http": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17], "i": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17], "imag": [5, 14, 15, 16, 17], "immedi": [2, 15], "implement": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], "import": [2, 3, 5, 14, 17], "importantli": 14, "improv": [8, 9, 11, 17], "inact": 16, "includ": 17, "indic": [14, 17], "industri": [12, 14, 15], "infer": 4, "info": [0, 15, 17], "inform": 0, "infrastructur": [13, 14, 17], "init": 15, "initi": [3, 17], "inlin": 17, "input_tensor": 14, "insight": 16, "inspir": 12, "instal": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], "institut": 15, "instructor": 14, "integr": [14, 15, 17], "intellig": 8, "intens": 15, "interact": 14, "intern": 17, "invari": 5, "ipynb": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "issu": [0, 13, 17], "its": 5, "janapa": [], "jump": 17, "jupyt": [16, 17], "just": [14, 16], "kei": [14, 15], "kernel": [14, 17], "kernels_dev": 11, "know": 14, "knowledg": 10, "l": 15, "lab": 17, "larger": [6, 14, 15], "latenc": 12, "launch": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14], "layer": [2, 4, 5, 14, 15, 17], "layer_1": 4, "layer_2": 4, "layer_n": 4, "layers_dev": 3, "layout": 1, "leaderboard": 15, "learn": 16, "learnabl": 5, "lectur": 15, "length": 15, "let": 0, "level": 16, "librari": [0, 11, 15, 17], "lifecycl": 13, "like": [4, 5, 14], "line": 14, "linear": [3, 14, 15, 17], "linux": 0, "live": 16, "ll": [0, 1, 7, 10, 11], "load": [14, 15, 17], "loader": 6, "local": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "loop": [5, 9, 13, 15, 17], "loss": [9, 14, 15, 17], "lr": 14, "machin": 6, "maco": 0, "magic": [14, 17], "magnitud": 10, "main": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "maintain": [0, 13], "mainten": [0, 13], "make": [5, 7, 15, 17], "manag": [0, 1, 6, 13, 17], "map": 5, "master": [0, 1, 2, 4, 6, 7, 8, 9, 10, 11], "math": [14, 16], "mathemat": [7, 15, 17], "matrix": [3, 14, 15, 16, 17], "matter": [10, 11, 16], "matur": 0, "maximum": 10, "mean": 14, "meaning": 12, "measur": [9, 11, 14, 15, 17], "mechan": 5, "memori": [0, 1, 6, 10, 11, 15, 17], "messag": [14, 17], "methodologi": 12, "metric": [0, 9, 14, 15, 17], "minut": [14, 16], "ml": [1, 6, 14, 15, 16, 17], "mlop": [14, 15, 17], "mlops_dev": 13, "mlp": [4, 15], "mlperf": [12, 15, 17], "mlsysbook": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "model": [0, 6, 9, 11, 12, 13, 14, 15, 17], "modern": [5, 11, 14], "modifi": [15, 16], "modul": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15], "moment": 14, "momentum": 8, "monitor": [0, 13, 14, 17], "more": [10, 16], "most": [2, 14, 15, 17], "move": 11, "mse": 9, "mse_loss": 14, "mseloss": 14, "multi": [4, 14, 15, 17], "multipl": [3, 14, 15, 16, 17], "my": 17, "mybind": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "myself": [14, 17], "n": 1, "naiv": 3, "name": 17, "nbgrader": [0, 1, 2, 14, 17], "nbgrader_config": 15, "need": [0, 14, 16], "network": [1, 6, 7, 8, 10, 14, 15, 16, 17], "networks_dev": 4, "neural": [1, 6, 7, 8, 10, 14, 15, 16, 17], "new": 17, "next": 14, "nn": 14, "nonlinear": [15, 16, 17], "note": 15, "notebook": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17], "novel": 15, "now": [16, 17], "number": 4, "numpi": [11, 14], "o": 6, "off": 10, "offic": 15, "offlin": 12, "onc": 15, "ongo": [15, 16], "open": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17], "oper": [1, 5, 7, 14, 15, 16, 17], "optim": [0, 6, 14, 15, 17], "optimizers_dev": 8, "orchestr": [9, 14, 15], "org": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17], "organ": [14, 17], "other": [0, 5, 14], "our": [14, 15], "out": 14, "output": 14, "over": 13, "overal": 15, "overview": 15, "own": [16, 17], "pace": 14, "packag": [0, 17], "parallel": 11, "param": 14, "paramet": 5, "part": [0, 14], "pass": [4, 15], "pattern": [2, 4, 5, 6, 8, 11, 14, 15, 17], "pedagog": 14, "per": 15, "perceptron": [4, 17], "perfect": [5, 14, 15, 16, 17], "perform": [0, 1, 6, 7, 9, 11, 13], "persist": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17], "person": [0, 14], "pipelin": [5, 6, 13, 14, 15, 17], "pitfal": 12, "platform": 0, "possibl": [7, 14, 17], "power": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "practic": [14, 15, 17], "pre": 16, "precis": 0, "pred": 14, "prefer": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "preprocess": [14, 15, 17], "principl": 11, "privat": 15, "process": [6, 9, 11, 15, 17], "product": 0, "profession": [12, 14, 15, 17], "profil": [11, 15], "progress": 15, "project": [12, 15, 16], "propag": [7, 15, 17], "proper": [0, 12], "properli": 0, "proven": 14, "provid": [15, 17], "prune": [10, 14, 15, 17], "pursu": [14, 15], "py": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17], "python": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17], "pytorch": [14, 17], "qualiti": 15, "quantiz": [10, 14, 15, 17], "queri": 0, "question": 17, "rang": 15, "rate": [8, 14, 15, 17], "readi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16], "real": [0, 3, 6, 9, 10, 11, 15, 16], "realli": 1, "reddi": [], "reduc": [10, 17], "reduct": [14, 15], "regress": 9, "relationship": 16, "releas": [15, 17], "reliabl": 17, "relu": [2, 14, 16, 17], "remov": 10, "repeat": 14, "repo": [14, 16], "report": [12, 15, 17], "requir": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], "research": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15], "resourc": [], "respond": 13, "result": [2, 12, 16], "retrain": 13, "return": 14, "reusabl": 6, "role": [14, 15], "run": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "satisfi": 16, "save": 16, "scaffold": 14, "scalabl": 6, "scale": 0, "scenario": 12, "schedul": [8, 14, 15, 17], "scratch": [3, 5, 6, 10, 15, 16, 17], "second": 14, "see": [2, 3, 5, 8, 13, 14, 16, 17], "self": [14, 17], "semest": 15, "sequenti": [4, 14, 15, 17], "seriou": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "server": 0, "session": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16], "setup": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14], "setup_dev": [0, 17], "sgd": [8, 14, 15, 17], "sh": [15, 17], "shallow": 4, "shape": 1, "share": 5, "should": 14, "show": [0, 17], "shuffl": [6, 17], "sigmoid": [2, 14, 17], "simd": 11, "simpl": [4, 14, 16], "simul": 15, "size": [10, 14, 15, 17], "skill": 0, "skip": 15, "slack": 15, "slide": [5, 15], "small": 14, "smaller": 10, "softmax": [2, 17], "softwar": 0, "solut": [0, 14], "sourc": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17], "spatial": [5, 15, 17], "specif": [1, 15, 17], "stack": [3, 17], "standard": [11, 12, 15], "statist": [12, 15, 17], "statu": 17, "step": 14, "strategi": [3, 8, 17], "structur": 10, "style": [11, 14, 15, 17], "submiss": [15, 17], "success": [14, 17], "successfulli": 14, "suggest": 17, "support": [0, 17], "switch": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "system": [1, 6, 7, 12, 16], "systemat": [11, 14, 15, 17], "take": 15, "tanh": [2, 14, 17], "target": [14, 15], "teach": [0, 14, 15, 16], "team": 17, "techniqu": 10, "temporari": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "tensor": [2, 14, 15, 16, 17], "tensor_dev": 1, "test": [0, 1, 2, 3, 14], "than": [6, 11, 14, 15, 17], "thei": [9, 14], "them": [6, 14], "themselv": 13, "theori": [], "thi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "think": 6, "through": [2, 4, 7, 8, 15], "throughout": 14, "throughput": 12, "time": 13, "timeout": 16, "tinytorch": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17], "tito": [14, 15], "togeth": [3, 9, 13, 16, 17], "toi": 17, "tool": [11, 14, 15], "total": 15, "track": [0, 7, 17], "trade": 10, "tradit": 14, "train": [0, 7, 8, 10, 13], "training_dev": 9, "transform": [1, 2, 3, 5, 14, 15, 16], "translat": 5, "trigger": 13, "troubleshoot": 0, "truli": 7, "try": 17, "turn": [14, 15], "tutori": 15, "type": [1, 6, 17], "under": [14, 16], "understand": [0, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15], "unimport": 10, "unit": 15, "univers": 14, "up": 14, "updat": [0, 17], "us": 0, "usag": 10, "user": [14, 15], "util": 11, "v": [14, 16], "v2": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "valid": [12, 14, 15, 17], "valuabl": 16, "variabl": 7, "variant": 17, "ve": 16, "vector": 11, "verbos": 15, "verifi": [15, 17], "version": [0, 15], "view": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "vijai": [], "virtual": [15, 17], "vision": [5, 14, 15, 17], "visual": [2, 4, 14, 15, 16, 17], "wa": 0, "wai": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "want": [14, 16, 17], "we": [3, 4, 9, 11, 12, 13], "week": 14, "weekli": 15, "weight": [3, 10, 14, 15, 17], "welcom": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "what": 0, "when": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "where": [1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13], "while": 14, "why": [2, 5, 6, 10, 11, 13, 16], "wide": 4, "window": [0, 5], "without": [13, 16], "work": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16], "workflow": [0, 1, 2, 13, 14, 16], "workload": 11, "world": [9, 10, 14, 15, 17], "write": 14, "x": [4, 14], "you": [0, 1, 7, 10, 11], "your": 16, "yourself": 14}, "titles": ["Setup - TinyTorch System Configuration", "Tensor - Core Data Structure", "Activations - Nonlinearity in Neural Networks", "Layers - Building Blocks of Neural Networks", "Networks - Neural Network Architectures", "CNN - Convolutional Neural Networks", "DataLoader - Data Loading and Preprocessing", "Autograd - Automatic Differentiation Engine", "Optimizers - Gradient-Based Parameter Updates", "Training - Complete Neural Network Training Pipeline", "Compression & Optimization - Making AI Models Efficient", "Kernels - Hardware-Optimized ML Operations", "Benchmarking - Systematic ML Performance Evaluation", "MLOps - Production ML Systems", "Tiny\ud83d\udd25Torch: Build your own Machine Learning framework from scratch.", "\ud83d\udc68\u200d\ud83c\udfeb Classroom Use Guide", "\ud83d\udd2c Quick Exploration Path", "\ud83c\udfd7\ufe0f Serious Development Path"], "titleterms": {"0": [14, 16], "1": [0, 14, 15, 16, 17], "10": [14, 15], "11": 15, "13": 14, "14": 15, "2": [0, 14, 15, 17], "3": [0, 14, 15, 16, 17], "4": [0, 15, 16, 17], "5": [14, 17], "6": [14, 15, 17], "7": [15, 17], "8": 17, "9": 14, "The": [0, 17], "Then": 16, "achiev": 14, "activ": [2, 14, 16], "adjust": 15, "adventur": 14, "ai": 10, "analyz": [7, 8, 12], "approach": 17, "architectur": 4, "assess": 15, "assign": 15, "assur": 17, "autograd": 7, "autom": 15, "automat": 7, "awar": 0, "base": 8, "benchmark": 12, "big": [0, 14], "block": [3, 14, 15, 17], "build": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], "chapter": [14, 16], "choos": [14, 17], "classroom": 15, "cli": 17, "clone": 15, "cnn": 5, "code": 17, "complet": [9, 15], "complex": [14, 17], "compress": 10, "configur": 0, "context": 0, "convolut": 5, "core": 1, "cours": 15, "custom": 15, "data": [1, 6, 17], "dataload": 6, "deep": [], "deploi": 13, "develop": [0, 17], "differ": 14, "differenti": 7, "distribut": 15, "each": 14, "educ": 14, "effici": 10, "engag": 14, "engin": [7, 14], "environ": 17, "evalu": 12, "exampl": 14, "expect": 14, "experi": 16, "explor": 16, "export": 17, "feedback": [14, 15, 17], "first": 15, "flexibl": 15, "foundat": [14, 15, 17], "framework": 14, "from": 14, "function": 14, "gener": 15, "get": [15, 17], "go": [], "goal": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "grade": 15, "gradient": 8, "guid": 15, "hardwar": 11, "help": 17, "here": 16, "how": 14, "immedi": [14, 16, 17], "import": 16, "infrastructur": 15, "initi": 15, "instructor": 15, "interact": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "kernel": 11, "launch": [15, 16], "layer": [3, 16], "learn": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17], "limit": 16, "ll": [4, 5, 6, 14, 16, 17], "load": 6, "machin": 14, "make": [10, 14], "manag": 15, "materi": 15, "matter": 0, "minut": 17, "ml": [0, 11, 12, 13], "mlop": 13, "model": 10, "modul": [14, 17], "monitor": 15, "nbgrader": 15, "need": 15, "network": [2, 3, 4, 5, 9], "neural": [2, 3, 4, 5, 9], "next": 16, "nonlinear": 2, "now": 14, "oper": 11, "optim": [8, 9, 10, 11], "option": 15, "outcom": [14, 15, 17], "own": 14, "pace": 15, "paramet": 8, "path": [14, 16, 17], "pedagog": 15, "perform": [12, 14, 15, 17], "philosophi": 14, "pictur": [0, 14], "pipelin": 9, "point": 15, "preprocess": 6, "prerequisit": 15, "previou": [], "principl": 14, "prior": 14, "product": [13, 14, 15, 17], "profession": 0, "progress": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17], "proven": [15, 17], "qualiti": 17, "quick": [14, 15, 16, 17], "readi": [14, 17], "real": [14, 17], "recommend": 16, "reflect": [4, 5, 6], "reproduc": 0, "requir": 16, "result": [14, 15, 17], "review": 15, "right": 14, "rubric": 15, "save": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "scratch": 14, "seriou": 17, "setup": [0, 15, 16, 17], "small": [], "start": [14, 15, 16, 17], "statu": 15, "step": [15, 16, 17], "structur": [1, 15], "student": [14, 15, 17], "suggest": 15, "support": 15, "system": [0, 13, 14, 15, 17], "systemat": 12, "tast": 14, "tensor": 1, "test": [15, 17], "thi": [14, 17], "three": 14, "tini": 14, "tinytorch": [0, 14], "tito": 17, "torch": 14, "train": [9, 14, 15, 17], "try": [14, 16], "understand": [1, 2, 3, 14, 16, 17], "up": 16, "updat": 8, "us": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17], "variat": 15, "wai": 14, "week": [15, 17], "what": [4, 5, 6, 14, 16, 17], "why": [0, 14, 17], "work": [14, 17], "workflow": [15, 17], "you": [4, 5, 6, 14, 16, 17], "your": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17]}}) \ No newline at end of file diff --git a/book/intro.md b/book/intro.md index 62379d3b..4277ea4d 100644 --- a/book/intro.md +++ b/book/intro.md @@ -51,7 +51,7 @@ TinyTorch grew out of the CS249r: Tiny Machine Learning Systems course at Harvar ## 🎯 What You'll Build -### **Progressive Complexity - Each Module Builds on Previous Work** +### **Progressive Complexity - Each Module Builds on Prior Work** ```{admonition} 🏗️ Foundation (Modules 0-2) :class: note @@ -121,7 +121,7 @@ model = Sequential([ --- -## 📊 Proven Student Outcomes +## 📊 Expected Student Outcomes ```{admonition} Real Results from Real Students :class: success @@ -142,11 +142,10 @@ model = Sequential([ ## 🌟 What Makes This Different -### **🔬 Real Data, Real Systems** -- Work with **CIFAR-10** (not toy datasets) -- **Production-style** code organization -- **Performance considerations** and engineering practices -- **Professional development** workflow with automated testing +### **🔬 Engineering Principles** +- **Production-style** code organization throughout every module +- **Performance-focused** engineering and optimization practices +- **Professional development** workflow with automated testing and CI ### **🚀 Immediate Feedback** - Code works **immediately** after implementation @@ -188,19 +187,16 @@ Traditional ML Course: TinyTorch Approach: ├── import torch ├── class Tensor: ├── model = nn.Linear(10, 1) │ def __add__(self, other): ... ├── loss = nn.MSELoss() │ def backward(self): ... -├── optimizer.step() ├── class Linear: -└── "How does this work?" 🤷 │ def forward(self, x): +└── optimizer.step() ├── class Linear: + │ def forward(self, x): │ return x @ self.weight + self.bias ├── def mse_loss(pred, target): │ return ((pred - target) ** 2).mean() ├── class SGD: │ def step(self): - │ param.data -= lr * param.grad - └── "I implemented every line!" 💪 + └── param.data -= lr * param.grad + +Go from "How does this work?" 🤷 to "I implemented every line!" 💪 ``` -**Result:** You become the person others come to when they need to understand "how PyTorch actually works under the hood." - ---- - -*Built with ❤️ for hands-on ML systems education. Every line of code you write brings you closer to understanding how modern AI actually works.* +**Result:** You become the person others come to when they need to understand "how PyTorch actually works under the hood." Every line of code you write brings you closer to understanding how modern AI works.