Module 0: Setup - TinyTorch System Configuration#
+Setup - TinyTorch System Configuration#
Welcome to TinyTorch! This setup module configures your personal TinyTorch installation and teaches you the NBGrader workflow.
π― Learning Goals
@@ -570,7 +570,7 @@ document.write(` title="next page"> diff --git a/book/_build/html/chapters/01-setup.html b/book/_build/html/chapters/01-setup.html new file mode 100644 index 00000000..d0ab6c62 --- /dev/null +++ b/book/_build/html/chapters/01-setup.html @@ -0,0 +1,654 @@ + + + + + + + + + + + +
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/book/_build/html/chapters/01-tensor.html b/book/_build/html/chapters/01-tensor.html
index 77ab1ec7..1a17fe24 100644
--- a/book/_build/html/chapters/01-tensor.html
+++ b/book/_build/html/chapters/01-tensor.html
@@ -9,7 +9,7 @@
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Setup - TinyTorch System Configuration
+ +
+
+
+
+
+
+
+
+ Contents
+Setup - TinyTorch System Configuration#
+Welcome to TinyTorch! This setup module configures your personal TinyTorch installation and teaches you the NBGrader workflow.
+
+
+π― Learning Goals
+-
+
Configure your personal TinyTorch installation with custom information
+Learn to query system information using Python modules
+Master the NBGrader workflow: implement β test β export
+Create functions that become part of your tinytorch package
+Understand solution blocks, hidden tests, and automated grading
+
The Big Picture: Why Configuration Matters in ML Systems#
+Configuration is the foundation of any production ML system. In this module, youβll learn:
+1. System Awareness#
+Real ML systems need to understand their environment:
+-
+
Hardware constraints: Memory, CPU cores, GPU availability
+Software dependencies: Python version, library compatibility
+Platform differences: Linux servers, macOS development, Windows deployment
+
2. Reproducibility#
+Configuration enables reproducible ML:
+-
+
Environment documentation: Exactly what system was used
+Dependency management: Precise versions and requirements
+Debugging support: System info helps troubleshoot issues
+
3. Professional Development#
+Proper configuration shows engineering maturity:
+-
+
Attribution: Your work is properly credited
+Collaboration: Others can understand and extend your setup
+Maintenance: Systems can be updated and maintained
+
4. ML Systems Context#
+This connects to broader ML engineering:
+-
+
Model deployment: Different environments need different configs
+Monitoring: System metrics help track performance
+Scaling: Understanding hardware helps optimize training
+
Letβs build the foundation of your ML systems engineering skills!
+π Interactive Learning#
+Choose your preferred way to engage with this module:
+
+
+
+
+
+πΎ Save Your Progress
+Binder sessions are temporary! Download your completed notebook when done, or switch to local development for persistent work.
+Ready for serious development? β ποΈ Local Setup Guide
+Foundation
@@ -401,7 +401,7 @@ document.write(`
-
Module 1: Tensor - Core Data Structure
+Tensor - Core Data Structure
@@ -424,8 +424,8 @@ document.write(`
-
-
+
Module 2: Activations - Nonlinearity in Neural Networks — Tinyπ₯Torch: Build ML Systems from Scratch
+ Activations - Nonlinearity in Neural Networks — Tinyπ₯Torch: Build ML Systems from Scratch
@@ -65,8 +65,8 @@ const thebe_selector_output = ".output, .cell_output"
-
-
+
+
@@ -176,7 +176,7 @@ const thebe_selector_output = ".output, .cell_output"
Module 1: Tensor - Core Data Structure#
+Tensor - Core Data Structure#
Welcome to the Tensor module! This is where TinyTorch really begins. Youβll implement the fundamental data structure that powers all ML systems.
π― Learning Goals
@@ -523,7 +523,7 @@ document.write(` diff --git a/book/_build/html/chapters/02-activations.html b/book/_build/html/chapters/02-activations.html index 23d805e5..2cea26b8 100644 --- a/book/_build/html/chapters/02-activations.html +++ b/book/_build/html/chapters/02-activations.html @@ -9,7 +9,7 @@ -Foundation
@@ -401,7 +401,7 @@ document.write(`
-
Module 2: Activations - Nonlinearity in Neural Networks
+Activations - Nonlinearity in Neural Networks
@@ -424,8 +424,8 @@ document.write(`
-
-
+
Tensor - Core Data Structure — Tinyπ₯Torch: Build ML Systems from Scratch
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to main content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Activations - Nonlinearity in Neural Networks — Tinyπ₯Torch: Build ML Systems from Scratch
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to main content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Module 3: Layers - Building Blocks of Neural Networks — Tinyπ₯Torch: Build ML Systems from Scratch
+ Layers - Building Blocks of Neural Networks — Tinyπ₯Torch: Build ML Systems from Scratch
@@ -65,8 +65,8 @@ const thebe_selector_output = ".output, .cell_output"
-
-
+
+
@@ -176,7 +176,7 @@ const thebe_selector_output = ".output, .cell_output"
Module 2: Activations - Nonlinearity in Neural Networks#
+Activations - Nonlinearity in Neural Networks#
Welcome to the Activations module! This is where neural networks get their power through nonlinearity.
π― Learning Goals
@@ -523,7 +523,7 @@ document.write(` diff --git a/book/_build/html/chapters/02-tensor.html b/book/_build/html/chapters/02-tensor.html new file mode 100644 index 00000000..9d1644bd --- /dev/null +++ b/book/_build/html/chapters/02-tensor.html @@ -0,0 +1,609 @@ + + + + + + + + + + + +
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/book/_build/html/chapters/03-activations.html b/book/_build/html/chapters/03-activations.html
new file mode 100644
index 00000000..89c1c657
--- /dev/null
+++ b/book/_build/html/chapters/03-activations.html
@@ -0,0 +1,609 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Tensor - Core Data Structure
+ +
+
+
+
+
+
+
+
+ Contents
+Tensor - Core Data Structure#
+Welcome to the Tensor module! This is where TinyTorch really begins. Youβll implement the fundamental data structure that powers all ML systems.
+
+
+π― Learning Goals
+-
+
Understand tensors as N-dimensional arrays with ML-specific operations
+Implement a complete Tensor class with arithmetic operations
+Handle shape management, data types, and memory layout
+Build the foundation for neural networks and automatic differentiation
+Master the NBGrader workflow with comprehensive testing
+
Build β Use β Understand#
+-
+
Build: Create the Tensor class with core operations
+Use: Perform tensor arithmetic and transformations
+Understand: How tensors form the foundation of ML systems
+
π Interactive Learning#
+Choose your preferred way to engage with this module:
+
+
+
+
+
+πΎ Save Your Progress
+Binder sessions are temporary! Download your completed notebook when done, or switch to local development for persistent work.
+Ready for serious development? β ποΈ Local Setup Guide
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/book/_build/html/chapters/03-layers.html b/book/_build/html/chapters/03-layers.html
index 2d19288d..d3d57c91 100644
--- a/book/_build/html/chapters/03-layers.html
+++ b/book/_build/html/chapters/03-layers.html
@@ -9,7 +9,7 @@
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Activations - Nonlinearity in Neural Networks
+ +
+
+
+
+
+
+
+
+ Contents
+Activations - Nonlinearity in Neural Networks#
+Welcome to the Activations module! This is where neural networks get their power through nonlinearity.
+
+
+π― Learning Goals
+-
+
Understand why activation functions are essential for neural networks
+Implement the four most important activation functions: ReLU, Sigmoid, Tanh, and Softmax
+Visualize how activations transform data and enable complex learning
+See how activations work with layers to build powerful networks
+Master the NBGrader workflow with comprehensive testing
+
Build β Use β Understand#
+-
+
Build: Activation functions that add nonlinearity
+Use: Transform tensors and see immediate results
+Understand: How nonlinearity enables complex pattern learning
+
π Interactive Learning#
+Choose your preferred way to engage with this module:
+
+
+
+
+
+πΎ Save Your Progress
+Binder sessions are temporary! Download your completed notebook when done, or switch to local development for persistent work.
+Ready for serious development? β ποΈ Local Setup Guide
+Foundation
@@ -401,7 +401,7 @@ document.write(`
-
Module 3: Layers - Building Blocks of Neural Networks
+Layers - Building Blocks of Neural Networks
@@ -424,8 +424,8 @@ document.write(`
-
-
+
Layers - Building Blocks of Neural Networks — Tinyπ₯Torch: Build ML Systems from Scratch
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to main content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Module 4: Networks - Neural Network Architectures — Tinyπ₯Torch: Build ML Systems from Scratch
+ Networks - Neural Network Architectures — Tinyπ₯Torch: Build ML Systems from Scratch
@@ -65,8 +65,8 @@ const thebe_selector_output = ".output, .cell_output"
-
-
+
+
@@ -176,7 +176,7 @@ const thebe_selector_output = ".output, .cell_output"
Module 3: Layers - Building Blocks of Neural Networks#
+Layers - Building Blocks of Neural Networks#
Welcome to the Layers module! This is where we build the fundamental components that stack together to form neural networks.
π― Learning Goals
@@ -523,7 +523,7 @@ document.write(` diff --git a/book/_build/html/chapters/04-layers.html b/book/_build/html/chapters/04-layers.html new file mode 100644 index 00000000..9247ce81 --- /dev/null +++ b/book/_build/html/chapters/04-layers.html @@ -0,0 +1,609 @@ + + + + + + + + + + + +
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/book/_build/html/chapters/04-networks.html b/book/_build/html/chapters/04-networks.html
index cf678d9a..93547c1f 100644
--- a/book/_build/html/chapters/04-networks.html
+++ b/book/_build/html/chapters/04-networks.html
@@ -9,7 +9,7 @@
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Layers - Building Blocks of Neural Networks
+ +
+
+
+
+
+
+
+
+ Contents
+Layers - Building Blocks of Neural Networks#
+Welcome to the Layers module! This is where we build the fundamental components that stack together to form neural networks.
+
+
+π― Learning Goals
+-
+
Understand how matrix multiplication powers neural networks
+Implement naive matrix multiplication from scratch for deep understanding
+Build the Dense (Linear) layer - the foundation of all neural networks
+Learn weight initialization strategies and their importance
+See how layers compose with activations to create powerful networks
+
Build β Use β Understand#
+-
+
Build: Matrix multiplication and Dense layers from scratch
+Use: Create and test layers with real data
+Understand: How linear transformations enable feature learning
+
π Interactive Learning#
+Choose your preferred way to engage with this module:
+
+
+
+
+
+πΎ Save Your Progress
+Binder sessions are temporary! Download your completed notebook when done, or switch to local development for persistent work.
+Ready for serious development? β ποΈ Local Setup Guide
+Foundation
@@ -401,7 +401,7 @@ document.write(`
-
Module 4: Networks - Neural Network Architectures
+Networks - Neural Network Architectures
@@ -425,8 +425,8 @@ document.write(`
-
-
+
Module 5: CNN - Convolutional Neural Networks — Tinyπ₯Torch: Build ML Systems from Scratch
+ CNN - Convolutional Neural Networks — Tinyπ₯Torch: Build ML Systems from Scratch
@@ -65,8 +65,8 @@ const thebe_selector_output = ".output, .cell_output"
-
-
+
+
@@ -176,7 +176,7 @@ const thebe_selector_output = ".output, .cell_output"
Module 4: Networks - Neural Network Architectures#
+Networks - Neural Network Architectures#
Welcome to the Networks module! This is where we compose layers into complete neural network architectures.
π― Learning Goals
@@ -535,7 +535,7 @@ document.write(` diff --git a/book/_build/html/chapters/05-cnn.html b/book/_build/html/chapters/05-cnn.html index b2cc7971..d517cd09 100644 --- a/book/_build/html/chapters/05-cnn.html +++ b/book/_build/html/chapters/05-cnn.html @@ -9,7 +9,7 @@ -Foundation
@@ -401,7 +401,7 @@ document.write(`
-
Module 5: CNN - Convolutional Neural Networks
+CNN - Convolutional Neural Networks
@@ -425,8 +425,8 @@ document.write(`
-
-
+
Networks - Neural Network Architectures — Tinyπ₯Torch: Build ML Systems from Scratch
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to main content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ CNN - Convolutional Neural Networks — Tinyπ₯Torch: Build ML Systems from Scratch
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to main content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Module 6: DataLoader - Data Loading and Preprocessing — Tinyπ₯Torch: Build ML Systems from Scratch
+ DataLoader - Data Loading and Preprocessing — Tinyπ₯Torch: Build ML Systems from Scratch
@@ -65,8 +65,8 @@ const thebe_selector_output = ".output, .cell_output"
-
-
+
+
@@ -176,7 +176,7 @@ const thebe_selector_output = ".output, .cell_output"
Module 5: CNN - Convolutional Neural Networks#
+CNN - Convolutional Neural Networks#
Welcome to the CNN module! Here youβll implement the core building block of modern computer vision: the convolutional layer.
π― Learning Goals
@@ -535,7 +535,7 @@ document.write(` diff --git a/book/_build/html/chapters/05-networks.html b/book/_build/html/chapters/05-networks.html new file mode 100644 index 00000000..288da09c --- /dev/null +++ b/book/_build/html/chapters/05-networks.html @@ -0,0 +1,622 @@ + + + + + + + + + + + +
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/book/_build/html/chapters/06-cnn.html b/book/_build/html/chapters/06-cnn.html
new file mode 100644
index 00000000..09b021e5
--- /dev/null
+++ b/book/_build/html/chapters/06-cnn.html
@@ -0,0 +1,622 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Networks - Neural Network Architectures
+ +
+
+
+
+
+
+
+
+ Contents
+Networks - Neural Network Architectures#
+Welcome to the Networks module! This is where we compose layers into complete neural network architectures.
+
+
+π― Learning Goals
+-
+
Understand networks as function composition:
f(x) = layer_n(...layer_2(layer_1(x)))
+Build the Sequential network architecture for composing layers
+Create common network patterns like MLPs (Multi-Layer Perceptrons)
+Visualize network architectures and understand their capabilities
+Master forward pass inference through complete networks
+
Build β Use β Reflect#
+-
+
Build: Sequential networks that compose layers into complete architectures
+Use: Create different network patterns and run inference
+Reflect: How architecture design affects network behavior and capability
+
What Youβll Learn#
+By the end of this module, youβll understand:
+-
+
How simple layers combine to create complex behaviors
+The fundamental Sequential architecture pattern
+How to build MLPs with any number of layers
+Different network architectures (shallow, deep, wide)
+How neural networks approximate complex functions
+
π Interactive Learning#
+Choose your preferred way to engage with this module:
+
+
+
+
+
+πΎ Save Your Progress
+Binder sessions are temporary! Download your completed notebook when done, or switch to local development for persistent work.
+Ready for serious development? β ποΈ Local Setup Guide
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/book/_build/html/chapters/06-dataloader.html b/book/_build/html/chapters/06-dataloader.html
index 9af3c867..88ce20a1 100644
--- a/book/_build/html/chapters/06-dataloader.html
+++ b/book/_build/html/chapters/06-dataloader.html
@@ -9,7 +9,7 @@
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ CNN - Convolutional Neural Networks
+ +
+
+
+
+
+
+
+
+ Contents
+CNN - Convolutional Neural Networks#
+Welcome to the CNN module! Here youβll implement the core building block of modern computer vision: the convolutional layer.
+
+
+π― Learning Goals
+-
+
Understand the convolution operation and its importance in computer vision
+Implement Conv2D with explicit for-loops to understand the sliding window mechanism
+Build convolutional layers that can detect spatial patterns in images
+Compose Conv2D with other layers to build complete convolutional networks
+See how convolution enables parameter sharing and translation invariance
+
Build β Use β Reflect#
+-
+
Build: Conv2D layer using sliding window convolution from scratch
+Use: Transform images and see feature maps emerge
+Reflect: How CNNs learn hierarchical spatial patterns
+
What Youβll Learn#
+By the end of this module, youβll understand:
+-
+
How convolution works as a sliding window operation
+Why convolution is perfect for spatial data like images
+How to build learnable convolutional layers
+The CNN pipeline: Conv2D β Activation β Flatten β Dense
+How parameter sharing makes CNNs efficient
+
π Interactive Learning#
+Choose your preferred way to engage with this module:
+
+
+
+
+
+πΎ Save Your Progress
+Binder sessions are temporary! Download your completed notebook when done, or switch to local development for persistent work.
+Ready for serious development? β ποΈ Local Setup Guide
+Foundation
@@ -401,7 +401,7 @@ document.write(`
-
Module 6: DataLoader - Data Loading and Preprocessing
+DataLoader - Data Loading and Preprocessing
@@ -425,8 +425,8 @@ document.write(`
-
-
+
Module 7: Autograd - Automatic Differentiation Engine — Tinyπ₯Torch: Build ML Systems from Scratch
+ Autograd - Automatic Differentiation Engine — Tinyπ₯Torch: Build ML Systems from Scratch
@@ -65,8 +65,8 @@ const thebe_selector_output = ".output, .cell_output"
-
-
+
+
@@ -176,7 +176,7 @@ const thebe_selector_output = ".output, .cell_output"
Module 6: DataLoader - Data Loading and Preprocessing#
+DataLoader - Data Loading and Preprocessing#
Welcome to the DataLoader module! This is where youβll learn how to efficiently load, process, and manage data for machine learning systems.
π― Learning Goals
@@ -535,7 +535,7 @@ document.write(` diff --git a/book/_build/html/chapters/07-autograd.html b/book/_build/html/chapters/07-autograd.html index 6a54ff2a..e20902bd 100644 --- a/book/_build/html/chapters/07-autograd.html +++ b/book/_build/html/chapters/07-autograd.html @@ -9,7 +9,7 @@ -Foundation
@@ -401,7 +401,7 @@ document.write(`
-
Module 7: Autograd - Automatic Differentiation Engine
+Autograd - Automatic Differentiation Engine
@@ -424,8 +424,8 @@ document.write(`
-
-
+
DataLoader - Data Loading and Preprocessing — Tinyπ₯Torch: Build ML Systems from Scratch
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to main content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Autograd - Automatic Differentiation Engine — Tinyπ₯Torch: Build ML Systems from Scratch
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to main content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Module 8: Optimizers - Gradient-Based Parameter Updates — Tinyπ₯Torch: Build ML Systems from Scratch
+ Optimizers - Gradient-Based Parameter Updates — Tinyπ₯Torch: Build ML Systems from Scratch
@@ -65,8 +65,8 @@ const thebe_selector_output = ".output, .cell_output"
-
-
+
+
@@ -176,7 +176,7 @@ const thebe_selector_output = ".output, .cell_output"
Module 7: Autograd - Automatic Differentiation Engine#
+Autograd - Automatic Differentiation Engine#
Welcome to the Autograd module! This is where TinyTorch becomes truly powerful. Youβll implement the automatic differentiation engine that makes neural network training possible.
π― Learning Goals
@@ -523,7 +523,7 @@ document.write(` diff --git a/book/_build/html/chapters/07-dataloader.html b/book/_build/html/chapters/07-dataloader.html new file mode 100644 index 00000000..84890282 --- /dev/null +++ b/book/_build/html/chapters/07-dataloader.html @@ -0,0 +1,622 @@ + + + + + + + + + + + +
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/book/_build/html/chapters/08-autograd.html b/book/_build/html/chapters/08-autograd.html
new file mode 100644
index 00000000..059ed633
--- /dev/null
+++ b/book/_build/html/chapters/08-autograd.html
@@ -0,0 +1,609 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ DataLoader - Data Loading and Preprocessing
+ +
+
+
+
+
+
+
+
+ Contents
+DataLoader - Data Loading and Preprocessing#
+Welcome to the DataLoader module! This is where youβll learn how to efficiently load, process, and manage data for machine learning systems.
+
+
+π― Learning Goals
+-
+
Understand data pipelines as the foundation of ML systems
+Implement efficient data loading with memory management and batching
+Build reusable dataset abstractions for different data types
+Master the Dataset and DataLoader pattern used in all ML frameworks
+Learn systems thinking for data engineering and I/O optimization
+
Build β Use β Reflect#
+-
+
Build: Create dataset classes and data loaders from scratch
+Use: Load real datasets and feed them to neural networks
+Reflect: How data engineering affects system performance and scalability
+
What Youβll Learn#
+By the end of this module, youβll understand:
+-
+
The Dataset pattern for consistent data access
+How DataLoaders enable efficient batch processing
+Why batching and shuffling are crucial for ML
+How to handle datasets larger than memory
+The connection between data engineering and model performance
+
π Interactive Learning#
+Choose your preferred way to engage with this module:
+
+
+
+
+
+πΎ Save Your Progress
+Binder sessions are temporary! Download your completed notebook when done, or switch to local development for persistent work.
+Ready for serious development? β ποΈ Local Setup Guide
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/book/_build/html/chapters/08-optimizers.html b/book/_build/html/chapters/08-optimizers.html
index a413e0ef..da20e1c9 100644
--- a/book/_build/html/chapters/08-optimizers.html
+++ b/book/_build/html/chapters/08-optimizers.html
@@ -9,7 +9,7 @@
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Autograd - Automatic Differentiation Engine
+ +
+
+
+
+
+
+
+
+ Contents
+Autograd - Automatic Differentiation Engine#
+Welcome to the Autograd module! This is where TinyTorch becomes truly powerful. Youβll implement the automatic differentiation engine that makes neural network training possible.
+
+
+π― Learning Goals
+-
+
Understand how automatic differentiation works through computational graphs
+Implement the Variable class that tracks gradients and operations
+Build backward propagation for gradient computation
+Create the foundation for neural network training
+Master the mathematical concepts behind backpropagation
+
Build β Use β Analyze#
+-
+
Build: Create the Variable class and gradient computation system
+Use: Perform automatic differentiation on complex expressions
+Analyze: Understand how gradients flow through computational graphs
+
π Interactive Learning#
+Choose your preferred way to engage with this module:
+
+
+
+
+
+πΎ Save Your Progress
+Binder sessions are temporary! Download your completed notebook when done, or switch to local development for persistent work.
+Ready for serious development? β ποΈ Local Setup Guide
+Foundation
@@ -401,7 +401,7 @@ document.write(`
-
Module 8: Optimizers - Gradient-Based Parameter Updates
+Optimizers - Gradient-Based Parameter Updates
@@ -424,8 +424,8 @@ document.write(`
-
-
+
Optimizers - Gradient-Based Parameter Updates — Tinyπ₯Torch: Build ML Systems from Scratch
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to main content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Module 9: Training - Complete Neural Network Training Pipeline — Tinyπ₯Torch: Build ML Systems from Scratch
+ Training - Complete Neural Network Training Pipeline — Tinyπ₯Torch: Build ML Systems from Scratch
@@ -65,8 +65,8 @@ const thebe_selector_output = ".output, .cell_output"
-
-
+
+
@@ -176,7 +176,7 @@ const thebe_selector_output = ".output, .cell_output"
Module 8: Optimizers - Gradient-Based Parameter Updates#
+Optimizers - Gradient-Based Parameter Updates#
Welcome to the Optimizers module! This is where neural networks learn to improve through intelligent parameter updates.
π― Learning Goals
@@ -523,7 +523,7 @@ document.write(` diff --git a/book/_build/html/chapters/09-optimizers.html b/book/_build/html/chapters/09-optimizers.html new file mode 100644 index 00000000..026e2ecf --- /dev/null +++ b/book/_build/html/chapters/09-optimizers.html @@ -0,0 +1,609 @@ + + + + + + + + + + + +
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/book/_build/html/chapters/09-training.html b/book/_build/html/chapters/09-training.html
index c137e418..8e78b80a 100644
--- a/book/_build/html/chapters/09-training.html
+++ b/book/_build/html/chapters/09-training.html
@@ -9,7 +9,7 @@
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Optimizers - Gradient-Based Parameter Updates
+ +
+
+
+
+
+
+
+
+ Contents
+Optimizers - Gradient-Based Parameter Updates#
+Welcome to the Optimizers module! This is where neural networks learn to improve through intelligent parameter updates.
+
+
+π― Learning Goals
+-
+
Understand gradient descent and how optimizers use gradients to update parameters
+Implement SGD with momentum for accelerated convergence
+Build Adam optimizer with adaptive learning rates
+Master learning rate scheduling strategies
+See how optimizers enable effective neural network training
+
Build β Use β Analyze#
+-
+
Build: Core optimization algorithms (SGD, Adam)
+Use: Apply optimizers to train neural networks
+Analyze: Compare optimizer behavior and convergence patterns
+
π Interactive Learning#
+Choose your preferred way to engage with this module:
+
+
+
+
+
+πΎ Save Your Progress
+Binder sessions are temporary! Download your completed notebook when done, or switch to local development for persistent work.
+Ready for serious development? β ποΈ Local Setup Guide
+Foundation
@@ -401,7 +401,7 @@ document.write(`
-
Module 9: Training - Complete Neural Network Training Pipeline
+Training - Complete Neural Network Training Pipeline
@@ -424,8 +424,8 @@ document.write(`
-
-
+
Module 10: Compression & Optimization - Making AI Models Efficient — Tinyπ₯Torch: Build ML Systems from Scratch
+ Compression & Optimization - Making AI Models Efficient — Tinyπ₯Torch: Build ML Systems from Scratch
@@ -65,8 +65,8 @@ const thebe_selector_output = ".output, .cell_output"
-
-
+
+
@@ -176,7 +176,7 @@ const thebe_selector_output = ".output, .cell_output"
Module 9: Training - Complete Neural Network Training Pipeline#
+Training - Complete Neural Network Training Pipeline#
Welcome to the Training module! This is where we bring everything together to train neural networks on real data.
π― Learning Goals
@@ -523,7 +523,7 @@ document.write(` diff --git a/book/_build/html/chapters/10-compression.html b/book/_build/html/chapters/10-compression.html index fe30d03c..fb4020cb 100644 --- a/book/_build/html/chapters/10-compression.html +++ b/book/_build/html/chapters/10-compression.html @@ -9,7 +9,7 @@ -Foundation
@@ -401,7 +401,7 @@ document.write(`
-
Module 10: Compression & Optimization - Making AI Models Efficient
+Compression & Optimization - Making AI Models Efficient
@@ -424,8 +424,8 @@ document.write(`
-
-
+
Training - Complete Neural Network Training Pipeline — Tinyπ₯Torch: Build ML Systems from Scratch
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to main content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Compression & Optimization - Making AI Models Efficient — Tinyπ₯Torch: Build ML Systems from Scratch
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to main content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Module 11: Kernels - Hardware-Optimized ML Operations — Tinyπ₯Torch: Build ML Systems from Scratch
+ Kernels - Hardware-Optimized ML Operations — Tinyπ₯Torch: Build ML Systems from Scratch
@@ -65,8 +65,8 @@ const thebe_selector_output = ".output, .cell_output"
-
-
+
+
@@ -176,7 +176,7 @@ const thebe_selector_output = ".output, .cell_output"
Module 10: Compression & Optimization - Making AI Models Efficient#
+Compression & Optimization - Making AI Models Efficient#
Welcome to the Compression module! This is where youβll learn to make neural networks smaller, faster, and more efficient for real-world deployment.
π― Learning Goals
@@ -524,7 +524,7 @@ document.write(` diff --git a/book/_build/html/chapters/10-training.html b/book/_build/html/chapters/10-training.html new file mode 100644 index 00000000..966eb5eb --- /dev/null +++ b/book/_build/html/chapters/10-training.html @@ -0,0 +1,609 @@ + + + + + + + + + + + +
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/book/_build/html/chapters/11-compression.html b/book/_build/html/chapters/11-compression.html
new file mode 100644
index 00000000..7faf518d
--- /dev/null
+++ b/book/_build/html/chapters/11-compression.html
@@ -0,0 +1,610 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Training - Complete Neural Network Training Pipeline
+ +
+
+
+
+
+
+
+
+ Contents
+Training - Complete Neural Network Training Pipeline#
+Welcome to the Training module! This is where we bring everything together to train neural networks on real data.
+
+
+π― Learning Goals
+-
+
Understand loss functions and how they measure model performance
+Implement essential loss functions: MSE, CrossEntropy, and BinaryCrossEntropy
+Build evaluation metrics for classification and regression
+Create a complete training loop that orchestrates the entire process
+Master checkpointing and model persistence for real-world deployment
+
Build β Use β Optimize#
+-
+
Build: Loss functions, metrics, and training orchestration
+Use: Train complete models on real datasets
+Optimize: Analyze training dynamics and improve performance
+
π Interactive Learning#
+Choose your preferred way to engage with this module:
+
+
+
+
+
+πΎ Save Your Progress
+Binder sessions are temporary! Download your completed notebook when done, or switch to local development for persistent work.
+Ready for serious development? β ποΈ Local Setup Guide
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/book/_build/html/chapters/11-kernels.html b/book/_build/html/chapters/11-kernels.html
index 0755d647..0618a6d5 100644
--- a/book/_build/html/chapters/11-kernels.html
+++ b/book/_build/html/chapters/11-kernels.html
@@ -9,7 +9,7 @@
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Compression & Optimization - Making AI Models Efficient
+ +
+
+
+
+
+
+
+
+ Contents
+Compression & Optimization - Making AI Models Efficient#
+Welcome to the Compression module! This is where youβll learn to make neural networks smaller, faster, and more efficient for real-world deployment.
+
+
+π― Learning Goals
+-
+
Understand how model size affects deployment and why compression matters
+Implement magnitude-based pruning to remove unimportant weights
+Master quantization to reduce memory usage by 75%
+Build knowledge distillation for training compact models
+Create structured pruning to optimize network architectures
+Compare compression techniques and their trade-offs
+
Build β Use β Optimize#
+-
+
Build: Four compression techniques from scratch
+Use: Apply compression to real neural networks
+Optimize: Combine techniques for maximum efficiency gains
+
π Interactive Learning#
+Choose your preferred way to engage with this module:
+
+
+
+
+
+πΎ Save Your Progress
+Binder sessions are temporary! Download your completed notebook when done, or switch to local development for persistent work.
+Ready for serious development? β ποΈ Local Setup Guide
+Foundation
@@ -401,7 +401,7 @@ document.write(`
-
Module 11: Kernels - Hardware-Optimized ML Operations
+Kernels - Hardware-Optimized ML Operations
@@ -424,8 +424,8 @@ document.write(`
-
-
+
Module 12: Benchmarking - Systematic ML Performance Evaluation — Tinyπ₯Torch: Build ML Systems from Scratch
+ Benchmarking - Systematic ML Performance Evaluation — Tinyπ₯Torch: Build ML Systems from Scratch
@@ -65,8 +65,8 @@ const thebe_selector_output = ".output, .cell_output"
-
-
+
+
@@ -176,7 +176,7 @@ const thebe_selector_output = ".output, .cell_output"
Module 11: Kernels - Hardware-Optimized ML Operations#
+Kernels - Hardware-Optimized ML Operations#
Welcome to the Kernels module! This is where we move beyond NumPy to understand how ML operations are optimized for modern hardware. Youβll implement custom kernels that run faster than standard library functions.
π― Learning Goals
@@ -524,7 +524,7 @@ document.write(` diff --git a/book/_build/html/chapters/12-benchmarking.html b/book/_build/html/chapters/12-benchmarking.html index 24b1039c..29e60bf9 100644 --- a/book/_build/html/chapters/12-benchmarking.html +++ b/book/_build/html/chapters/12-benchmarking.html @@ -9,7 +9,7 @@ -Foundation
@@ -401,7 +401,7 @@ document.write(`
-
Module 12: Benchmarking - Systematic ML Performance Evaluation
+Benchmarking - Systematic ML Performance Evaluation
@@ -424,8 +424,8 @@ document.write(`
-
-
+
Kernels - Hardware-Optimized ML Operations — Tinyπ₯Torch: Build ML Systems from Scratch
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to main content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Benchmarking - Systematic ML Performance Evaluation — Tinyπ₯Torch: Build ML Systems from Scratch
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to main content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Module 13: MLOps - Production ML Systems — Tinyπ₯Torch: Build ML Systems from Scratch
+ MLOps - Production ML Systems — Tinyπ₯Torch: Build ML Systems from Scratch
@@ -65,7 +65,7 @@ const thebe_selector_output = ".output, .cell_output"
-
+
@@ -175,7 +175,7 @@ const thebe_selector_output = ".output, .cell_output"
-
+
π Ready to Start?
ποΈ Big Picture: Why Build from Scratch?
diff --git a/book/_build/html/objects.inv b/book/_build/html/objects.inv
index 48c6dd1a..3328b196 100644
Binary files a/book/_build/html/objects.inv and b/book/_build/html/objects.inv differ
diff --git a/book/_build/html/search.html b/book/_build/html/search.html
index 2339d5b5..0c79fae7 100644
--- a/book/_build/html/search.html
+++ b/book/_build/html/search.html
@@ -175,29 +175,29 @@ const thebe_selector_output = ".output, .cell_output"
diff --git a/book/_build/html/searchindex.js b/book/_build/html/searchindex.js
index 66f9f587..48a2e128 100644
--- a/book/_build/html/searchindex.js
+++ b/book/_build/html/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"docnames": ["chapters/00-setup", "chapters/01-tensor", "chapters/02-activations", "chapters/03-layers", "chapters/04-networks", "chapters/05-cnn", "chapters/06-dataloader", "chapters/07-autograd", "chapters/08-optimizers", "chapters/09-training", "chapters/10-compression", "chapters/11-kernels", "chapters/12-benchmarking", "chapters/13-mlops", "intro", "usage-paths/classroom-use", "usage-paths/quick-exploration", "usage-paths/serious-development"], "filenames": ["chapters/00-setup.md", "chapters/01-tensor.md", "chapters/02-activations.md", "chapters/03-layers.md", "chapters/04-networks.md", "chapters/05-cnn.md", "chapters/06-dataloader.md", "chapters/07-autograd.md", "chapters/08-optimizers.md", "chapters/09-training.md", "chapters/10-compression.md", "chapters/11-kernels.md", "chapters/12-benchmarking.md", "chapters/13-mlops.md", "intro.md", "usage-paths/classroom-use.md", "usage-paths/quick-exploration.md", "usage-paths/serious-development.md"], "titles": ["Module 0: Setup - TinyTorch System Configuration", "Module 1: Tensor - Core Data Structure", "Module 2: Activations - Nonlinearity in Neural Networks", "Module 3: Layers - Building Blocks of Neural Networks", "Module 4: Networks - Neural Network Architectures", "Module 5: CNN - Convolutional Neural Networks", "Module 6: DataLoader - Data Loading and Preprocessing", "Module 7: Autograd - Automatic Differentiation Engine", "Module 8: Optimizers - Gradient-Based Parameter Updates", "Module 9: Training - Complete Neural Network Training Pipeline", "Module 10: Compression & Optimization - Making AI Models Efficient", "Module 11: Kernels - Hardware-Optimized ML Operations", "Module 12: Benchmarking - Systematic ML Performance Evaluation", "Module 13: MLOps - Production ML Systems", "Tiny\ud83d\udd25Torch: Build Machine Learning Systems from Scratch", "\ud83d\udc68\u200d\ud83c\udfeb Classroom Use Guide", "\ud83d\udd2c Quick Exploration Path", "\ud83c\udfd7\ufe0f Serious Development Path"], "terms": {"welcom": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "thi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "person": [0, 14], "instal": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], "teach": [0, 14, 15, 16], "you": [0, 1, 7, 10, 11], "nbgrader": [0, 1, 2, 14, 17], "workflow": [0, 1, 2, 13, 14, 16], "custom": [0, 11, 14, 17], "inform": 0, "queri": 0, "us": 0, "python": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17], "master": [0, 1, 2, 4, 6, 7, 8, 9, 10, 11], "implement": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], "test": [0, 1, 2, 3, 14], "export": [0, 15], "creat": [0, 1, 3, 4, 6, 7, 9, 10, 11, 12, 13, 14, 15, 16, 17], "function": [0, 2, 4, 9, 11, 15, 16, 17], "becom": [0, 7, 14, 16], "part": [0, 14], "packag": [0, 17], "understand": [0, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15], "solut": [0, 14], "block": [0, 5, 16], "hidden": 0, "autom": [0, 13, 14, 17], "grade": [0, 14, 17], "foundat": [0, 1, 3, 6, 7], "ani": [0, 4, 16], "product": 0, "In": 0, "ll": [0, 1, 7, 10, 11], "real": [0, 3, 6, 9, 10, 11, 15, 16], "need": [0, 14, 16], "environ": [0, 14, 15, 16], "hardwar": [0, 15, 17], "constraint": 0, "memori": [0, 1, 6, 10, 11, 15, 17], "cpu": [0, 11], "core": [0, 5, 8, 14, 15, 17], "gpu": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "avail": 0, "softwar": 0, "depend": [0, 17], "version": [0, 15], "librari": [0, 11, 15, 17], "compat": 0, "platform": 0, "differ": [0, 4, 6, 12], "linux": 0, "server": 0, "maco": 0, "window": [0, 5], "deploy": [0, 9, 10, 13, 14, 15, 17], "enabl": [0, 2, 3, 5, 6, 8, 14, 17], "document": [0, 15], "exactli": 0, "what": 0, "wa": 0, "manag": [0, 1, 6, 13, 17], "precis": 0, "requir": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], "debug": 0, "support": [0, 17], "info": [0, 15, 17], "help": 0, "troubleshoot": 0, "issu": [0, 13, 17], "proper": [0, 12], "show": [0, 17], "engin": [0, 6, 14, 15, 17], "matur": 0, "attribut": 0, "work": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16], "properli": 0, "credit": 0, "collabor": 0, "other": [0, 5, 14], "can": [0, 5, 14, 15, 16], "extend": [0, 15], "mainten": [0, 13], "updat": [0, 17], "maintain": [0, 13], "connect": [0, 6], "broader": 0, "model": [0, 6, 9, 11, 12, 13, 14, 15, 17], "config": 0, "monitor": [0, 13, 14, 17], "metric": [0, 9, 14, 15, 17], "track": [0, 7, 17], "perform": [0, 1, 6, 7, 9, 11, 13], "scale": 0, "optim": [0, 6, 14, 15, 17], "train": [0, 7, 8, 10, 13], "let": 0, "s": [0, 14], "build": 0, "skill": 0, "choos": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "prefer": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "wai": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "engag": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "launch": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14], "binder": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "run": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "browser": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "No": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "open": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17], "colab": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "googl": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "access": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "cloud": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "comput": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17], "power": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "view": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "sourc": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17], "brows": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15], "code": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16], "session": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16], "ar": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "temporari": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "download": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "complet": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 16, 17], "notebook": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17], "when": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "done": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "switch": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "local": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "persist": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17], "readi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16], "seriou": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "guid": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "where": [1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13], "tinytorch": [1, 7, 12, 13, 15, 16, 17], "realli": 1, "begin": 1, "fundament": [1, 3, 4], "all": [1, 3, 6, 13, 14, 15, 16, 17], "ml": [1, 6, 14, 15, 16, 17], "system": [1, 6, 7, 12, 16], "n": 1, "dimension": [1, 14, 15, 17], "arrai": [1, 14, 15, 17], "specif": [1, 15, 17], "oper": [1, 5, 7, 14, 15, 16, 17], "class": [1, 6, 7, 14, 15, 16, 17], "arithmet": 1, "handl": [1, 6, 14, 17], "shape": 1, "type": [1, 6, 17], "layout": 1, "neural": [1, 6, 7, 8, 10, 14, 15, 16, 17], "network": [1, 6, 7, 8, 10, 14, 15, 16, 17], "automat": [1, 13, 14, 15, 17], "differenti": [1, 14, 15, 17], "comprehens": [1, 2, 14, 15, 17], "transform": [1, 2, 3, 5, 15, 16], "how": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 16, 17], "form": [1, 3], "develop": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16], "setup": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14], "get": [2, 14, 16], "through": [2, 4, 7, 8, 15], "why": [2, 5, 6, 10, 11, 13, 16], "essenti": [2, 9], "four": [2, 10, 12], "most": [2, 14, 15, 17], "import": [2, 3, 5, 14, 17], "relu": [2, 14, 16, 17], "sigmoid": [2, 14, 17], "tanh": [2, 14, 17], "softmax": [2, 17], "visual": [2, 4, 14, 15, 16, 17], "data": [2, 3, 5, 9, 15, 16], "complex": [2, 4, 7], "see": [2, 3, 5, 8, 13, 14, 16, 17], "layer": [2, 4, 5, 14, 15, 17], "add": [2, 15], "tensor": [2, 14, 15, 16, 17], "immedi": [2, 15], "result": [2, 12, 16], "pattern": [2, 4, 5, 6, 8, 11, 14, 15, 17], "we": [3, 4, 9, 11, 12, 13], "compon": [3, 12, 13, 14, 15, 16, 17], "stack": [3, 17], "togeth": [3, 9, 13, 16, 17], "matrix": [3, 14, 15, 16, 17], "multipl": [3, 14, 15, 16, 17], "naiv": 3, "from": [3, 5, 6, 10, 15, 16, 17], "scratch": [3, 5, 6, 10, 15, 16, 17], "deep": [3, 4, 15, 17], "dens": [3, 5, 14, 15, 16, 17], "linear": [3, 14, 15, 17], "weight": [3, 10, 15, 17], "initi": [3, 17], "strategi": [3, 8, 17], "compos": [3, 4, 5], "activ": [3, 5, 15, 17], "featur": [3, 5, 17], "composit": [4, 15, 17], "f": 4, "x": [4, 14], "layer_n": 4, "layer_2": 4, "layer_1": 4, "sequenti": [4, 14, 15, 17], "common": [4, 12], "like": [4, 5, 14], "mlp": [4, 15], "multi": [4, 14, 15, 17], "perceptron": [4, 17], "capabl": 4, "forward": [4, 14, 15, 17], "pass": [4, 15], "infer": 4, "design": 4, "affect": [4, 6, 10], "behavior": [4, 8], "By": [4, 5, 6, 14], "end": [4, 5, 6, 14, 16, 17], "simpl": [4, 14, 16], "combin": [4, 10, 15, 16], "The": [4, 5, 6, 16], "number": 4, "shallow": 4, "wide": 4, "approxim": 4, "here": 5, "modern": [5, 11, 14], "vision": [5, 14, 15, 17], "its": 5, "conv2d": [5, 15], "explicit": 5, "loop": [5, 9, 13, 15, 17], "slide": [5, 15], "mechan": 5, "detect": [5, 13], "spatial": [5, 15, 17], "imag": [5, 14, 15, 16, 17], "paramet": 5, "share": 5, "translat": 5, "invari": 5, "map": 5, "emerg": 5, "hierarch": 5, "perfect": [5, 14, 15, 16, 17], "learnabl": 5, "pipelin": [5, 6, 13, 14, 15, 17], "flatten": 5, "make": [5, 7, 15, 17], "effici": [5, 6, 11, 17], "process": [6, 9, 11, 15, 17], "machin": 6, "batch": [6, 14, 15, 17], "reusabl": 6, "dataset": [6, 9, 14, 15, 17], "abstract": [6, 14], "framework": [6, 12, 14, 15, 16, 17], "think": 6, "i": [6, 14, 16, 17], "o": 6, "loader": 6, "feed": 6, "them": [6, 14], "scalabl": 6, "consist": [6, 14, 17], "shuffl": [6, 17], "crucial": [6, 16], "larger": [6, 14, 15], "than": [6, 11, 14, 15, 17], "between": [6, 16], "truli": 7, "possibl": [7, 14, 17], "graph": [7, 15, 17], "variabl": 7, "gradient": [7, 15, 17], "backward": 7, "propag": [7, 15, 17], "mathemat": [7, 15, 17], "concept": 7, "behind": [7, 14], "backpropag": [7, 14, 15, 17], "express": 7, "flow": 7, "improv": [8, 9, 11, 17], "intellig": 8, "descent": [8, 17], "sgd": [8, 14, 15, 17], "momentum": 8, "acceler": 8, "converg": [8, 15, 17], "adam": [8, 14, 15, 17], "adapt": 8, "rate": [8, 14, 15, 17], "schedul": [8, 14, 15, 17], "effect": 8, "algorithm": [8, 11, 15], "appli": [8, 10, 11, 12], "compar": [8, 10], "bring": [9, 14], "everyth": [9, 15, 17], "loss": [9, 14, 15, 17], "thei": [9, 14], "measur": [9, 11, 14, 15, 17], "mse": 9, "crossentropi": 9, "binarycrossentropi": 9, "evalu": [9, 14, 15, 17], "classif": [9, 16, 17], "regress": 9, "orchestr": [9, 14, 15], "entir": 9, "checkpoint": [9, 17], "world": [9, 10, 14, 15, 17], "analyz": 9, "dynam": [9, 17], "smaller": 10, "faster": [10, 11], "more": [10, 16], "size": [10, 14, 15, 17], "matter": [10, 11, 16], "magnitud": 10, "base": [10, 15], "prune": [10, 14, 15, 17], "remov": 10, "unimport": 10, "quantiz": [10, 14, 15, 17], "reduc": [10, 17], "usag": 10, "75": [10, 14, 15, 17], "knowledg": 10, "distil": 10, "compact": 10, "structur": 10, "architectur": [10, 12, 14, 15, 17], "techniqu": 10, "trade": 10, "off": 10, "maximum": 10, "gain": [10, 16], "move": 11, "beyond": 11, "numpi": [11, 14], "standard": [11, 12, 15], "vector": 11, "simd": 11, "principl": 11, "better": [11, 14, 15, 17], "cach": 11, "util": 11, "parallel": 11, "style": [11, 14, 15, 17], "profil": [11, 15], "tool": [11, 14, 15], "compress": [11, 14, 15, 17], "workload": 11, "systemat": [11, 14, 15, 17], "industri": [12, 14, 15], "methodolog": 12, "inspir": 12, "mlperf": [12, 15, 17], "scenario": 12, "latenc": 12, "throughput": 12, "offlin": 12, "statist": [12, 15, 17], "valid": [12, 14, 15, 17], "meaning": 12, "profession": [12, 14, 15, 17], "report": [12, 15, 17], "project": [12, 15, 16], "avoid": 12, "pitfal": 12, "gener": [12, 17], "confid": 12, "close": 13, "lifecycl": 13, "degrad": 13, "over": 13, "time": 13, "without": [13, 16], "drift": 13, "retrain": 13, "trigger": 13, "comparison": 13, "infrastructur": [13, 14, 17], "respond": 13, "themselv": 13, "own": [14, 16, 17], "pytorch": [14, 17], "ground": 14, "up": 14, "cours": 14, "have": [14, 16, 17], "built": [14, 15, 17], "cifar": [14, 15, 17], "magic": [14, 17], "reduct": [14, 15], "load": [14, 15, 17], "importantli": 14, "actual": 14, "under": [14, 16], "hood": [14, 16], "explor": 14, "minut": [14, 16], "want": [14, 16, 17], "about": [14, 16], "click": [14, 16], "interact": 14, "feel": 14, "8": [14, 15], "week": 14, "myself": [14, 17], "fork": [14, 16], "repo": [14, 16], "full": [14, 15], "tito": [14, 15], "cli": [14, 15], "14": 14, "assign": [14, 17], "mlop": [14, 15, 17], "classroom": [14, 16], "instructor": 14, "integr": [14, 15, 17], "200": [14, 15, 17], "across": [14, 15, 17], "flexibl": 14, "pace": 14, "16": [14, 15], "pedagog": 14, "turn": [14, 15], "kei": [14, 15], "educ": [14, 16], "1": 14, "math": [14, 16], "4": 14, "chain": 14, "cnn": [14, 17], "convolut": [14, 15, 17], "7": 14, "dataload": [14, 17], "preprocess": [14, 15, 17], "autograd": [14, 15, 17], "11": [14, 17], "kernel": [14, 17], "high": [14, 15, 16, 17], "benchmark": [14, 15, 17], "continu": [14, 15, 17], "def": 14, "return": 14, "should": 14, "output": 14, "input_tensor": 14, "784": 14, "128": 14, "repeat": 14, "everi": [14, 17], "fit": [14, 16], "after": [14, 15, 16, 17], "95": [14, 15], "vs": [14, 16], "20": [14, 15, 16], "befor": 14, "90": [14, 15], "deepli": [14, 15, 17], "15": 14, "85": [14, 15], "80": [14, 15], "user": [14, 15], "pursu": [14, 15], "advanc": [14, 15], "role": [14, 15], "30": [14, 15], "graduat": [14, 15], "our": [14, 15], "codebas": [14, 15], "know": 14, "happen": 14, "academ": [14, 15], "successfulli": 14, "univers": 14, "toi": [14, 17], "organ": [14, 17], "consider": [14, 17], "practic": [14, 15, 17], "indic": [14, 17], "success": [14, 17], "messag": [14, 17], "ensur": [14, 15, 17], "aha": 14, "moment": 14, "hello_world": 14, "next": 14, "deploi": [14, 17], "gap": 14, "step": 14, "carefulli": 14, "scaffold": 14, "just": [14, 16], "second": 14, "first": [14, 16, 17], "tradit": 14, "approach": [14, 16], "nn": 14, "mseloss": 14, "doe": [14, 15], "come": 14, "hand": [14, 16], "line": 14, "write": 14, "closer": 14, "ai": [14, 16], "provid": [15, 17], "progress": 15, "00": [15, 17], "13": [15, 17], "correctli": 15, "durat": 15, "git": [15, 17], "http": [15, 17], "github": [15, 17], "com": [15, 17], "org": [15, 17], "cd": [15, 17], "bin": [15, 17], "sh": [15, 17], "doctor": [15, 17], "verifi": [15, 17], "init": 15, "modul": 15, "00_setup": [15, 17], "releas": [15, 17], "ls": 15, "simul": 15, "submiss": [15, 17], "collect": 15, "goal": 15, "qualiti": 15, "assur": 15, "01_tensor": 15, "02_activ": 15, "nonlinear": [15, 16, 17], "25": 15, "03_layer": 15, "5": [15, 16], "04_network": 15, "35": 15, "construct": 15, "05_cnn": 15, "06_dataload": 15, "07_autograd": 15, "40": 15, "9": 15, "08_optim": 15, "analysi": 15, "09_train": 15, "10_compress": 15, "target": 15, "12": [15, 17], "11_kernel": 15, "12_benchmark": 15, "13_mlop": 15, "semest": 15, "Or": 15, "onc": 15, "rang": 15, "03": [15, 17], "auto": [15, 17], "detail": 15, "gradebook": 15, "check": [15, 17], "overal": 15, "verbos": 15, "analyt": 15, "unit": 15, "correct": [15, 17], "cross": [15, 17], "enforc": 15, "145": 15, "135": 15, "total": 15, "445": 15, "60": 15, "do": 15, "clear": 15, "comment": 15, "docstr": 15, "intens": 15, "per": 15, "dive": 15, "research": 15, "competit": 15, "leaderboard": 15, "challeng": 15, "novel": 15, "beginn": 15, "friendli": 15, "extra": 15, "tutori": 15, "earli": 15, "skip": 15, "basic": 15, "focu": 15, "overview": 15, "expect": 15, "nbgrader_config": 15, "py": [15, 17], "modifi": [15, 16], "length": 15, "institut": 15, "ongo": [15, 16], "direct": [15, 17], "commun": 15, "privat": 15, "slack": 15, "discord": 15, "lectur": 15, "note": 15, "offic": 15, "hour": 15, "weekli": 15, "virtual": [15, 17], "ever": 15, "take": 15, "anyth": 16, "button": 16, "live": 16, "jupyt": [16, 17], "your": 16, "pre": 16, "configur": 16, "abil": 16, "account": 16, "creation": 16, "now": [16, 17], "familiar": 16, "timeout": 16, "inact": 16, "save": 16, "great": 16, "For": 16, "relationship": 16, "learn": 16, "level": 16, "api": 16, "satisfi": 16, "ve": 16, "valuabl": 16, "insight": 16, "commit": 16, "my": 17, "A": 17, "includ": 17, "10": 17, "clone": 17, "lab": 17, "setup_dev": 17, "c": 17, "hello_tinytorch": 17, "name": 17, "01": 17, "02": 17, "04": 17, "05": 17, "06": 17, "07": 17, "08": 17, "variant": 17, "09": 17, "awar": 17, "intern": 17, "health": 17, "statu": 17, "dev": 17, "100": 17, "inlin": 17, "classifi": 17, "reliabl": 17, "coverag": 17, "follow": 17, "error": 17, "guidanc": 17, "experi": 17, "new": 17, "jump": 17, "try": 17, "discuss": 17, "question": 17, "bug": 17, "suggest": 17, "contact": 17, "team": 17, "self": 17, "awai": 17}, "objects": {}, "objtypes": {}, "objnames": {}, "titleterms": {"modul": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17], "0": [0, 14, 16], "setup": [0, 15, 16, 17], "tinytorch": [0, 14], "system": [0, 13, 14, 15, 17], "configur": 0, "learn": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17], "goal": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "The": [0, 17], "big": [0, 14], "pictur": [0, 14], "why": [0, 14, 17], "matter": 0, "ml": [0, 11, 12, 13], "1": [0, 1, 15, 17], "awar": 0, "2": [0, 2, 14, 15, 16, 17], "reproduc": 0, "3": [0, 3, 14, 15, 16, 17], "profession": 0, "develop": [0, 17], "4": [0, 4, 15, 17], "context": 0, "interact": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "save": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "your": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17], "progress": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17], "tensor": 1, "core": 1, "data": [1, 6, 14, 17], "structur": [1, 15], "build": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], "us": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17], "understand": [1, 2, 3, 14, 16, 17], "activ": [2, 14, 16], "nonlinear": 2, "neural": [2, 3, 4, 5, 9], "network": [2, 3, 4, 5, 9], "layer": [3, 16], "block": [3, 14, 15, 17], "architectur": 4, "reflect": [4, 5, 6], "what": [4, 5, 6, 14, 16, 17], "you": [4, 5, 6, 14, 16, 17], "ll": [4, 5, 6, 14, 16, 17], "5": [5, 14, 17], "cnn": 5, "convolut": 5, "6": [6, 14, 15, 17], "dataload": 6, "load": 6, "preprocess": 6, "7": [7, 15, 17], "autograd": 7, "automat": 7, "differenti": 7, "engin": 7, "analyz": [7, 8, 12], "8": [8, 17], "optim": [8, 9, 10, 11], "gradient": 8, "base": 8, "paramet": 8, "updat": 8, "9": [9, 14], "train": [9, 14, 15, 17], "complet": [9, 15], "pipelin": 9, "10": [10, 14, 15], "compress": 10, "make": [10, 14], "ai": 10, "model": 10, "effici": 10, "11": [11, 15], "kernel": 11, "hardwar": 11, "oper": 11, "12": 12, "benchmark": 12, "systemat": 12, "perform": [12, 14, 15, 17], "evalu": 12, "13": [13, 14], "mlop": 13, "product": [13, 14, 15, 17], "deploi": 13, "tini": 14, "torch": 14, "machin": 14, "from": 14, "scratch": 14, "achiev": 14, "choos": [14, 17], "path": [14, 16, 17], "three": 14, "wai": 14, "engag": 14, "complex": [14, 17], "each": 14, "previou": 14, "work": [14, 17], "foundat": [14, 15, 17], "philosophi": 14, "exampl": 14, "how": 14, "function": 14, "proven": [14, 15, 17], "student": [14, 15, 17], "outcom": [14, 15, 17], "real": [14, 17], "result": [14, 15, 17], "thi": [14, 17], "differ": 14, "immedi": [14, 16, 17], "feedback": [14, 15, 17], "readi": [14, 17], "start": [14, 15, 16, 17], "adventur": 14, "quick": [14, 15, 16, 17], "tast": 14, "try": [14, 16], "chapter": [14, 16], "right": 14, "now": 14, "classroom": 15, "guid": 15, "cours": 15, "infrastructur": 15, "instructor": 15, "step": [15, 16, 17], "clone": 15, "initi": 15, "nbgrader": 15, "gener": 15, "first": 15, "assign": 15, "test": [15, 17], "workflow": [15, 17], "pace": 15, "week": [15, 17], "14": 15, "manag": 15, "grade": 15, "statu": 15, "monitor": 15, "assess": 15, "autom": 15, "point": 15, "distribut": 15, "suggest": 15, "rubric": 15, "pedagog": 15, "custom": 15, "option": 15, "flexibl": 15, "variat": 15, "prerequisit": 15, "adjust": 15, "get": [15, 17], "review": 15, "materi": 15, "need": 15, "launch": [15, 16], "support": 15, "explor": 16, "requir": 16, "experi": 16, "recommend": 16, "here": 16, "Then": 16, "up": 16, "import": 16, "limit": 16, "next": 16, "seriou": 17, "minut": 17, "code": 17, "environ": 17, "export": 17, "tito": 17, "cli": 17, "qualiti": 17, "assur": 17, "approach": 17, "help": 17}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 6, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.intersphinx": 1, "sphinxcontrib.bibtex": 9, "sphinx": 56}})
\ No newline at end of file
+Search.setIndex({"docnames": ["chapters/01-setup", "chapters/02-tensor", "chapters/03-activations", "chapters/04-layers", "chapters/05-networks", "chapters/06-cnn", "chapters/07-dataloader", "chapters/08-autograd", "chapters/09-optimizers", "chapters/10-training", "chapters/11-compression", "chapters/12-kernels", "chapters/13-benchmarking", "chapters/14-mlops", "intro", "usage-paths/classroom-use", "usage-paths/quick-exploration", "usage-paths/serious-development"], "filenames": ["chapters/01-setup.md", "chapters/02-tensor.md", "chapters/03-activations.md", "chapters/04-layers.md", "chapters/05-networks.md", "chapters/06-cnn.md", "chapters/07-dataloader.md", "chapters/08-autograd.md", "chapters/09-optimizers.md", "chapters/10-training.md", "chapters/11-compression.md", "chapters/12-kernels.md", "chapters/13-benchmarking.md", "chapters/14-mlops.md", "intro.md", "usage-paths/classroom-use.md", "usage-paths/quick-exploration.md", "usage-paths/serious-development.md"], "titles": ["Setup - TinyTorch System Configuration", "Tensor - Core Data Structure", "Activations - Nonlinearity in Neural Networks", "Layers - Building Blocks of Neural Networks", "Networks - Neural Network Architectures", "CNN - Convolutional Neural Networks", "DataLoader - Data Loading and Preprocessing", "Autograd - Automatic Differentiation Engine", "Optimizers - Gradient-Based Parameter Updates", "Training - Complete Neural Network Training Pipeline", "Compression & Optimization - Making AI Models Efficient", "Kernels - Hardware-Optimized ML Operations", "Benchmarking - Systematic ML Performance Evaluation", "MLOps - Production ML Systems", "Tiny\ud83d\udd25Torch: Build Machine Learning Systems from Scratch", "\ud83d\udc68\u200d\ud83c\udfeb Classroom Use Guide", "\ud83d\udd2c Quick Exploration Path", "\ud83c\udfd7\ufe0f Serious Development Path"], "terms": {"welcom": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "thi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "person": [0, 14], "instal": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], "teach": [0, 14, 15, 16], "you": [0, 1, 7, 10, 11], "nbgrader": [0, 1, 2, 14, 17], "workflow": [0, 1, 2, 13, 14, 16], "custom": [0, 11, 14, 17], "inform": 0, "queri": 0, "us": 0, "python": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17], "master": [0, 1, 2, 4, 6, 7, 8, 9, 10, 11], "implement": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], "test": [0, 1, 2, 3, 14], "export": [0, 15], "creat": [0, 1, 3, 4, 6, 7, 9, 10, 11, 12, 13, 15, 16, 17], "function": [0, 2, 4, 9, 11, 15, 16, 17], "becom": [0, 7, 14, 16], "part": [0, 14], "packag": [0, 17], "understand": [0, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15], "solut": [0, 14], "block": [0, 5, 16], "hidden": 0, "autom": [0, 13, 14, 17], "grade": [0, 14, 17], "foundat": [0, 1, 3, 6, 7], "ani": [0, 4, 16], "product": 0, "In": 0, "ll": [0, 1, 7, 10, 11], "real": [0, 3, 6, 9, 10, 11, 15, 16], "need": [0, 14, 16], "environ": [0, 14, 15, 16], "hardwar": [0, 15, 17], "constraint": 0, "memori": [0, 1, 6, 10, 11, 15, 17], "cpu": [0, 11], "core": [0, 5, 8, 14, 15, 17], "gpu": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "avail": 0, "softwar": 0, "depend": [0, 17], "version": [0, 15], "librari": [0, 11, 15, 17], "compat": 0, "platform": 0, "differ": [0, 4, 6, 12], "linux": 0, "server": 0, "maco": 0, "window": [0, 5], "deploy": [0, 9, 10, 13, 14, 15, 17], "enabl": [0, 2, 3, 5, 6, 8, 14, 17], "document": [0, 15], "exactli": 0, "what": 0, "wa": 0, "manag": [0, 1, 6, 13, 17], "precis": 0, "requir": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], "debug": 0, "support": [0, 17], "info": [0, 15, 17], "help": 0, "troubleshoot": 0, "issu": [0, 13, 17], "proper": [0, 12], "show": [0, 17], "engin": [0, 6, 14, 15, 17], "matur": 0, "attribut": 0, "work": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16], "properli": 0, "credit": 0, "collabor": 0, "other": [0, 5, 14], "can": [0, 5, 14, 15, 16], "extend": [0, 15], "mainten": [0, 13], "updat": [0, 17], "maintain": [0, 13], "connect": [0, 6], "broader": 0, "model": [0, 6, 9, 11, 12, 13, 14, 15, 17], "config": 0, "monitor": [0, 13, 14, 17], "metric": [0, 9, 14, 15, 17], "track": [0, 7, 17], "perform": [0, 1, 6, 7, 9, 11, 13], "scale": 0, "optim": [0, 6, 14, 15, 17], "train": [0, 7, 8, 10, 13], "let": 0, "s": [0, 14], "build": 0, "skill": 0, "choos": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "prefer": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "wai": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "engag": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "launch": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14], "binder": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "run": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "browser": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "No": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "open": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17], "colab": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "googl": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "access": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "cloud": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "comput": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17], "power": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "view": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "sourc": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 17], "brows": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15], "code": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16], "session": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16], "ar": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "temporari": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "download": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "complet": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 16, 17], "notebook": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 17], "when": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "done": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "switch": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "local": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "persist": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17], "readi": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 16], "seriou": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16], "guid": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 16], "where": [1, 2, 3, 4, 6, 7, 8, 9, 10, 11, 12, 13], "tinytorch": [1, 7, 12, 13, 15, 16, 17], "realli": 1, "begin": 1, "fundament": [1, 3, 4], "all": [1, 3, 6, 13, 14, 15, 16, 17], "ml": [1, 6, 14, 15, 16, 17], "system": [1, 6, 7, 12, 16], "n": 1, "dimension": [1, 14, 15, 17], "arrai": [1, 14, 15, 17], "specif": [1, 15, 17], "oper": [1, 5, 7, 14, 15, 16, 17], "class": [1, 6, 7, 14, 15, 16, 17], "arithmet": 1, "handl": [1, 6, 14, 17], "shape": 1, "type": [1, 6, 17], "layout": 1, "neural": [1, 6, 7, 8, 10, 14, 15, 16, 17], "network": [1, 6, 7, 8, 10, 14, 15, 16, 17], "automat": [1, 13, 14, 15, 17], "differenti": [1, 14, 15, 17], "comprehens": [1, 2, 14, 15, 17], "transform": [1, 2, 3, 5, 15, 16], "how": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 16, 17], "form": [1, 3], "develop": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16], "setup": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14], "get": [2, 14, 16], "through": [2, 4, 7, 8, 15], "why": [2, 5, 6, 10, 11, 13, 16], "essenti": [2, 9], "four": [2, 10, 12], "most": [2, 14, 15, 17], "import": [2, 3, 5, 14, 17], "relu": [2, 14, 16, 17], "sigmoid": [2, 14, 17], "tanh": [2, 14, 17], "softmax": [2, 17], "visual": [2, 4, 14, 15, 16, 17], "data": [2, 3, 5, 9, 15, 16], "complex": [2, 4, 7], "see": [2, 3, 5, 8, 13, 14, 16, 17], "layer": [2, 4, 5, 14, 15, 17], "add": [2, 15], "tensor": [2, 14, 15, 16, 17], "immedi": [2, 15], "result": [2, 12, 16], "pattern": [2, 4, 5, 6, 8, 11, 14, 15, 17], "we": [3, 4, 9, 11, 12, 13], "compon": [3, 12, 13, 14, 15, 16, 17], "stack": [3, 17], "togeth": [3, 9, 13, 16, 17], "matrix": [3, 14, 15, 16, 17], "multipl": [3, 14, 15, 16, 17], "naiv": 3, "from": [3, 5, 6, 10, 15, 16, 17], "scratch": [3, 5, 6, 10, 15, 16, 17], "deep": [3, 4, 15, 17], "dens": [3, 5, 14, 15, 16, 17], "linear": [3, 14, 15, 17], "weight": [3, 10, 14, 15, 17], "initi": [3, 17], "strategi": [3, 8, 17], "compos": [3, 4, 5], "activ": [3, 5, 15, 17], "featur": [3, 5, 17], "composit": [4, 15, 17], "f": 4, "x": [4, 14], "layer_n": 4, "layer_2": 4, "layer_1": 4, "sequenti": [4, 14, 15, 17], "common": [4, 12], "like": [4, 5, 14], "mlp": [4, 15], "multi": [4, 14, 15, 17], "perceptron": [4, 17], "capabl": 4, "forward": [4, 14, 15, 17], "pass": [4, 15], "infer": 4, "design": 4, "affect": [4, 6, 10], "behavior": [4, 8], "By": [4, 5, 6, 14], "end": [4, 5, 6, 14, 16, 17], "simpl": [4, 14, 16], "combin": [4, 10, 15, 16], "The": [4, 5, 6, 16], "number": 4, "shallow": 4, "wide": 4, "approxim": 4, "here": 5, "modern": [5, 11, 14], "vision": [5, 14, 15, 17], "its": 5, "conv2d": [5, 15], "explicit": 5, "loop": [5, 9, 13, 15, 17], "slide": [5, 15], "mechan": 5, "detect": [5, 13], "spatial": [5, 15, 17], "imag": [5, 14, 15, 16, 17], "paramet": 5, "share": 5, "translat": 5, "invari": 5, "map": 5, "emerg": 5, "hierarch": 5, "perfect": [5, 14, 15, 16, 17], "learnabl": 5, "pipelin": [5, 6, 13, 14, 15, 17], "flatten": 5, "make": [5, 7, 15, 17], "effici": [5, 6, 11, 17], "process": [6, 9, 11, 15, 17], "machin": 6, "batch": [6, 14, 15, 17], "reusabl": 6, "dataset": [6, 9, 14, 15, 17], "abstract": [6, 14], "framework": [6, 12, 14, 15, 16, 17], "think": 6, "i": [6, 14, 16, 17], "o": 6, "loader": 6, "feed": 6, "them": [6, 14], "scalabl": 6, "consist": [6, 14, 17], "shuffl": [6, 17], "crucial": [6, 16], "larger": [6, 14, 15], "than": [6, 11, 14, 15, 17], "between": [6, 16], "truli": 7, "possibl": [7, 14, 17], "graph": [7, 15, 17], "variabl": 7, "gradient": [7, 15, 17], "backward": [7, 14], "propag": [7, 15, 17], "mathemat": [7, 15, 17], "concept": 7, "behind": [7, 14], "backpropag": [7, 14, 15, 17], "express": 7, "flow": 7, "improv": [8, 9, 11, 17], "intellig": 8, "descent": [8, 17], "sgd": [8, 14, 15, 17], "momentum": 8, "acceler": 8, "converg": [8, 15, 17], "adam": [8, 14, 15, 17], "adapt": 8, "rate": [8, 14, 15, 17], "schedul": [8, 14, 15, 17], "effect": 8, "algorithm": [8, 11, 15], "appli": [8, 10, 11, 12], "compar": [8, 10], "bring": [9, 14], "everyth": [9, 15, 17], "loss": [9, 14, 15, 17], "thei": [9, 14], "measur": [9, 11, 14, 15, 17], "mse": 9, "crossentropi": 9, "binarycrossentropi": 9, "evalu": [9, 14, 15, 17], "classif": [9, 16, 17], "regress": 9, "orchestr": [9, 14, 15], "entir": 9, "checkpoint": [9, 17], "world": [9, 10, 14, 15, 17], "analyz": 9, "dynam": [9, 17], "smaller": 10, "faster": [10, 11], "more": [10, 16], "size": [10, 14, 15, 17], "matter": [10, 11, 16], "magnitud": 10, "base": [10, 15], "prune": [10, 14, 15, 17], "remov": 10, "unimport": 10, "quantiz": [10, 14, 15, 17], "reduc": [10, 17], "usag": 10, "75": [10, 14, 15, 17], "knowledg": 10, "distil": 10, "compact": 10, "structur": 10, "architectur": [10, 12, 14, 15, 17], "techniqu": 10, "trade": 10, "off": 10, "maximum": 10, "gain": [10, 16], "move": 11, "beyond": 11, "numpi": [11, 14], "standard": [11, 12, 15], "vector": 11, "simd": 11, "principl": 11, "better": [11, 14, 15, 17], "cach": 11, "util": 11, "parallel": 11, "style": [11, 14, 15, 17], "profil": [11, 15], "tool": [11, 14, 15], "compress": [11, 14, 15, 17], "workload": 11, "systemat": [11, 14, 15, 17], "industri": [12, 14, 15], "methodolog": 12, "inspir": 12, "mlperf": [12, 15, 17], "scenario": 12, "latenc": 12, "throughput": 12, "offlin": 12, "statist": [12, 15, 17], "valid": [12, 14, 15, 17], "meaning": 12, "profession": [12, 14, 15, 17], "report": [12, 15, 17], "project": [12, 15, 16], "avoid": 12, "pitfal": 12, "gener": [12, 17], "confid": 12, "close": 13, "lifecycl": 13, "degrad": 13, "over": 13, "time": 13, "without": [13, 16], "drift": 13, "retrain": 13, "trigger": 13, "comparison": 13, "infrastructur": [13, 14, 17], "respond": 13, "themselv": 13, "own": [14, 16, 17], "pytorch": [14, 17], "ground": 14, "up": 14, "cours": 14, "have": [14, 16, 17], "built": [14, 15, 17], "cifar": [14, 15, 17], "magic": [14, 17], "reduct": [14, 15], "load": [14, 15, 17], "importantli": 14, "actual": 14, "under": [14, 16], "hood": [14, 16], "explor": 14, "minut": [14, 16], "want": [14, 16, 17], "about": [14, 16], "click": [14, 16], "interact": 14, "feel": 14, "8": [14, 15], "week": 14, "myself": [14, 17], "fork": [14, 16], "repo": [14, 16], "full": [14, 15], "tito": [14, 15], "cli": [14, 15], "14": 14, "assign": [14, 17], "mlop": [14, 15, 17], "classroom": [14, 16], "instructor": 14, "integr": [14, 15, 17], "200": [14, 15, 17], "across": [14, 15, 17], "flexibl": 14, "pace": 14, "16": [14, 15], "pedagog": 14, "turn": [14, 15], "kei": [14, 15], "educ": [14, 16], "1": [], "math": [14, 16], "4": 14, "chain": 14, "cnn": [14, 17], "convolut": [14, 15, 17], "7": 14, "dataload": [14, 17], "preprocess": [14, 15, 17], "autograd": [14, 15, 17], "11": [14, 17], "kernel": [14, 17], "high": [14, 15, 16, 17], "benchmark": [14, 15, 17], "continu": [14, 15, 17], "def": 14, "return": 14, "should": 14, "output": 14, "input_tensor": 14, "784": 14, "128": 14, "repeat": 14, "everi": [14, 17], "fit": [14, 16], "after": [14, 15, 16, 17], "95": [14, 15], "vs": [14, 16], "20": [14, 15, 16], "befor": 14, "90": [14, 15], "deepli": [14, 15, 17], "15": 14, "85": [14, 15], "80": [14, 15], "user": [14, 15], "pursu": [14, 15], "advanc": [14, 15], "role": [14, 15], "30": [14, 15], "graduat": [14, 15], "our": [14, 15], "codebas": [14, 15], "know": 14, "happen": 14, "academ": [14, 15], "successfulli": 14, "univers": 14, "toi": [14, 17], "organ": [14, 17], "consider": [14, 17], "practic": [14, 15, 17], "indic": [14, 17], "success": [14, 17], "messag": [14, 17], "ensur": [14, 15, 17], "aha": 14, "moment": 14, "hello_world": 14, "next": 14, "deploi": [14, 17], "gap": 14, "step": 14, "carefulli": 14, "scaffold": 14, "just": [14, 16], "second": 14, "first": [14, 16, 17], "tradit": 14, "approach": [14, 16], "nn": 14, "mseloss": 14, "doe": [14, 15], "come": 14, "hand": [14, 16], "line": 14, "write": 14, "closer": 14, "ai": [14, 16], "provid": [15, 17], "progress": 15, "00": [15, 17], "13": [15, 17], "correctli": 15, "durat": 15, "git": [15, 17], "http": [15, 17], "github": [15, 17], "com": [15, 17], "org": [15, 17], "cd": [15, 17], "bin": [15, 17], "sh": [15, 17], "doctor": [15, 17], "verifi": [15, 17], "init": 15, "modul": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15], "00_setup": [15, 17], "releas": [15, 17], "ls": 15, "simul": 15, "submiss": [15, 17], "collect": 15, "goal": 15, "qualiti": 15, "assur": 15, "01_tensor": 15, "02_activ": 15, "nonlinear": [15, 16, 17], "25": 15, "03_layer": 15, "5": [15, 16], "04_network": 15, "35": 15, "construct": 15, "05_cnn": 15, "06_dataload": 15, "07_autograd": 15, "40": 15, "9": 15, "08_optim": 15, "analysi": 15, "09_train": 15, "10_compress": 15, "target": [14, 15], "12": [15, 17], "11_kernel": 15, "12_benchmark": 15, "13_mlop": 15, "semest": 15, "Or": 15, "onc": 15, "rang": 15, "03": [15, 17], "auto": [15, 17], "detail": 15, "gradebook": 15, "check": [15, 17], "overal": 15, "verbos": 15, "analyt": 15, "unit": 15, "correct": [15, 17], "cross": [15, 17], "enforc": 15, "145": 15, "135": 15, "total": 15, "445": 15, "60": 15, "do": 15, "clear": 15, "comment": 15, "docstr": 15, "intens": 15, "per": 15, "dive": 15, "research": 15, "competit": 15, "leaderboard": 15, "challeng": 15, "novel": 15, "beginn": 15, "friendli": 15, "extra": 15, "tutori": 15, "earli": 15, "skip": 15, "basic": 15, "focu": 15, "overview": 15, "expect": 15, "nbgrader_config": 15, "py": [15, 17], "modifi": [15, 16], "length": 15, "institut": 15, "ongo": [15, 16], "direct": [15, 17], "commun": 15, "privat": 15, "slack": 15, "discord": 15, "lectur": 15, "note": 15, "offic": 15, "hour": 15, "weekli": 15, "virtual": [15, 17], "ever": 15, "take": 15, "anyth": 16, "button": 16, "live": 16, "jupyt": [16, 17], "your": 16, "pre": 16, "configur": 16, "abil": 16, "account": 16, "creation": 16, "now": [16, 17], "familiar": 16, "timeout": 16, "inact": 16, "save": 16, "great": 16, "For": 16, "relationship": 16, "learn": 16, "level": 16, "api": 16, "satisfi": 16, "ve": 16, "valuabl": 16, "insight": 16, "commit": 16, "my": 17, "A": 17, "includ": 17, "10": 17, "clone": 17, "lab": 17, "setup_dev": 17, "c": 17, "hello_tinytorch": 17, "name": 17, "01": 17, "02": 17, "04": 17, "05": 17, "06": 17, "07": 17, "08": 17, "variant": 17, "09": 17, "awar": 17, "intern": 17, "health": 17, "statu": 17, "dev": 17, "100": 17, "inlin": 17, "classifi": 17, "reliabl": 17, "coverag": 17, "follow": 17, "error": 17, "guidanc": 17, "experi": 17, "new": 17, "jump": 17, "try": 17, "discuss": 17, "question": 17, "bug": 17, "suggest": 17, "contact": 17, "team": 17, "self": [14, 17], "awai": 17, "__add__": 14, "bia": 14, "mse_loss": 14, "pred": 14, "mean": 14, "param": 14, "lr": 14, "grad": 14, "2": 16}, "objects": {}, "objtypes": {}, "objnames": {}, "titleterms": {"modul": [14, 17], "0": [14, 16], "setup": [0, 15, 16, 17], "tinytorch": [0, 14], "system": [0, 13, 14, 15, 17], "configur": 0, "learn": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17], "goal": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "The": [0, 17], "big": [0, 14], "pictur": [0, 14], "why": [0, 14, 17], "matter": 0, "ml": [0, 11, 12, 13], "1": [0, 14, 15, 16, 17], "awar": 0, "2": [0, 14, 15, 17], "reproduc": 0, "3": [0, 14, 15, 16, 17], "profession": 0, "develop": [0, 17], "4": [0, 15, 16, 17], "context": 0, "interact": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "save": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], "your": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17], "progress": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17], "tensor": 1, "core": 1, "data": [1, 6, 14, 17], "structur": [1, 15], "build": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17], "us": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17], "understand": [1, 2, 3, 14, 16, 17], "activ": [2, 14, 16], "nonlinear": 2, "neural": [2, 3, 4, 5, 9], "network": [2, 3, 4, 5, 9], "layer": [3, 16], "block": [3, 14, 15, 17], "architectur": 4, "reflect": [4, 5, 6], "what": [4, 5, 6, 14, 16, 17], "you": [4, 5, 6, 14, 16, 17], "ll": [4, 5, 6, 14, 16, 17], "5": [14, 17], "cnn": 5, "convolut": 5, "6": [14, 15, 17], "dataload": 6, "load": 6, "preprocess": 6, "7": [15, 17], "autograd": 7, "automat": 7, "differenti": 7, "engin": 7, "analyz": [7, 8, 12], "8": 17, "optim": [8, 9, 10, 11], "gradient": 8, "base": 8, "paramet": 8, "updat": 8, "9": 14, "train": [9, 14, 15, 17], "complet": [9, 15], "pipelin": 9, "10": [14, 15], "compress": 10, "make": [10, 14], "ai": 10, "model": 10, "effici": 10, "11": 15, "kernel": 11, "hardwar": 11, "oper": 11, "12": [], "benchmark": 12, "systemat": 12, "perform": [12, 14, 15, 17], "evalu": 12, "13": 14, "mlop": 13, "product": [13, 14, 15, 17], "deploi": 13, "tini": 14, "torch": 14, "machin": 14, "from": 14, "scratch": 14, "achiev": 14, "choos": [14, 17], "path": [14, 16, 17], "three": 14, "wai": 14, "engag": 14, "complex": [14, 17], "each": 14, "previou": 14, "work": [14, 17], "foundat": [14, 15, 17], "philosophi": 14, "exampl": 14, "how": 14, "function": 14, "proven": [14, 15, 17], "student": [14, 15, 17], "outcom": [14, 15, 17], "real": [14, 17], "result": [14, 15, 17], "thi": [14, 17], "differ": 14, "immedi": [14, 16, 17], "feedback": [14, 15, 17], "readi": [14, 17], "start": [14, 15, 16, 17], "adventur": 14, "quick": [14, 15, 16, 17], "tast": 14, "try": [14, 16], "chapter": [14, 16], "right": 14, "now": 14, "classroom": 15, "guid": 15, "cours": 15, "infrastructur": 15, "instructor": 15, "step": [15, 16, 17], "clone": 15, "initi": 15, "nbgrader": 15, "gener": 15, "first": 15, "assign": 15, "test": [15, 17], "workflow": [15, 17], "pace": 15, "week": [15, 17], "14": 15, "manag": 15, "grade": 15, "statu": 15, "monitor": 15, "assess": 15, "autom": 15, "point": 15, "distribut": 15, "suggest": 15, "rubric": 15, "pedagog": 15, "custom": 15, "option": 15, "flexibl": 15, "variat": 15, "prerequisit": 15, "adjust": 15, "get": [15, 17], "review": 15, "materi": 15, "need": 15, "launch": [15, 16], "support": 15, "explor": 16, "requir": 16, "experi": 16, "recommend": 16, "here": 16, "Then": 16, "up": 16, "import": 16, "limit": 16, "next": 16, "seriou": 17, "minut": 17, "code": 17, "environ": 17, "export": 17, "tito": 17, "cli": 17, "qualiti": 17, "assur": 17, "approach": 17, "help": 17}, "envversion": {"sphinx.domains.c": 2, "sphinx.domains.changeset": 1, "sphinx.domains.citation": 1, "sphinx.domains.cpp": 6, "sphinx.domains.index": 1, "sphinx.domains.javascript": 2, "sphinx.domains.math": 2, "sphinx.domains.python": 3, "sphinx.domains.rst": 2, "sphinx.domains.std": 2, "sphinx.ext.intersphinx": 1, "sphinxcontrib.bibtex": 9, "sphinx": 56}})
\ No newline at end of file
diff --git a/book/_build/html/usage-paths/classroom-use.html b/book/_build/html/usage-paths/classroom-use.html
index 818a5797..fad6182e 100644
--- a/book/_build/html/usage-paths/classroom-use.html
+++ b/book/_build/html/usage-paths/classroom-use.html
@@ -65,7 +65,7 @@ const thebe_selector_output = ".output, .cell_output"
-
+
@@ -176,29 +176,29 @@ const thebe_selector_output = ".output, .cell_output"
@@ -779,7 +779,7 @@ tito nbgrader analytics
@@ -866,11 +866,11 @@ tito nbgrader autograde
diff --git a/book/_build/html/usage-paths/quick-exploration.html b/book/_build/html/usage-paths/quick-exploration.html
index 957f12b4..97c6beb4 100644
--- a/book/_build/html/usage-paths/quick-exploration.html
+++ b/book/_build/html/usage-paths/quick-exploration.html
@@ -176,29 +176,29 @@ const thebe_selector_output = ".output, .cell_output"
@@ -413,9 +413,9 @@ document.write(`
@@ -704,9 +704,9 @@ tito nbgrader autogradeπ Ready to Start?#
Module 12: Benchmarking - Systematic ML Performance Evaluation#
+Benchmarking - Systematic ML Performance Evaluation#
Welcome to the Benchmarking module! This is where we learn to systematically evaluate ML systems using industry-standard methodology inspired by MLPerf.
π― Learning Goals
@@ -523,7 +523,7 @@ document.write(` diff --git a/book/_build/html/chapters/12-kernels.html b/book/_build/html/chapters/12-kernels.html new file mode 100644 index 00000000..4c76ff5c --- /dev/null +++ b/book/_build/html/chapters/12-kernels.html @@ -0,0 +1,610 @@ + + + + + + + + + + + +
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/book/_build/html/chapters/13-benchmarking.html b/book/_build/html/chapters/13-benchmarking.html
new file mode 100644
index 00000000..1052b4d6
--- /dev/null
+++ b/book/_build/html/chapters/13-benchmarking.html
@@ -0,0 +1,609 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Kernels - Hardware-Optimized ML Operations
+ +
+
+
+
+
+
+
+
+ Contents
+Kernels - Hardware-Optimized ML Operations#
+Welcome to the Kernels module! This is where we move beyond NumPy to understand how ML operations are optimized for modern hardware. Youβll implement custom kernels that run faster than standard library functions.
+
+
+π― Learning Goals
+-
+
Understand why custom kernels matter for ML performance
+Implement vectorized operations using SIMD principles
+Master memory-efficient algorithms for better cache utilization
+Build parallel processing patterns for CPU and GPU-style computing
+Create performance profiling tools to measure and optimize code
+Apply kernel optimizations to compressed model operations
+
Build β Use β Optimize#
+-
+
Build: Custom operations, vectorization, and memory optimization
+Use: Apply optimized kernels to real ML workloads
+Optimize: Profile, measure, and improve performance systematically
+
π Interactive Learning#
+Choose your preferred way to engage with this module:
+
+
+
+
+
+πΎ Save Your Progress
+Binder sessions are temporary! Download your completed notebook when done, or switch to local development for persistent work.
+Ready for serious development? β ποΈ Local Setup Guide
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/book/_build/html/chapters/13-mlops.html b/book/_build/html/chapters/13-mlops.html
index a066d285..0e8fc7e4 100644
--- a/book/_build/html/chapters/13-mlops.html
+++ b/book/_build/html/chapters/13-mlops.html
@@ -9,7 +9,7 @@
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Benchmarking - Systematic ML Performance Evaluation
+ +
+
+
+
+
+
+
+
+ Contents
+Benchmarking - Systematic ML Performance Evaluation#
+Welcome to the Benchmarking module! This is where we learn to systematically evaluate ML systems using industry-standard methodology inspired by MLPerf.
+
+
+π― Learning Goals
+-
+
Understand the four-component MLPerf benchmarking architecture
+Implement different benchmark scenarios (latency, throughput, offline)
+Apply statistical validation for meaningful results
+Create professional performance reports for ML projects
+Learn to avoid common benchmarking pitfalls
+
Build β Use β Analyze#
+-
+
Build: Benchmarking framework with proper statistical validation
+Use: Apply systematic evaluation to your TinyTorch models
+Analyze: Generate professional reports with statistical confidence
+
π Interactive Learning#
+Choose your preferred way to engage with this module:
+
+
+
+
+
+πΎ Save Your Progress
+Binder sessions are temporary! Download your completed notebook when done, or switch to local development for persistent work.
+Ready for serious development? β ποΈ Local Setup Guide
+Foundation
@@ -400,7 +400,7 @@ document.write(`
-
@@ -427,7 +427,7 @@ document.write(`
π Ready to Start?
ποΈ Big Picture: Why Build from Scratch?
@@ -616,21 +616,28 @@ document.write(`
-Module 13: MLOps - Production ML Systems
+MLOps - Production ML Systems
@@ -423,8 +423,8 @@ document.write(`
-
-
+
diff --git a/book/_build/html/chapters/14-mlops.html b/book/_build/html/chapters/14-mlops.html
new file mode 100644
index 00000000..99a8efa5
--- /dev/null
+++ b/book/_build/html/chapters/14-mlops.html
@@ -0,0 +1,599 @@
+
+
+
+
+
+
+
+
+
+
+
+ MLOps - Production ML Systems — Tinyπ₯Torch: Build ML Systems from Scratch
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Skip to main content
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/book/_build/html/intro.html b/book/_build/html/intro.html
index 5fc212d9..b20533f3 100644
--- a/book/_build/html/intro.html
+++ b/book/_build/html/intro.html
@@ -175,29 +175,29 @@ const thebe_selector_output = ".output, .cell_output"
Module 13: MLOps - Production ML Systems#
+MLOps - Production ML Systems#
Welcome to the MLOps module! This is where we close the loop on the complete ML system lifecycle.
π― Learning Goals
@@ -522,7 +522,7 @@ document.write(`
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/book/_build/html/genindex.html b/book/_build/html/genindex.html
index 6cee819a..0db72cb5 100644
--- a/book/_build/html/genindex.html
+++ b/book/_build/html/genindex.html
@@ -173,29 +173,29 @@ const thebe_selector_output = ".output, .cell_output"
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ MLOps - Production ML Systems
+ +
+
+
+
+
+
+
+
+ Contents
+MLOps - Production ML Systems#
+Welcome to the MLOps module! This is where we close the loop on the complete ML system lifecycle.
+
+
+π― Learning Goals
+-
+
Understand why ML models degrade over time without maintenance
+Implement performance monitoring and drift detection systems
+Build automated retraining triggers that use your training pipeline
+Create model comparison and deployment workflows
+See how all TinyTorch components work together in production
+
Build β Use β Deploy#
+-
+
Build: Complete MLOps infrastructure for model lifecycle management
+Use: Deploy and monitor ML systems that automatically respond to issues
+Deploy: Create production-ready systems that maintain themselves over time
+
π Interactive Learning#
+Choose your preferred way to engage with this module:
+
+
+
+
+
+πΎ Save Your Progress
+Binder sessions are temporary! Download your completed notebook when done, or switch to local development for persistent work.
+Ready for serious development? β ποΈ Local Setup Guide
+Foundation
Building Blocks
Training Systems
Production & Performance
Foundation
Building Blocks
Training Systems
Production & Performance
Ready to build? β ποΈ Serious Development (Fork repo and build your ML framework)
Teaching a class? β π¨βπ« Classroom Use (Complete course infrastructure)
Quick Taste: Try Chapter 0 Right Now#
-Want to see what TinyTorch feels like? Launch the Setup chapter in Binder and implement your first TinyTorch function in 2 minutes!
+Quick Taste: Try Chapter 1 Right Now#
+Want to see what TinyTorch feels like? Launch the Setup chapter in Binder and implement your first TinyTorch function in 2 minutes!
ποΈ Big Picture: Why Build from Scratch?#
Most ML education teaches you to use frameworks. TinyTorch teaches you to understand them.
-Traditional ML Course: TinyTorch Approach:
-βββ import torch βββ Build your own tensors
-βββ model = nn.Linear(10, 1) βββ Implement Linear layers
-βββ loss = nn.MSELoss() βββ Create loss functions
-βββ optimizer.step() βββ Build optimizers
-βββ "How does this work?" π€· βββ "I built this!" πͺ
+Traditional ML Course: TinyTorch Approach:
+βββ import torch βββ class Tensor:
+βββ model = nn.Linear(10, 1) β def __add__(self, other): ...
+βββ loss = nn.MSELoss() β def backward(self): ...
+βββ optimizer.step() βββ class Linear:
+βββ "How does this work?" π€· β def forward(self, x):
+ β return x @ self.weight + self.bias
+ βββ def mse_loss(pred, target):
+ β return ((pred - target) ** 2).mean()
+ βββ class SGD:
+ β def step(self):
+ β param.data -= lr * param.grad
+ βββ "I implemented every line!" πͺ
Result: You become the person others come to when they need to understand βhow PyTorch actually works under the hood.β
@@ -722,7 +729,7 @@ document.write(`
Foundation
Building Blocks
Training Systems
Production & Performance
Foundation
Building Blocks
Training Systems
Production & Performance
1. Review Course Materials#
Browse the course overview
-Test the setup module
+Test the setup module
Foundation
Building Blocks
Training Systems
Production & Performance
- π Launch Immediately (0 Setup Required)
- π Recommended Exploration Path
- β οΈ Important Limitations @@ -460,32 +460,32 @@ document.write(`
Understand the TinyTorch development workflow
Get familiar with the educational approach
See how components fit together
Implement your first ML function (ReLU)
See immediate visual results
Understand why nonlinearity matters
Create the building blocks of neural networks
Combine your ReLU with matrix operations
See how simple math becomes powerful AI
- π Launch Immediately (0 Setup Required)
- π Recommended Exploration Path
- β οΈ Important Limitations diff --git a/book/_build/html/usage-paths/serious-development.html b/book/_build/html/usage-paths/serious-development.html index 18ab8255..aa9abdc2 100644 --- a/book/_build/html/usage-paths/serious-development.html +++ b/book/_build/html/usage-paths/serious-development.html @@ -176,29 +176,29 @@ const thebe_selector_output = ".output, .cell_output"
π Recommended Exploration Path#
-Start Here: Chapter 0 - Setup#
+Start Here: Chapter 1 - Setup#
π Launch Setup Chapter
+π Launch Setup Chapter
Then Try: Chapter 2 - Activations#
+Then Try: Chapter 3 - Activations#
Build Up: Chapter 3 - Layers#
+Build Up: Chapter 4 - Layers#
@@ -585,9 +585,9 @@ document.write(`
Foundation
Building Blocks
Training Systems
Production & Performance
Choose Your Module#
-New to ML frameworks? β Start with Setup
-Have ML experience? β Jump to Tensors
-Want to see the vision? β Try Activations
New to ML frameworks? β Start with Setup +Have ML experience? β Jump to Tensors +Want to see the vision? β Try Activations