Update documentation after module reordering

All module references updated to reflect new ordering:
- Module 15: Quantization (was 16)
- Module 16: Compression (was 17)
- Module 17: Memoization (was 15)

Updated by module-developer and website-manager agents:
- Module ABOUT files with correct numbers and prerequisites
- Cross-references and "What's Next" chains
- Website navigation (_toc.yml) and content
- Learning path progression in LEARNING_PATH.md
- Profile milestone completion message (Module 17)

Pedagogical flow now: Profile → Quantize → Prune → Cache → Accelerate

modules/01_tensor/ABOUT.md

@@ -0,0 +1,328 @@
+---
+title: "Tensor"
+description: "Core tensor data structure and operations"
+module_number: 1
+tier: "foundation"
+difficulty: "beginner"
+time_estimate: "4-6 hours"
+prerequisites: ["Environment Setup"]
+next_module: "02. Activations"
+learning_objectives:
+  - "Understand tensors as N-dimensional arrays and their role in ML systems"
+  - "Implement a complete Tensor class with arithmetic and shape operations"
+  - "Handle memory management, data types, and broadcasting efficiently"
+  - "Recognize how tensor operations form the foundation of PyTorch/TensorFlow"
+  - "Analyze computational complexity and memory usage of tensor operations"
+---
+
+# 01. Tensor
+
+**🏗️ FOUNDATION TIER** | Difficulty: ⭐ (1/4) | Time: 4-6 hours
+
+**Build N-dimensional arrays from scratch - the foundation of all ML computations.**
+
+---
+
+## What You'll Build
+
+The **Tensor** class is the fundamental data structure of machine learning. It represents N-dimensional arrays and provides operations for manipulation, computation, and transformation.
+
+By the end of this module, you'll have a working Tensor implementation that handles:
+
+- Creating and initializing N-dimensional arrays
+- Arithmetic operations (addition, multiplication, division, powers)
+- Shape manipulation (reshape, transpose, broadcasting)
+- Reductions (sum, mean, min, max along any axis)
+- Memory-efficient data storage and copying
+
+### Example Usage
+
+```python
+from tinytorch.core.tensor import Tensor
+
+# Create tensors
+x = Tensor([[1.0, 2.0], [3.0, 4.0]])
+y = Tensor([[0.5, 1.5], [2.5, 3.5]])
+
+# Properties
+print(x.shape)    # (2, 2)
+print(x.size)     # 4
+print(x.dtype)    # float64
+
+# Operations
+z = x + y         # Addition
+w = x * y         # Element-wise multiplication
+p = x ** 2        # Exponentiation
+
+# Shape manipulation
+reshaped = x.reshape(4, 1)
+transposed = x.T
+
+# Reductions
+total = x.sum()             # Scalar sum
+means = x.mean(axis=0)      # Mean along axis
+```
+
+---
+
+## Learning Pattern: Build → Use → Understand
+
+### 1. Build
+Implement the Tensor class from scratch using NumPy as the underlying array library. You'll create constructors, operator overloading, shape manipulation methods, and reduction operations.
+
+### 2. Use
+Apply your Tensor implementation to real problems: matrix multiplication, data normalization, statistical computations. Test with various shapes and data types.
+
+### 3. Understand
+Grasp the systems-level implications: why tensor operations dominate compute time, how memory layout affects performance, and how broadcasting enables efficient computations without data copying.
+
+---
+
+## Learning Objectives
+
+By completing this module, you will:
+
+1. **Systems Understanding**: Recognize tensors as the universal data structure in ML frameworks, understanding how all neural network operations decompose into tensor primitives
+
+2. **Core Implementation**: Build a complete Tensor class supporting arithmetic, shape manipulation, and reductions with proper error handling
+
+3. **Pattern Recognition**: Understand broadcasting rules and how they enable efficient computations across different tensor shapes
+
+4. **Framework Connection**: See how your implementation mirrors PyTorch's `torch.Tensor` and TensorFlow's `tf.Tensor` design
+
+5. **Performance Trade-offs**: Analyze memory usage vs computation speed, understanding when to copy data vs create views
+
+---
+
+## Why This Matters
+
+### Production Context
+
+Every modern ML framework is built on tensors:
+
+- **PyTorch**: `torch.Tensor` is the core class - all operations work with tensors
+- **TensorFlow**: `tf.Tensor` represents data flowing through computation graphs  
+- **JAX**: `jax.numpy.ndarray` extends NumPy with automatic differentiation
+- **NumPy**: The foundation - understanding tensors starts here
+
+By building your own Tensor class, you'll understand what happens when you call `torch.matmul()` or `tf.reduce_sum()` - not just the API, but the actual computation.
+
+### Systems Reality Check
+
+**Performance Note**: Tensor operations dominate training time. A single matrix multiplication in a linear layer might take 90% of forward pass time. Understanding tensor internals is essential for optimization.
+
+**Memory Note**: Large models store billions of parameters as tensors. A GPT-3 scale model requires 350GB of memory just for weights (175B parameters × 2 bytes for FP16). Efficient tensor memory management is critical.
+
+---
+
+## Implementation Guide
+
+### Prerequisites Check
+
+Verify your environment is ready:
+
+```bash
+tito system doctor
+```
+
+All checks should pass before starting implementation.
+
+### Development Workflow
+
+```bash
+# Navigate to tensor module
+cd modules/01_tensor/
+
+# Open development file (choose your preferred method)
+jupyter lab tensor_dev.py          # Jupytext (recommended)
+# OR
+code tensor_dev.py                 # Direct Python editing
+```
+
+### Step-by-Step Build
+
+#### Step 1: Tensor Class Foundation
+
+Create the basic Tensor class with initialization and properties:
+
+```python
+class Tensor:
+    def __init__(self, data, dtype=None):
+        """Initialize tensor from Python list or NumPy array"""
+        self.data = np.array(data, dtype=dtype)
+    
+    @property
+    def shape(self):
+        """Return tensor shape"""
+        return self.data.shape
+    
+    @property
+    def size(self):
+        """Return total number of elements"""
+        return self.data.size
+```
+
+**Why this matters**: Properties enable clean API design - users can write `x.shape` instead of `x.get_shape()`, matching PyTorch conventions.
+
+#### Step 2: Arithmetic Operations
+
+Implement operator overloading for element-wise operations:
+
+```python
+def __add__(self, other):
+    """Element-wise addition"""
+    return Tensor(self.data + other.data)
+
+def __mul__(self, other):
+    """Element-wise multiplication"""
+    return Tensor(self.data * other.data)
+```
+
+**Systems insight**: These operations vectorize automatically via NumPy, achieving ~100x speedup over Python loops. This is why frameworks use tensors.
+
+#### Step 3: Shape Manipulation
+
+Implement reshape, transpose, and broadcasting:
+
+```python
+def reshape(self, *shape):
+    """Return tensor with new shape"""
+    return Tensor(self.data.reshape(*shape))
+
+@property
+def T(self):
+    """Return transposed tensor"""
+    return Tensor(self.data.T)
+```
+
+**Memory consideration**: Reshape and transpose often return *views* (no data copying) for efficiency. Understanding views vs copies is crucial for memory optimization.
+
+#### Step 4: Reductions
+
+Implement aggregation operations along axes:
+
+```python
+def sum(self, axis=None):
+    """Sum tensor elements along axis"""
+    return Tensor(self.data.sum(axis=axis))
+
+def mean(self, axis=None):
+    """Mean of tensor elements along axis"""
+    return Tensor(self.data.mean(axis=axis))
+```
+
+**Production pattern**: Reductions are fundamental - every loss function uses them. Understanding axis semantics prevents bugs in multi-dimensional operations.
+
+---
+
+## Testing Your Implementation
+
+### Inline Tests
+
+Test within your development file:
+
+```python
+# Create test tensors
+x = Tensor([[1, 2], [3, 4]])
+y = Tensor([[5, 6], [7, 8]])
+
+# Test operations
+assert x.shape == (2, 2)
+assert (x + y).data.tolist() == [[6, 8], [10, 12]]
+assert x.sum().data == 10
+print("✓ Basic operations working")
+```
+
+### Module Export & Validation
+
+```bash
+# Export your implementation to TinyTorch package
+tito export 01
+
+# Run comprehensive test suite
+tito test 01
+```
+
+**Expected output**:
+```
+✓ All tests passed! [25/25]
+✓ Module 01 complete!
+```
+
+---
+
+## Where This Code Lives
+
+After export, your Tensor implementation becomes part of the TinyTorch package:
+
+```python
+# Other modules and future code can now import YOUR implementation:
+from tinytorch.core.tensor import Tensor
+
+# Used throughout TinyTorch:
+from tinytorch.core.layers import Linear      # Uses Tensor for weights
+from tinytorch.core.activations import ReLU   # Operates on Tensors
+from tinytorch.core.autograd import backward  # Computes Tensor gradients
+```
+
+**Package structure**:
+```
+tinytorch/
+├── core/
+│   ├── tensor.py  ← YOUR implementation exports here
+│   ├── activations.py
+│   ├── layers.py
+│   └── ...
+```
+
+---
+
+## Systems Thinking Questions
+
+Reflect on these questions as you build (no right/wrong answers):
+
+1. **Complexity Analysis**: Why is matrix multiplication O(n³) for n×n matrices? How does this affect training time for large models?
+
+2. **Memory Trade-offs**: When should reshape create a view vs copy data? What are the performance implications?
+
+3. **Production Scaling**: A GPT-3 scale model has 175 billion parameters. How much memory is required to store these as FP32 tensors? As FP16?
+
+4. **Design Decisions**: Why do frameworks like PyTorch store data as NumPy arrays internally? What are alternatives?
+
+5. **Framework Comparison**: How does your Tensor class differ from `torch.Tensor`? What features are missing? Why might those features matter?
+
+---
+
+## Real-World Connections
+
+### Industry Applications
+
+- **Deep Learning Training**: All neural network layers operate on tensors (Linear, Conv2d, Attention all perform tensor operations)
+- **Scientific Computing**: Tensors represent multidimensional data (climate models, molecular simulations)
+- **Computer Vision**: Images are 3D tensors (height × width × channels)
+- **NLP**: Text embeddings are 2D tensors (sequence_length × embedding_dim)
+
+### Research Applications
+
+- **Automatic Differentiation**: Frameworks like PyTorch track tensor operations to compute gradients
+- **Distributed Training**: Large models split tensors across GPUs using tensor parallelism
+- **Quantization**: Tensors can be stored in reduced precision (INT8 instead of FP32) for efficiency
+
+---
+
+## What's Next?
+
+**Congratulations!** You've built the foundation of TinyTorch. Your Tensor class will power everything that follows - from activation functions to complete neural networks.
+
+Next, you'll add nonlinearity to enable networks to learn complex patterns.
+
+**Module 02: Activations** - Implement ReLU, Sigmoid, Tanh, and other activation functions that transform tensor values
+
+[Continue to Module 02: Activations →](02-activations.html)
+
+---
+
+**Need Help?**
+- [Ask in GitHub Discussions](https://github.com/mlsysbook/TinyTorch/discussions)
+- [View Tensor API Reference](../appendices/api-reference.html#tensor)
+- [Report Issues](https://github.com/mlsysbook/TinyTorch/issues)

modules/02_activations/ABOUT.md

@@ -0,0 +1,217 @@
+---
+title: "Activation Functions"
+description: "Neural network activation functions (ReLU, Sigmoid, Tanh, Softmax)"
+difficulty: "⭐⭐"
+time_estimate: "3-4 hours"
+prerequisites: []
+next_steps: []
+learning_objectives: []
+---
+
+# 02. Activations
+
+**🏗️ FOUNDATION TIER** | Difficulty: ⭐⭐ (2/4) | Time: 3-4 hours
+
+## Overview
+
+Implement the mathematical functions that give neural networks their power to learn complex patterns. Without activation functions, neural networks would just be linear transformations—with them, you unlock the ability to learn any function.
+
+## Learning Objectives
+
+By the end of this module, you will be able to:
+
+- **Understand the critical role** of activation functions in enabling neural networks to learn non-linear patterns
+- **Implement three core activation functions**: ReLU, Sigmoid, and Tanh with proper numerical stability
+- **Apply mathematical reasoning** to understand function properties, ranges, and appropriate use cases
+- **Debug and test** activation implementations using both automated tests and visual analysis
+- **Connect theory to practice** by understanding when and why to use each activation function
+
+## Build → Use → Analyze
+
+This module follows TinyTorch's **Build → Use → Analyze** framework:
+
+1. **Build**: Implement ReLU, Sigmoid, and Tanh activation functions with numerical stability
+2. **Use**: Apply these functions in testing scenarios and visualize their mathematical behavior
+3. **Analyze**: Compare function properties, performance characteristics, and appropriate use cases through quantitative analysis
+
+## Implementation Guide
+
+### Core Activation Functions
+```python
+# ReLU: Simple but powerful
+relu = ReLU()
+output = relu(Tensor([-2, -1, 0, 1, 2]))  # [0, 0, 0, 1, 2]
+
+# Sigmoid: Probabilistic outputs
+sigmoid = Sigmoid()
+output = sigmoid(Tensor([0, 1, -1]))      # [0.5, 0.73, 0.27]
+
+# Tanh: Zero-centered activation
+tanh = Tanh()
+output = tanh(Tensor([0, 1, -1]))         # [0, 0.76, -0.76]
+```
+
+### ReLU (Rectified Linear Unit)
+- **Formula**: `f(x) = max(0, x)`
+- **Properties**: Simple, sparse, unbounded, most commonly used
+- **Implementation**: Element-wise maximum with zero
+- **Use Cases**: Hidden layers in most modern architectures
+
+### Sigmoid Activation
+- **Formula**: `f(x) = 1 / (1 + e^(-x))`
+- **Properties**: Bounded to (0,1), smooth, probabilistic interpretation
+- **Implementation**: Numerically stable version preventing overflow
+- **Use Cases**: Binary classification, attention mechanisms, gates
+
+### Tanh (Hyperbolic Tangent)
+- **Formula**: `f(x) = tanh(x)`
+- **Properties**: Bounded to (-1,1), zero-centered, symmetric
+- **Implementation**: Direct NumPy implementation with shape preservation
+- **Use Cases**: Hidden layers, RNNs, when zero-centered outputs are beneficial
+
+## Getting Started
+
+### Prerequisites
+Ensure you have completed the tensor module and understand basic tensor operations:
+
+   ```bash
+# Activate TinyTorch environment
+   source bin/activate-tinytorch.sh
+
+# Verify tensor module is working
+tito test --module tensor
+   ```
+
+### Development Workflow
+1. **Open the development file**: `modules/03_activations/activations_dev.py`
+2. **Implement functions progressively**: Start with ReLU, then Sigmoid (numerical stability), then Tanh
+3. **Test each implementation**: Use inline tests for immediate feedback
+4. **Visualize function behavior**: Leverage plotting sections for mathematical understanding
+5. **Export and verify**: `tito export --module activations && tito test --module activations`
+
+## Testing
+
+### Comprehensive Test Suite
+Run the full test suite to verify mathematical correctness:
+
+   ```bash
+# TinyTorch CLI (recommended)
+   tito test --module activations
+
+# Direct pytest execution
+python -m pytest tests/ -k activations -v
+```
+
+### Test Coverage Areas
+- ✅ **Mathematical Correctness**: Verify function outputs match expected mathematical formulas
+- ✅ **Numerical Stability**: Test with extreme values and edge cases
+- ✅ **Shape Preservation**: Ensure input and output tensors have identical shapes
+- ✅ **Range Validation**: Confirm outputs fall within expected ranges
+- ✅ **Integration Testing**: Verify compatibility with tensor operations
+
+### Inline Testing & Visualization
+The module includes comprehensive educational feedback:
+```python
+# Example inline test output
+🔬 Unit Test: ReLU activation...
+✅ ReLU handles negative inputs correctly
+✅ ReLU preserves positive inputs
+✅ ReLU output range is [0, ∞)
+📈 Progress: ReLU ✓
+
+# Visual feedback with plotting
+📊 Plotting ReLU behavior across range [-5, 5]...
+📈 Function visualization shows expected behavior
+   ```
+
+### Manual Testing Examples
+```python
+from tinytorch.core.tensor import Tensor
+from activations_dev import ReLU, Sigmoid, Tanh
+
+# Test with various inputs
+x = Tensor([[-2.0, -1.0, 0.0, 1.0, 2.0]])
+
+relu = ReLU()
+sigmoid = Sigmoid()
+tanh = Tanh()
+
+print("Input:", x.data)
+print("ReLU:", relu(x).data)      # [0, 0, 0, 1, 2]
+print("Sigmoid:", sigmoid(x).data) # [0.12, 0.27, 0.5, 0.73, 0.88]
+print("Tanh:", tanh(x).data)      # [-0.96, -0.76, 0, 0.76, 0.96]
+```
+
+## Systems Thinking Questions
+
+### Real-World Applications
+- **Computer Vision**: ReLU activations enable CNNs to learn hierarchical features (like those in ResNet, VGG)
+- **Natural Language Processing**: Sigmoid/Tanh functions power LSTM and GRU gates for memory control
+- **Recommendation Systems**: Sigmoid activations provide probability outputs for binary predictions
+- **Generative Models**: Different activations shape the output distributions in GANs and VAEs
+
+### Mathematical Properties Comparison
+| Function | Input Range | Output Range | Zero Point | Key Property |
+|----------|-------------|--------------|------------|--------------|
+| ReLU     | (-∞, ∞)     | [0, ∞)       | f(0) = 0   | Sparse, unbounded |
+| Sigmoid  | (-∞, ∞)     | (0, 1)       | f(0) = 0.5 | Probabilistic |
+| Tanh     | (-∞, ∞)     | (-1, 1)      | f(0) = 0   | Zero-centered |
+
+### Numerical Stability Considerations
+- **ReLU**: No stability issues (simple max operation)
+- **Sigmoid**: Requires careful implementation to prevent `exp()` overflow
+- **Tanh**: Generally stable, but NumPy implementation handles edge cases
+
+### Performance and Gradient Properties
+- **ReLU**: Fastest computation, sparse gradients, can cause "dying ReLU" problem
+- **Sigmoid**: Moderate computation, smooth gradients, susceptible to vanishing gradients
+- **Tanh**: Moderate computation, stronger gradients than sigmoid, zero-centered helps optimization
+
+## 🎉 Ready to Build?
+
+The activations module is where neural networks truly come alive! You're about to implement the mathematical functions that transform simple linear operations into powerful pattern recognition systems.
+
+Every major breakthrough in deep learning—from image recognition to language models—relies on the functions you're about to build. Take your time, understand the mathematics, and enjoy creating the foundation of intelligent systems!
+
+
+
+
+Choose your preferred way to engage with this module:
+
+````{grid} 1 2 3 3
+
+```{grid-item-card} 🚀 Launch Binder
+:link: https://mybinder.org/v2/gh/mlsysbook/TinyTorch/main?filepath=modules/03_activations/activations_dev.ipynb
+:class-header: bg-light
+
+Run this module interactively in your browser. No installation required!
+```
+
+```{grid-item-card} ⚡ Open in Colab  
+:link: https://colab.research.google.com/github/mlsysbook/TinyTorch/blob/main/modules/03_activations/activations_dev.ipynb
+:class-header: bg-light
+
+Use Google Colab for GPU access and cloud compute power.
+```
+
+```{grid-item-card} 📖 View Source
+:link: https://github.com/mlsysbook/TinyTorch/blob/main/modules/03_activations/activations_dev.py
+:class-header: bg-light
+
+Browse the Python source code and understand the implementation.
+```
+
+````
+
+```{admonition} 💾 Save Your Progress
+:class: tip
+**Binder sessions are temporary!** Download your completed notebook when done, or switch to local development for persistent work.
+
+```
+
+---
+
+<div class="prev-next-area">
+<a class="left-prev" href="../chapters/02_tensor.html" title="previous page">← Previous Module</a>
+<a class="right-next" href="../chapters/04_layers.html" title="next page">Next Module →</a>
+</div>

modules/03_layers/ABOUT.md

@@ -0,0 +1,226 @@
+---
+title: "Layers"
+description: "Neural network layers (Linear, activation layers)"
+difficulty: "⭐⭐"
+time_estimate: "4-5 hours"
+prerequisites: []
+next_steps: []
+learning_objectives: []
+---
+
+# 03. Layers
+
+**🏗️ FOUNDATION TIER** | Difficulty: ⭐⭐ (2/4) | Time: 4-5 hours
+
+## Overview
+
+Build the fundamental transformations that compose into neural networks. This module teaches you that layers are simply functions that transform tensors, and neural networks are just sophisticated function composition using these building blocks.
+
+## Learning Objectives
+
+By completing this module, you will be able to:
+
+1. **Understand layers as mathematical functions** that transform tensors through well-defined operations
+2. **Implement Dense layers** using matrix multiplication and bias addition (`y = Wx + b`)
+3. **Integrate activation functions** to combine linear transformations with nonlinearity
+4. **Compose building blocks** by chaining layers into complete neural network architectures
+5. **Debug layer implementations** using shape analysis and mathematical properties
+
+## Why This Matters
+
+### Production Context
+
+Layers are the building blocks of every neural network in production:
+
+- **Image Recognition** uses Dense layers for final classification (ResNet, EfficientNet)
+- **Language Models** compose thousands of transformer layers (GPT, BERT, Claude)
+- **Recommendation Systems** stack Dense layers to learn user-item interactions
+- **Autonomous Systems** chain convolutional and Dense layers for perception
+
+### Historical Context
+
+The evolution of layer abstractions enabled modern deep learning:
+
+- **1943**: McCulloch-Pitts neuron - first artificial neuron model
+- **1958**: Rosenblatt's Perceptron - single-layer learning algorithm
+- **1986**: Backpropagation - enabled training multi-layer networks
+- **2012**: AlexNet - proved deep layers (8 layers) revolutionize computer vision
+- **2017**: Transformers - layer composition scaled to 96+ layers in modern LLMs
+
+## Build → Use → Understand
+
+This module follows the foundational pedagogy for building blocks:
+
+1. **Build**: Implement Dense layer class with initialization, forward pass, and parameter management
+2. **Use**: Transform data through layer operations and compose multi-layer networks
+3. **Understand**: Analyze how layer composition creates expressivity and why architecture design matters
+
+## Implementation Guide
+
+### Core Layer Implementation
+```python
+# Dense layer: fundamental building block
+layer = Dense(input_size=3, output_size=2)
+x = Tensor([[1.0, 2.0, 3.0]])
+y = layer(x)  # Shape transformation: (1, 3) → (1, 2)
+
+# With activation functions
+relu = ReLU()
+activated = relu(y)  # Apply nonlinearity
+
+# Chaining operations
+layer1 = Dense(784, 128)  # Image → hidden
+layer2 = Dense(128, 10)   # Hidden → classes
+activation = ReLU()
+
+# Forward pass composition
+x = Tensor([[1.0, 2.0, 3.0, ...]])  # Input data
+h1 = activation(layer1(x))           # First transformation
+output = layer2(h1)                  # Final prediction
+```
+
+### Dense Layer Implementation
+- **Mathematical foundation**: Linear transformation `y = Wx + b`
+- **Weight initialization**: Xavier/Glorot uniform initialization for stable gradients
+- **Bias handling**: Optional bias terms for translation invariance
+- **Shape management**: Automatic handling of batch dimensions and matrix operations
+
+### Activation Layer Integration
+- **ReLU integration**: Most common activation for hidden layers
+- **Sigmoid integration**: Probability outputs for binary classification
+- **Tanh integration**: Zero-centered outputs for better optimization
+- **Composition patterns**: Standard ways to combine layers and activations
+
+## Testing
+
+Run the complete test suite to verify your implementation:
+
+```bash
+# TinyTorch CLI (recommended)
+tito test --module layers
+
+# Direct pytest execution
+python -m pytest tests/ -k layers -v
+```
+
+### Test Coverage Areas
+- ✅ **Layer Functionality**: Verify Dense layers perform correct linear transformations
+- ✅ **Weight Initialization**: Ensure proper weight initialization for training stability
+- ✅ **Shape Preservation**: Confirm layers handle batch dimensions correctly
+- ✅ **Activation Integration**: Test seamless combination with activation functions
+- ✅ **Network Composition**: Verify layers can be chained into complete networks
+
+### Inline Testing & Development
+The module includes educational feedback during development:
+```python
+# Example inline test output
+🔬 Unit Test: Dense layer functionality...
+✅ Dense layer computes y = Wx + b correctly
+✅ Weight initialization within expected range
+✅ Output shape matches expected dimensions
+📈 Progress: Dense Layer ✓
+
+# Integration testing
+🔬 Unit Test: Layer composition...
+✅ Multiple layers chain correctly
+✅ Activations integrate seamlessly
+📈 Progress: Layer Composition ✓
+```
+
+### Manual Testing Examples
+```python
+from tinytorch.core.tensor import Tensor
+from layers_dev import Dense
+from activations_dev import ReLU
+
+# Test basic layer functionality
+layer = Dense(input_size=3, output_size=2)
+x = Tensor([[1.0, 2.0, 3.0]])
+y = layer(x)
+print(f"Input shape: {x.shape}, Output shape: {y.shape}")
+
+# Test layer composition
+layer1 = Dense(3, 4)
+layer2 = Dense(4, 2)
+relu = ReLU()
+
+# Forward pass
+h1 = relu(layer1(x))
+output = layer2(h1)
+print(f"Final output: {output.data}")
+```
+
+## Systems Thinking Questions
+
+### Real-World Applications
+- **Computer Vision**: Dense layers process flattened image features in CNNs (like VGG, ResNet final layers)
+- **Natural Language Processing**: Dense layers transform word embeddings in transformers and RNNs
+- **Recommendation Systems**: Dense layers combine user and item features for preference prediction
+- **Scientific Computing**: Dense layers approximate complex functions in physics simulations and engineering
+
+### Mathematical Foundations
+- **Linear Transformation**: `y = Wx + b` where W is the weight matrix and b is the bias vector
+- **Matrix Multiplication**: Efficient batch processing through vectorized operations
+- **Weight Initialization**: Xavier/Glorot initialization prevents vanishing/exploding gradients
+- **Function Composition**: Networks as nested function calls: `f3(f2(f1(x)))`
+
+### Neural Network Building Blocks
+- **Modularity**: Layers as reusable components that can be combined in different ways
+- **Standardized Interface**: All layers follow the same input/output pattern for easy composition
+- **Shape Consistency**: Automatic handling of batch dimensions and shape transformations
+- **Nonlinearity**: Activation functions between layers enable learning of complex patterns
+
+### Implementation Patterns
+- **Class-based Design**: Layers as objects with state (weights) and behavior (forward pass)
+- **Initialization Strategy**: Proper weight initialization for stable training dynamics
+- **Error Handling**: Graceful handling of shape mismatches and invalid inputs
+- **Testing Philosophy**: Comprehensive testing of mathematical properties and edge cases
+
+## 🎉 Ready to Build?
+
+You're about to build the fundamental building blocks that power every neural network! Dense layers might seem simple, but they're the workhorses of deep learning—from the final layers of image classifiers to the core components of language models.
+
+Understanding how these simple linear transformations compose into complex intelligence is one of the most beautiful insights in machine learning. Take your time, understand the mathematics, and enjoy building the foundation of artificial intelligence!
+
+ 
+
+
+Choose your preferred way to engage with this module:
+
+````{grid} 1 2 3 3
+
+```{grid-item-card} 🚀 Launch Binder
+:link: https://mybinder.org/v2/gh/mlsysbook/TinyTorch/main?filepath=modules/04_layers/layers_dev.ipynb
+:class-header: bg-light
+
+Run this module interactively in your browser. No installation required!
+```
+
+```{grid-item-card} ⚡ Open in Colab  
+:link: https://colab.research.google.com/github/mlsysbook/TinyTorch/blob/main/modules/04_layers/layers_dev.ipynb
+:class-header: bg-light
+
+Use Google Colab for GPU access and cloud compute power.
+```
+
+```{grid-item-card} 📖 View Source
+:link: https://github.com/mlsysbook/TinyTorch/blob/main/modules/04_layers/layers_dev.py
+:class-header: bg-light
+
+Browse the Python source code and understand the implementation.
+```
+
+````
+
+```{admonition} 💾 Save Your Progress
+:class: tip
+**Binder sessions are temporary!** Download your completed notebook when done, or switch to local development for persistent work.
+
+```
+
+---
+
+<div class="prev-next-area">
+<a class="left-prev" href="../chapters/03_activations.html" title="previous page">← Previous Module</a>
+<a class="right-next" href="../chapters/05_dense.html" title="next page">Next Module →</a>
+</div>

modules/04_losses/ABOUT.md

@@ -0,0 +1,217 @@
+---
+title: "Loss Functions"
+description: "Implement MSE and CrossEntropy loss functions for training neural networks"
+difficulty: 2
+time_estimate: "3-4 hours"
+prerequisites: ["Tensor", "Activations", "Layers"]
+next_steps: ["Autograd"]
+learning_objectives:
+  - "Implement MSE loss for regression tasks with proper numerical stability"
+  - "Build CrossEntropy loss for classification with log-sum-exp trick"
+  - "Understand mathematical properties of loss functions and their gradients"
+  - "Recognize how loss functions connect model outputs to optimization objectives"
+  - "Apply appropriate loss functions for different machine learning tasks"
+---
+
+# 04. Losses
+
+**🏗️ FOUNDATION TIER** | Difficulty: ⭐⭐ (2/4) | Time: 3-4 hours
+
+## Overview
+
+Implement the mathematical functions that measure how wrong your model's predictions are. Loss functions are the bridge between model outputs and the optimization process—they define what "better" means and drive the entire learning process.
+
+## Learning Objectives
+
+By completing this module, you will be able to:
+
+1. **Implement MSE loss** for regression tasks with numerically stable computation
+2. **Build CrossEntropy loss** for classification using the log-sum-exp trick for numerical stability
+3. **Understand mathematical properties** of loss landscapes and their impact on optimization
+4. **Recognize the role** of loss functions in connecting predictions to training objectives
+5. **Apply appropriate losses** for regression, binary classification, and multi-class classification
+
+## Why This Matters
+
+### Production Context
+
+Loss functions are fundamental to all machine learning systems:
+
+- **Recommendation Systems** use MSE and ranking losses to learn user preferences
+- **Image Classification** relies on CrossEntropy loss for category prediction (ImageNet, CIFAR-10)
+- **Language Models** use CrossEntropy to predict next tokens in GPT, Claude, and all LLMs
+- **Autonomous Driving** combines multiple losses for perception, planning, and control
+
+### Historical Context
+
+Loss functions evolved with machine learning itself:
+
+- **Least Squares (1805)**: Gauss invented MSE for astronomical orbit predictions
+- **Maximum Likelihood (1912)**: Fisher formalized statistical foundations of loss functions
+- **CrossEntropy (1950s)**: Information theory brought entropy-based losses to ML
+- **Modern Deep Learning (2012+)**: Careful loss design enables training billion-parameter models
+
+## Build → Use → Understand
+
+This module follows the classic pedagogy for foundational concepts:
+
+1. **Build**: Implement MSE and CrossEntropy loss functions from mathematical definitions
+2. **Use**: Apply losses to regression and classification tasks, seeing how they drive learning
+3. **Understand**: Analyze loss landscapes, gradients, and numerical stability considerations
+
+## Implementation Guide
+
+### Step 1: MSE (Mean Squared Error) Loss
+
+Implement L2 loss for regression:
+
+```python
+class MSELoss:
+    """Mean Squared Error loss for regression."""
+    
+    def __call__(self, predictions: Tensor, targets: Tensor) -> Tensor:
+        """
+        Compute MSE: (1/n) * Σ(predictions - targets)²
+        
+        Args:
+            predictions: Model outputs
+            targets: Ground truth values
+        Returns:
+            Scalar loss value
+        """
+        diff = predictions - targets
+        squared = diff * diff
+        return squared.mean()
+```
+
+### Step 2: CrossEntropy Loss
+
+Implement log-likelihood loss for classification:
+
+```python
+class CrossEntropyLoss:
+    """CrossEntropy loss for multi-class classification."""
+    
+    def __call__(self, logits: Tensor, targets: Tensor) -> Tensor:
+        """
+        Compute CrossEntropy with log-sum-exp trick for numerical stability.
+        
+        Args:
+            logits: Raw model outputs (before softmax)
+            targets: Class indices or one-hot vectors
+        Returns:
+            Scalar loss value
+        """
+        # Log-sum-exp trick for numerical stability
+        max_logits = logits.max(axis=1, keepdims=True)
+        exp_logits = (logits - max_logits).exp()
+        log_probs = logits - max_logits - exp_logits.sum(axis=1, keepdims=True).log()
+        
+        # Negative log-likelihood
+        return -log_probs.mean()
+```
+
+### Step 3: Loss Function Properties
+
+Understand key mathematical properties:
+
+- **Convexity**: MSE is convex; CrossEntropy is convex in logits
+- **Gradients**: Smooth gradients enable effective optimization
+- **Scale**: Loss magnitude affects learning rate tuning
+- **Numerical Stability**: Requires careful implementation (log-sum-exp trick)
+
+## Testing
+
+### Inline Tests
+
+The module includes immediate feedback:
+
+```python
+# Example inline test output
+🔬 Unit Test: MSE Loss...
+✅ MSE computes squared error correctly
+✅ MSE gradient flows properly
+✅ MSE handles batch dimensions correctly
+📈 Progress: MSE Loss ✓
+
+🔬 Unit Test: CrossEntropy Loss...
+✅ CrossEntropy numerically stable
+✅ CrossEntropy matches PyTorch implementation
+✅ CrossEntropy handles multi-class problems
+📈 Progress: CrossEntropy Loss ✓
+```
+
+### Export and Validate
+
+```bash
+# Export to package
+tito export --module 04_losses
+
+# Run test suite
+tito test --module 04_losses
+```
+
+## Where This Code Lives
+
+```
+tinytorch/
+├── nn/
+│   └── losses.py          # MSELoss, CrossEntropyLoss
+└── core/
+    └── tensor.py          # Underlying tensor operations
+```
+
+After export, use as:
+```python
+from tinytorch.nn import MSELoss, CrossEntropyLoss
+
+# For regression
+mse = MSELoss()
+loss = mse(predictions, targets)
+
+# For classification
+ce = CrossEntropyLoss()
+loss = ce(logits, labels)
+```
+
+## Systems Thinking Questions
+
+1. **Why does CrossEntropy require the log-sum-exp trick?** What numerical instability occurs without it?
+
+2. **How does loss scale affect learning?** If you multiply your loss by 100, what happens to gradients and learning?
+
+3. **Why do we use MSE for regression but CrossEntropy for classification?** What makes each appropriate for its task?
+
+4. **How do loss functions connect to probability theory?** What is the relationship between CrossEntropy and maximum likelihood?
+
+5. **What happens if you use the wrong loss function?** Try MSE for classification or CrossEntropy for regression—what breaks?
+
+## Real-World Connections
+
+### Industry Applications
+
+- **Computer Vision**: CrossEntropy trains all classification models (ResNet, EfficientNet, Vision Transformers)
+- **NLP**: CrossEntropy is the foundation of all language models (GPT, BERT, T5)
+- **Recommendation**: MSE and ranking losses optimize Netflix, Spotify, YouTube recommendations
+- **Robotics**: MSE trains continuous control policies for manipulation and navigation
+
+### Production Considerations
+
+- **Numerical Stability**: Log-sum-exp trick prevents overflow/underflow in production systems
+- **Loss Scaling**: Careful scaling enables mixed-precision training (FP16/BF16)
+- **Weighted Losses**: Class weights handle imbalanced datasets in production
+- **Custom Losses**: Production systems often combine multiple loss terms
+
+## What's Next?
+
+Now that you can measure prediction quality, you're ready for **Module 05: Autograd** where you'll learn how to automatically compute gradients of these loss functions, enabling the optimization that drives all of machine learning.
+
+**Preview**: Autograd will automatically compute ∂Loss/∂weights for any loss function you build, making training possible without manual gradient derivations!
+
+---
+
+**Need Help?**
+- Check the inline tests in `modules/04_losses/losses_dev.py`
+- Review mathematical derivations in the module comments
+- Compare your implementation against PyTorch's losses
+

modules/05_autograd/ABOUT.md

@@ -0,0 +1,260 @@
+---
+title: "Autograd"
+description: "Automatic differentiation engine for gradient computation"
+difficulty: "⭐⭐⭐⭐"
+time_estimate: "8-10 hours"
+prerequisites: []
+next_steps: []
+learning_objectives: []
+---
+
+# 05. Autograd
+
+**🏗️ FOUNDATION TIER** | Difficulty: ⭐⭐⭐⭐ (4/4) | Time: 6-8 hours
+
+## Overview
+
+Build the automatic differentiation engine that makes neural network training possible. This module implements the mathematical foundation that enables backpropagation—transforming TinyTorch from a static computation library into a dynamic, trainable ML framework.
+
+## Learning Objectives
+
+By the end of this module, you will be able to:
+
+- **Master automatic differentiation theory**: Understand computational graphs, chain rule application, and gradient flow
+- **Implement gradient tracking systems**: Build the Variable class that automatically computes and accumulates gradients
+- **Create differentiable operations**: Extend all mathematical operations to support backward propagation
+- **Apply backpropagation algorithms**: Implement the gradient computation that enables neural network optimization
+- **Integrate with ML systems**: Connect automatic differentiation with layers, networks, and training algorithms
+
+## Build → Use → Analyze
+
+This module follows TinyTorch's **Build → Use → Analyze** framework:
+
+1. **Build**: Implement Variable class and gradient computation system using mathematical differentiation rules
+2. **Use**: Apply automatic differentiation to complex expressions and neural network forward passes
+3. **Analyze**: Understand computational graph construction, memory usage, and performance characteristics of autodiff systems
+
+## Implementation Guide
+
+### Automatic Differentiation System
+```python
+# Variables track gradients automatically
+x = Variable(5.0, requires_grad=True)
+y = Variable(3.0, requires_grad=True)
+
+# Complex mathematical expressions
+z = x**2 + 2*x*y + y**3
+print(f"f(x,y) = {z.data}")  # Forward pass result
+
+# Automatic gradient computation
+z.backward()
+print(f"df/dx = {x.grad}")  # ∂f/∂x = 2x + 2y = 16
+print(f"df/dy = {y.grad}")  # ∂f/∂y = 2x + 3y² = 37
+```
+
+### Neural Network Integration
+```python
+# Seamless integration with existing TinyTorch components
+from tinytorch.core.layers import Dense
+from tinytorch.core.activations import ReLU
+
+# Create differentiable network
+x = Variable([[1.0, 2.0, 3.0]], requires_grad=True)
+layer1 = Dense(3, 4)  # Weights automatically become Variables
+layer2 = Dense(4, 1)
+relu = ReLU()
+
+# Forward pass builds computational graph
+h1 = relu(layer1(x))
+output = layer2(h1)
+loss = output.sum()
+
+# Backward pass computes all gradients
+loss.backward()
+
+# All parameters now have gradients
+print(f"Layer 1 weight gradients: {layer1.weights.grad.shape}")
+print(f"Layer 2 bias gradients: {layer2.bias.grad.shape}")
+print(f"Input gradients: {x.grad.shape}")
+```
+
+### Computational Graph Construction
+```python
+# Automatic graph building for complex operations
+def complex_function(x, y):
+    a = x * y          # Multiplication node
+    b = x + y          # Addition node  
+    c = a / b          # Division node
+    return c.sin()     # Trigonometric node
+
+x = Variable(2.0, requires_grad=True)
+y = Variable(3.0, requires_grad=True)
+result = complex_function(x, y)
+
+# Chain rule applied automatically through entire graph
+result.backward()
+print(f"Complex gradient dx: {x.grad}")
+print(f"Complex gradient dy: {y.grad}")
+```
+
+## Getting Started
+
+### Prerequisites
+Ensure you understand the mathematical building blocks:
+
+   ```bash
+# Activate TinyTorch environment
+   source bin/activate-tinytorch.sh
+
+# Verify prerequisite modules
+tito test --module tensor
+tito test --module activations
+tito test --module layers
+   ```
+
+### Development Workflow
+1. **Open the development file**: `modules/08_autograd/autograd_dev.py`
+2. **Implement Variable class**: Create gradient tracking wrapper around Tensors
+3. **Add basic operations**: Implement differentiable arithmetic (add, multiply, power)
+4. **Build backward propagation**: Implement chain rule for gradient computation
+5. **Extend to all operations**: Add gradients for activations, matrix operations, etc.
+6. **Export and verify**: `tito export --module autograd && tito test --module autograd`
+
+## Testing
+
+### Comprehensive Test Suite
+Run the full test suite to verify mathematical correctness:
+
+```bash
+# TinyTorch CLI (recommended)
+tito test --module autograd
+
+# Direct pytest execution
+python -m pytest tests/ -k autograd -v
+```
+
+### Test Coverage Areas
+- ✅ **Variable Creation**: Test gradient tracking initialization and properties
+- ✅ **Basic Operations**: Verify arithmetic operations compute correct gradients
+- ✅ **Chain Rule**: Ensure composite functions apply chain rule correctly
+- ✅ **Backpropagation**: Test gradient flow through complex computational graphs
+- ✅ **Neural Network Integration**: Verify seamless operation with layers and activations
+
+### Inline Testing & Mathematical Verification
+The module includes comprehensive mathematical validation:
+```python
+# Example inline test output
+🔬 Unit Test: Variable gradient tracking...
+✅ Variable creation with gradient tracking
+✅ Leaf variables correctly identified
+✅ Gradient accumulation works correctly
+📈 Progress: Variable System ✓
+
+# Mathematical verification
+🔬 Unit Test: Chain rule implementation...
+✅ f(x) = x² → df/dx = 2x ✓
+✅ f(x,y) = xy → df/dx = y, df/dy = x ✓
+✅ Complex compositions follow chain rule ✓
+📈 Progress: Differentiation Rules ✓
+```
+
+### Manual Testing Examples
+```python
+from autograd_dev import Variable
+import math
+
+# Test basic differentiation rules
+x = Variable(3.0, requires_grad=True)
+y = x**2
+y.backward()
+print(f"d(x²)/dx at x=3: {x.grad}")  # Should be 6
+
+# Test chain rule
+x = Variable(2.0, requires_grad=True)
+y = Variable(3.0, requires_grad=True)
+z = (x + y) * (x - y)  # Difference of squares
+z.backward()
+print(f"d/dx = {x.grad}")  # Should be 2x = 4
+print(f"d/dy = {y.grad}")  # Should be -2y = -6
+
+# Test with transcendental functions
+x = Variable(1.0, requires_grad=True)
+y = x.exp().log()  # Should equal x
+y.backward()
+print(f"d(exp(log(x)))/dx: {x.grad}")  # Should be 1
+```
+
+## Systems Thinking Questions
+
+### Real-World Applications
+- **Deep Learning Frameworks**: PyTorch, TensorFlow, JAX all use automatic differentiation for training
+- **Scientific Computing**: Automatic differentiation enables gradient-based optimization in physics, chemistry, engineering
+- **Financial Modeling**: Risk analysis and portfolio optimization use autodiff for sensitivity analysis
+- **Robotics**: Control systems use gradients for trajectory optimization and inverse kinematics
+
+### Mathematical Foundations
+- **Chain Rule**: ∂f/∂x = (∂f/∂u)(∂u/∂x) for composite functions f(u(x))
+- **Computational Graphs**: Directed acyclic graphs representing function composition
+- **Forward Mode vs Reverse Mode**: Different autodiff strategies with different computational complexities
+- **Gradient Accumulation**: Handling multiple computational paths to same variable
+
+### Automatic Differentiation Theory
+- **Dual Numbers**: Mathematical foundation using infinitesimals for forward-mode AD
+- **Reverse Accumulation**: Backpropagation as reverse-mode automatic differentiation
+- **Higher-Order Derivatives**: Computing gradients of gradients for advanced optimization
+- **Jacobian Computation**: Efficient computation of vector-valued function gradients
+
+### Implementation Patterns
+- **Gradient Function Storage**: Each operation stores its backward function in the computational graph
+- **Topological Sorting**: Ordering gradient computation to respect dependencies
+- **Memory Management**: Efficient storage and cleanup of intermediate values
+- **Numerical Stability**: Handling edge cases in gradient computation
+
+## 🎉 Ready to Build?
+
+You're about to implement the mathematical foundation that makes modern AI possible! Automatic differentiation is the invisible engine that powers every neural network, from simple classifiers to GPT and beyond.
+
+Understanding autodiff from first principles—implementing the Variable class and chain rule yourself—will give you deep insight into how deep learning really works. This is where mathematics meets software engineering to create something truly powerful. Take your time, understand each gradient rule, and enjoy building the heart of machine learning!
+
+ 
+
+
+Choose your preferred way to engage with this module:
+
+````{grid} 1 2 3 3
+
+```{grid-item-card} 🚀 Launch Binder
+:link: https://mybinder.org/v2/gh/mlsysbook/TinyTorch/main?filepath=modules/09_autograd/autograd_dev.ipynb
+:class-header: bg-light
+
+Run this module interactively in your browser. No installation required!
+```
+
+```{grid-item-card} ⚡ Open in Colab  
+:link: https://colab.research.google.com/github/mlsysbook/TinyTorch/blob/main/modules/09_autograd/autograd_dev.ipynb
+:class-header: bg-light
+
+Use Google Colab for GPU access and cloud compute power.
+```
+
+```{grid-item-card} 📖 View Source
+:link: https://github.com/mlsysbook/TinyTorch/blob/main/modules/09_autograd/autograd_dev.py
+:class-header: bg-light
+
+Browse the Python source code and understand the implementation.
+```
+
+````
+
+```{admonition} 💾 Save Your Progress
+:class: tip
+**Binder sessions are temporary!** Download your completed notebook when done, or switch to local development for persistent work.
+
+```
+
+---
+
+<div class="prev-next-area">
+<a class="left-prev" href="../chapters/08_dataloader.html" title="previous page">← Previous Module</a>
+<a class="right-next" href="../chapters/10_autograd.html" title="next page">Next Module →</a>
+</div>

modules/06_optimizers/ABOUT.md

@@ -0,0 +1,267 @@
+---
+title: "Optimizers"
+description: "Gradient-based parameter optimization algorithms"
+difficulty: "⭐⭐⭐⭐"
+time_estimate: "6-8 hours"
+prerequisites: []
+next_steps: []
+learning_objectives: []
+---
+
+# 06. Optimizers
+
+**🏗️ FOUNDATION TIER** | Difficulty: ⭐⭐⭐⭐ (4/4) | Time: 6-8 hours
+
+## Overview
+
+Build intelligent optimization algorithms that enable effective neural network training. This module implements the learning algorithms that power modern AI—from basic gradient descent to advanced adaptive methods that make training large-scale models possible.
+
+## Learning Objectives
+
+By the end of this module, you will be able to:
+
+- **Master gradient-based optimization theory**: Understand how gradients guide parameter updates and the mathematical foundations of learning
+- **Implement core optimization algorithms**: Build SGD, momentum, and Adam optimizers from mathematical first principles
+- **Design learning rate strategies**: Create scheduling systems that balance convergence speed with training stability
+- **Apply optimization in practice**: Use optimizers effectively in complete training workflows with real neural networks
+- **Analyze optimization dynamics**: Compare algorithm behavior, convergence patterns, and performance characteristics
+
+## Build → Use → Optimize
+
+This module follows TinyTorch's **Build → Use → Optimize** framework:
+
+1. **Build**: Implement gradient descent, SGD with momentum, Adam optimizer, and learning rate scheduling from mathematical foundations
+2. **Use**: Apply optimization algorithms to train neural networks and solve real optimization problems
+3. **Optimize**: Analyze convergence behavior, compare algorithm performance, and tune hyperparameters for optimal training
+
+## Implementation Guide
+
+### Core Optimization Algorithms
+```python
+# Gradient descent foundation
+def gradient_descent_step(parameter, learning_rate):
+    parameter.data = parameter.data - learning_rate * parameter.grad.data
+
+# SGD with momentum for accelerated convergence
+sgd = SGD(parameters=[w1, w2, bias], learning_rate=0.01, momentum=0.9)
+sgd.zero_grad()  # Clear previous gradients
+loss.backward()  # Compute new gradients
+sgd.step()       # Update parameters
+
+# Adam optimizer with adaptive learning rates
+adam = Adam(parameters=[w1, w2, bias], learning_rate=0.001, beta1=0.9, beta2=0.999)
+adam.zero_grad()
+loss.backward()
+adam.step()      # Adaptive updates per parameter
+```
+
+### Learning Rate Scheduling Systems
+```python
+# Strategic learning rate adjustment
+scheduler = StepLR(optimizer, step_size=10, gamma=0.1)
+
+# Training loop with scheduling
+for epoch in range(num_epochs):
+    for batch in dataloader:
+        optimizer.zero_grad()
+        loss = criterion(model(batch.inputs), batch.targets)
+        loss.backward()
+        optimizer.step()
+    
+    scheduler.step()  # Adjust learning rate each epoch
+    print(f"Epoch {epoch}, LR: {scheduler.get_last_lr()}")
+```
+
+### Complete Training Integration
+```python
+# Modern training workflow
+model = Sequential([Dense(784, 128), ReLU(), Dense(128, 10)])
+optimizer = Adam(model.parameters(), learning_rate=0.001)
+scheduler = StepLR(optimizer, step_size=20, gamma=0.5)
+
+# Training loop with optimization
+for epoch in range(num_epochs):
+    for batch_inputs, batch_targets in dataloader:
+        # Forward pass
+        predictions = model(batch_inputs)
+        loss = criterion(predictions, batch_targets)
+        
+        # Optimization step
+        optimizer.zero_grad()  # Clear gradients
+        loss.backward()        # Compute gradients
+        optimizer.step()       # Update parameters
+    
+    scheduler.step()  # Adjust learning rate
+```
+
+### Optimization Algorithm Implementations
+- **Gradient Descent**: Basic parameter update rule using gradients
+- **SGD with Momentum**: Velocity accumulation for smoother convergence
+- **Adam Optimizer**: Adaptive learning rates with bias correction
+- **Learning Rate Scheduling**: Strategic adjustment during training
+
+## Getting Started
+
+### Prerequisites
+Ensure you understand the mathematical foundations:
+
+```bash
+# Activate TinyTorch environment
+source bin/activate-tinytorch.sh
+
+# Verify prerequisite modules
+tito test --module tensor
+tito test --module autograd
+```
+
+### Development Workflow
+1. **Open the development file**: `modules/09_optimizers/optimizers_dev.py`
+2. **Implement gradient descent**: Start with basic parameter update mechanics
+3. **Build SGD with momentum**: Add velocity accumulation for acceleration
+4. **Create Adam optimizer**: Implement adaptive learning rates with moment estimation
+5. **Add learning rate scheduling**: Build strategic learning rate adjustment systems
+6. **Export and verify**: `tito export --module optimizers && tito test --module optimizers`
+
+## Testing
+
+### Comprehensive Test Suite
+Run the full test suite to verify optimization algorithm correctness:
+
+```bash
+# TinyTorch CLI (recommended)
+tito test --module optimizers
+
+# Direct pytest execution
+python -m pytest tests/ -k optimizers -v
+```
+
+### Test Coverage Areas
+- ✅ **Algorithm Implementation**: Verify SGD, momentum, and Adam compute correct parameter updates
+- ✅ **Mathematical Correctness**: Test against analytical solutions for convex optimization
+- ✅ **State Management**: Ensure proper momentum and moment estimation tracking
+- ✅ **Learning Rate Scheduling**: Verify step decay and scheduling functionality
+- ✅ **Training Integration**: Test optimizers in complete neural network training workflows
+
+### Inline Testing & Convergence Analysis
+The module includes comprehensive mathematical validation and convergence visualization:
+```python
+# Example inline test output
+🔬 Unit Test: SGD with momentum...
+✅ Parameter updates follow momentum equations
+✅ Velocity accumulation works correctly
+✅ Convergence achieved on test function
+📈 Progress: SGD with Momentum ✓
+
+# Optimization analysis
+🔬 Unit Test: Adam optimizer...
+✅ First moment estimation (m_t) computed correctly
+✅ Second moment estimation (v_t) computed correctly  
+✅ Bias correction applied properly
+✅ Adaptive learning rates working
+📈 Progress: Adam Optimizer ✓
+```
+
+### Manual Testing Examples
+```python
+from optimizers_dev import SGD, Adam, StepLR
+from autograd_dev import Variable
+
+# Test SGD on simple quadratic function
+x = Variable(10.0, requires_grad=True)
+sgd = SGD([x], learning_rate=0.1, momentum=0.9)
+
+for step in range(100):
+    sgd.zero_grad()
+    loss = x**2  # Minimize f(x) = x²
+    loss.backward()
+    sgd.step()
+    if step % 10 == 0:
+        print(f"Step {step}: x = {x.data:.4f}, loss = {loss.data:.4f}")
+
+# Test Adam convergence
+x = Variable([2.0, -3.0], requires_grad=True)
+adam = Adam([x], learning_rate=0.01)
+
+for step in range(50):
+    adam.zero_grad()
+    loss = (x[0]**2 + x[1]**2).sum()  # Minimize ||x||²
+    loss.backward()
+    adam.step()
+    if step % 10 == 0:
+        print(f"Step {step}: x = {x.data}, loss = {loss.data:.6f}")
+```
+
+## Systems Thinking Questions
+
+### Real-World Applications
+- **Large Language Models**: GPT, BERT training relies on Adam optimization for stable convergence
+- **Computer Vision**: ResNet, Vision Transformer training uses SGD with momentum for best final performance
+- **Recommendation Systems**: Online learning systems use adaptive optimizers for continuous model updates
+- **Reinforcement Learning**: Policy gradient methods depend on careful optimizer choice and learning rate tuning
+
+### Mathematical Foundations
+- **Gradient Descent**: θ_{t+1} = θ_t - α∇L(θ_t) where α is learning rate and ∇L is loss gradient
+- **Momentum**: v_{t+1} = βv_t + ∇L(θ_t), θ_{t+1} = θ_t - αv_{t+1} for accelerated convergence
+- **Adam**: Combines momentum with adaptive learning rates using first and second moment estimates
+- **Learning Rate Scheduling**: Strategic decay schedules balance exploration and exploitation
+
+### Optimization Theory
+- **Convex Optimization**: Guarantees global minimum for convex loss functions
+- **Non-convex Optimization**: Neural networks have complex loss landscapes with local minima
+- **Convergence Analysis**: Understanding when and why optimization algorithms reach good solutions
+- **Hyperparameter Sensitivity**: Learning rate is often the most critical hyperparameter
+
+### Performance Characteristics
+- **SGD**: Memory efficient, works well with large batches, good final performance
+- **Adam**: Fast initial convergence, works with small batches, requires more memory
+- **Learning Rate Schedules**: Often crucial for achieving best performance
+- **Algorithm Selection**: Problem-dependent choice based on data, model, and computational constraints
+
+## 🎉 Ready to Build?
+
+You're about to implement the algorithms that power all of modern AI! From the neural networks that recognize your voice to the language models that write code, they all depend on the optimization algorithms you're building.
+
+Understanding these algorithms from first principles—implementing momentum physics and adaptive learning rates yourself—will give you deep insight into why some training works and some doesn't. Take your time with the mathematics, test thoroughly, and enjoy building the intelligence behind intelligent systems!
+
+
+
+
+Choose your preferred way to engage with this module:
+
+````{grid} 1 2 3 3
+
+```{grid-item-card} 🚀 Launch Binder
+:link: https://mybinder.org/v2/gh/mlsysbook/TinyTorch/main?filepath=modules/10_optimizers/optimizers_dev.ipynb
+:class-header: bg-light
+
+Run this module interactively in your browser. No installation required!
+```
+
+```{grid-item-card} ⚡ Open in Colab  
+:link: https://colab.research.google.com/github/mlsysbook/TinyTorch/blob/main/modules/10_optimizers/optimizers_dev.ipynb
+:class-header: bg-light
+
+Use Google Colab for GPU access and cloud compute power.
+```
+
+```{grid-item-card} 📖 View Source
+:link: https://github.com/mlsysbook/TinyTorch/blob/main/modules/10_optimizers/optimizers_dev.py
+:class-header: bg-light
+
+Browse the Python source code and understand the implementation.
+```
+
+````
+
+```{admonition} 💾 Save Your Progress
+:class: tip
+**Binder sessions are temporary!** Download your completed notebook when done, or switch to local development for persistent work.
+
+```
+
+---
+
+<div class="prev-next-area">
+<a class="left-prev" href="../chapters/09_dataloader.html" title="previous page">← Previous Module</a>
+<a class="right-next" href="../chapters/11_optimizers.html" title="next page">Next Module →</a>
+</div>

modules/07_training/ABOUT.md

@@ -0,0 +1,353 @@
+---
+title: "Training"
+description: "Neural network training loops, loss functions, and metrics"
+difficulty: "⭐⭐⭐⭐"
+time_estimate: "8-10 hours"
+prerequisites: []
+next_steps: []
+learning_objectives: []
+---
+
+# 07. Training
+
+**🏗️ FOUNDATION TIER** | Difficulty: ⭐⭐⭐⭐ (4/4) | Time: 8-10 hours
+
+## Overview
+
+Build the complete training pipeline that brings all TinyTorch components together. This capstone module orchestrates data loading, model forward passes, loss computation, backpropagation, and optimization into the end-to-end training workflows that power modern AI systems.
+
+## Learning Objectives
+
+By the end of this module, you will be able to:
+
+- **Design complete training architectures**: Orchestrate all ML components into cohesive training systems
+- **Implement essential loss functions**: Build MSE, CrossEntropy, and BinaryCrossEntropy from mathematical foundations
+- **Create evaluation frameworks**: Develop metrics systems for classification, regression, and model performance assessment
+- **Build production training loops**: Implement robust training workflows with validation, logging, and progress tracking
+- **Master training dynamics**: Understand convergence, overfitting, generalization, and optimization in real scenarios
+
+## Build → Use → Optimize
+
+This module follows TinyTorch's **Build → Use → Optimize** framework:
+
+1. **Build**: Implement loss functions, evaluation metrics, and complete training orchestration systems
+2. **Use**: Train end-to-end neural networks on real datasets with full pipeline automation
+3. **Optimize**: Analyze training dynamics, debug convergence issues, and optimize training performance for production
+
+## NEW: Model Checkpointing & Evaluation Tools
+
+### Complete Training with Checkpointing
+This module now includes production features for our north star goal:
+
+```python
+from tinytorch.core.training import Trainer, CrossEntropyLoss, Accuracy
+from tinytorch.core.training import evaluate_model, plot_training_history
+
+# Train with automatic model checkpointing
+trainer = Trainer(model, CrossEntropyLoss(), Adam(lr=0.001), [Accuracy()])
+history = trainer.fit(
+    train_loader,
+    val_dataloader=test_loader,
+    epochs=30,
+    save_best=True,                    # ✅ NEW: Saves best model automatically
+    checkpoint_path='best_model.pkl',  # ✅ NEW: Checkpoint location
+    early_stopping_patience=5          # ✅ NEW: Stop if no improvement
+)
+
+# Load best model after training
+trainer.load_checkpoint('best_model.pkl')
+print(f"✅ Restored best model from epoch {trainer.current_epoch}")
+
+# Evaluate with comprehensive metrics
+results = evaluate_model(model, test_loader)
+print(f"Test Accuracy: {results['accuracy']:.2%}")
+print(f"Confusion Matrix:\n{results['confusion_matrix']}")
+
+# Visualize training progress
+plot_training_history(history)  # Shows loss and accuracy curves
+```
+
+### What's New in This Module
+- ✅ **`save_checkpoint()`/`load_checkpoint()`**: Save and restore model state during training
+- ✅ **`save_best=True`**: Automatically saves model with best validation performance
+- ✅ **`early_stopping_patience`**: Stop training when validation loss stops improving
+- ✅ **`evaluate_model()`**: Comprehensive model evaluation with confusion matrix
+- ✅ **`plot_training_history()`**: Visualize training and validation curves
+- ✅ **`compute_confusion_matrix()`**: Analyze classification errors by class
+
+## Implementation Guide
+
+### Complete Training Pipeline
+```python
+# End-to-end training system
+from tinytorch.core.training import Trainer
+from tinytorch.core.losses import CrossEntropyLoss
+from tinytorch.core.metrics import Accuracy
+
+# Define complete model architecture
+model = Sequential([
+    Dense(784, 128), ReLU(),
+    Dense(128, 64), ReLU(),
+    Dense(64, 10), Softmax()
+])
+
+# Configure training components
+optimizer = Adam(model.parameters(), learning_rate=0.001)
+loss_fn = CrossEntropyLoss()
+metrics = [Accuracy()]
+
+# Create and configure trainer
+trainer = Trainer(
+    model=model,
+    optimizer=optimizer, 
+    loss_fn=loss_fn,
+    metrics=metrics
+)
+
+# Train with comprehensive monitoring
+history = trainer.fit(
+    train_dataloader=train_loader,
+    val_dataloader=val_loader,
+    epochs=50,
+    verbose=True
+)
+```
+
+### Loss Function Library
+```python
+# Regression loss for continuous targets
+mse_loss = MeanSquaredError()
+regression_loss = mse_loss(predictions, continuous_targets)
+
+# Multi-class classification loss
+ce_loss = CrossEntropyLoss()
+classification_loss = ce_loss(logits, class_indices)
+
+# Binary classification loss
+bce_loss = BinaryCrossEntropyLoss()
+binary_loss = bce_loss(sigmoid_outputs, binary_labels)
+
+# All losses support batch processing and gradient computation
+loss.backward()  # Automatic differentiation integration
+```
+
+### Evaluation Metrics System
+```python
+# Classification performance measurement
+accuracy = Accuracy()
+acc_score = accuracy(predictions, true_labels)  # Returns 0.0 to 1.0
+
+# Regression error measurement  
+mae = MeanAbsoluteError()
+error = mae(predictions, targets)
+
+# Extensible metric framework
+class CustomMetric:
+    def __call__(self, y_pred, y_true):
+        # Implement custom evaluation logic
+        return custom_score
+
+metrics = [Accuracy(), CustomMetric()]
+trainer = Trainer(model, optimizer, loss_fn, metrics)
+```
+
+### Real-World Training Workflows
+```python
+# Train on CIFAR-10 with full pipeline
+from tinytorch.core.dataloader import CIFAR10Dataset, DataLoader
+
+# Load and prepare data
+train_dataset = CIFAR10Dataset("data/cifar10/", train=True, download=True)
+train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)
+val_loader = DataLoader(val_dataset, batch_size=32, shuffle=False)
+
+# Configure CNN for computer vision
+cnn_model = Sequential([
+    Conv2D(3, 16, kernel_size=3), ReLU(),
+    MaxPool2D(kernel_size=2),
+    Conv2D(16, 32, kernel_size=3), ReLU(),
+    Flatten(),
+    Dense(32 * 13 * 13, 128), ReLU(),
+    Dense(128, 10)
+])
+
+# Train with monitoring and validation
+trainer = Trainer(cnn_model, Adam(cnn_model.parameters()), CrossEntropyLoss(), [Accuracy()])
+history = trainer.fit(train_loader, val_loader, epochs=100)
+
+# Analyze training results
+print(f"Final train accuracy: {history['train_accuracy'][-1]:.4f}")
+print(f"Final val accuracy: {history['val_accuracy'][-1]:.4f}")
+```
+
+## Getting Started
+
+### Prerequisites
+Ensure you have completed the entire TinyTorch foundation:
+
+```bash
+# Activate TinyTorch environment
+source bin/activate-tinytorch.sh
+
+# Verify all prerequisite modules (this is the capstone!)
+tito test --module tensor
+tito test --module activations  
+tito test --module layers
+tito test --module networks
+tito test --module dataloader
+tito test --module autograd
+tito test --module optimizers
+```
+
+### Development Workflow
+1. **Open the development file**: `modules/10_training/training_dev.py`
+2. **Implement loss functions**: Build MSE, CrossEntropy, and BinaryCrossEntropy with proper gradients
+3. **Create metrics system**: Develop Accuracy and extensible evaluation framework
+4. **Build Trainer class**: Orchestrate training loop with validation and monitoring
+5. **Test end-to-end training**: Apply complete pipeline to real datasets and problems
+6. **Export and verify**: `tito export --module training && tito test --module training`
+
+## Testing
+
+### Comprehensive Test Suite
+Run the full test suite to verify complete training system functionality:
+
+```bash
+# TinyTorch CLI (recommended)
+tito test --module training
+
+# Direct pytest execution
+python -m pytest tests/ -k training -v
+```
+
+### Test Coverage Areas
+- ✅ **Loss Function Implementation**: Verify mathematical correctness and gradient computation
+- ✅ **Metrics System**: Test accuracy calculation and extensible framework
+- ✅ **Training Loop Orchestration**: Ensure proper coordination of all components
+- ✅ **End-to-End Training**: Verify complete workflows on real datasets
+- ✅ **Convergence Analysis**: Test training dynamics and optimization behavior
+
+### Inline Testing & Training Analysis
+The module includes comprehensive training validation and convergence monitoring:
+```python
+# Example inline test output
+🔬 Unit Test: CrossEntropy loss function...
+✅ Mathematical correctness verified
+✅ Gradient computation working
+✅ Batch processing supported
+📈 Progress: Loss Functions ✓
+
+# Training monitoring
+🔬 Unit Test: Complete training pipeline...
+✅ Trainer orchestrates all components correctly
+✅ Training loop converges on test problem
+✅ Validation monitoring working
+📈 Progress: End-to-End Training ✓
+
+# Real dataset training
+📊 Training on CIFAR-10 subset...
+Epoch 1/10: train_loss=2.345, train_acc=0.234, val_loss=2.123, val_acc=0.278
+Epoch 5/10: train_loss=1.456, train_acc=0.567, val_loss=1.543, val_acc=0.523
+✅ Model converging successfully
+```
+
+### Manual Testing Examples
+```python
+from training_dev import Trainer, CrossEntropyLoss, Accuracy
+from networks_dev import Sequential
+from layers_dev import Dense
+from activations_dev import ReLU, Softmax
+from optimizers_dev import Adam
+
+# Test complete training on synthetic data
+model = Sequential([Dense(4, 8), ReLU(), Dense(8, 3), Softmax()])
+optimizer = Adam(model.parameters(), learning_rate=0.01)
+loss_fn = CrossEntropyLoss()
+metrics = [Accuracy()]
+
+trainer = Trainer(model, optimizer, loss_fn, metrics)
+
+# Create simple dataset
+from dataloader_dev import SimpleDataset, DataLoader
+train_dataset = SimpleDataset(size=1000, num_features=4, num_classes=3)
+train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)
+
+# Train and monitor
+history = trainer.fit(train_loader, epochs=20, verbose=True)
+print(f"Training completed. Final accuracy: {history['train_accuracy'][-1]:.4f}")
+```
+
+## Systems Thinking Questions
+
+### Real-World Applications
+- **Production ML Systems**: Companies like Netflix, Google use similar training pipelines for recommendation and search systems
+- **Research Workflows**: Academic researchers use training frameworks like this for experimental model development
+- **MLOps Platforms**: Production training systems extend these patterns with distributed computing and monitoring
+- **Edge AI Training**: Federated learning systems use similar orchestration patterns across distributed devices
+
+### Training System Architecture
+- **Loss Functions**: Mathematical objectives that define what the model should learn
+- **Metrics**: Human-interpretable measures of model performance for monitoring and decision-making
+- **Training Loop**: Orchestration pattern that coordinates data loading, forward passes, backward passes, and optimization
+- **Validation Strategy**: Techniques for monitoring generalization and preventing overfitting
+
+### Machine Learning Engineering
+- **Training Dynamics**: Understanding convergence, overfitting, underfitting, and optimization landscapes
+- **Hyperparameter Tuning**: Systematic approaches to learning rate, batch size, and architecture selection
+- **Debugging Training**: Common failure modes and diagnostic techniques for training issues
+- **Production Considerations**: Scalability, monitoring, reproducibility, and deployment readiness
+
+### Systems Integration Patterns
+- **Component Orchestration**: How to coordinate multiple ML components into cohesive systems
+- **Error Handling**: Robust handling of training failures, data issues, and convergence problems
+- **Monitoring and Logging**: Tracking training progress, performance metrics, and system health
+- **Extensibility**: Design patterns that enable easy addition of new losses, metrics, and training strategies
+
+## 🎉 Ready to Build?
+
+You're about to complete the TinyTorch framework by building the training system that brings everything together! This is where all your hard work on tensors, layers, networks, data loading, gradients, and optimization culminates in a complete ML system.
+
+Training is the heart of machine learning—it's where models learn from data and become intelligent. You're building the same patterns used to train GPT, train computer vision models, and power production AI systems. Take your time, understand how all the pieces fit together, and enjoy creating something truly powerful!
+
+ 
+
+
+Choose your preferred way to engage with this module:
+
+````{grid} 1 2 3 3
+
+```{grid-item-card} 🚀 Launch Binder
+:link: https://mybinder.org/v2/gh/mlsysbook/TinyTorch/main?filepath=modules/11_training/training_dev.ipynb
+:class-header: bg-light
+
+Run this module interactively in your browser. No installation required!
+```
+
+```{grid-item-card} ⚡ Open in Colab  
+:link: https://colab.research.google.com/github/mlsysbook/TinyTorch/blob/main/modules/11_training/training_dev.ipynb
+:class-header: bg-light
+
+Use Google Colab for GPU access and cloud compute power.
+```
+
+```{grid-item-card} 📖 View Source
+:link: https://github.com/mlsysbook/TinyTorch/blob/main/modules/11_training/training_dev.py
+:class-header: bg-light
+
+Browse the Python source code and understand the implementation.
+```
+
+````
+
+```{admonition} 💾 Save Your Progress
+:class: tip
+**Binder sessions are temporary!** Download your completed notebook when done, or switch to local development for persistent work.
+
+```
+
+---
+
+<div class="prev-next-area">
+<a class="left-prev" href="../chapters/10_autograd.html" title="previous page">← Previous Module</a>
+<a class="right-next" href="../chapters/12_training.html" title="next page">Next Module →</a>
+</div>

modules/08_dataloader/ABOUT.md

@@ -0,0 +1,332 @@
+---
+title: "DataLoader - Data Pipeline Engineering"
+description: "Build production-grade data loading infrastructure for training at scale"
+difficulty: 3
+time_estimate: "5-6 hours"
+prerequisites: ["Tensor", "Layers", "Training"]
+next_steps: ["Spatial (CNNs)"]
+learning_objectives:
+  - "Design scalable data pipeline architectures for production ML systems"
+  - "Implement efficient dataset abstractions with batching and streaming"
+  - "Build preprocessing pipelines for normalization and data augmentation"
+  - "Understand memory-efficient data loading patterns for large datasets"
+  - "Apply systems thinking to I/O optimization and throughput engineering"
+---
+
+# 08. DataLoader
+
+**🏛️ ARCHITECTURE TIER** | Difficulty: ⭐⭐⭐ (3/4) | Time: 5-6 hours
+
+## Overview
+
+Build the data engineering infrastructure that feeds neural networks. This module implements production-grade data loading, preprocessing, and batching systems—the critical backbone that enables training on real-world datasets like CIFAR-10.
+
+## Learning Objectives
+
+By completing this module, you will be able to:
+
+1. **Design scalable data pipeline architectures** for production ML systems with proper abstractions and interfaces
+2. **Implement efficient dataset abstractions** with batching, shuffling, and streaming for memory-efficient training
+3. **Build preprocessing pipelines** for normalization, augmentation, and transformation with fit-transform patterns
+4. **Understand memory-efficient data loading patterns** for large datasets that don't fit in RAM
+5. **Apply systems thinking** to I/O optimization, caching strategies, and throughput engineering
+
+## Why This Matters
+
+### Production Context
+
+Every production ML system depends on robust data infrastructure:
+
+- **Netflix** uses sophisticated data pipelines to train recommendation models on billions of viewing records
+- **Tesla** processes terabytes of driving sensor data through efficient loading pipelines for autonomous driving
+- **OpenAI** built custom data loaders to train GPT models on hundreds of billions of tokens
+- **Meta** developed PyTorch's DataLoader (which you're reimplementing) to power research and production
+
+### Historical Context
+
+Data loading evolved from bottleneck to optimized system:
+
+- **Early ML (pre-2010)**: Small datasets fit entirely in memory; data loading was an afterthought
+- **ImageNet Era (2012)**: AlexNet required efficient loading of 1.2M images; preprocessing became critical
+- **Big Data ML (2015+)**: Streaming data pipelines became necessary for datasets too large for memory
+- **Modern Scale (2020+)**: Data loading is now a first-class systems problem with dedicated infrastructure teams
+
+The patterns you're building are the same ones used in production at scale.
+
+## Pedagogical Pattern: Build → Use → Analyze
+
+### 1. Build
+
+Implement from first principles:
+- Dataset abstraction with Python protocols (`__getitem__`, `__len__`)
+- DataLoader with batching, shuffling, and iteration
+- CIFAR-10 dataset loader with binary file parsing
+- Normalizer with fit-transform pattern
+- Memory-efficient streaming for large datasets
+
+### 2. Use
+
+Apply to real problems:
+- Load and preprocess CIFAR-10 (50,000 training images)
+- Create train/test data loaders with proper batching
+- Build preprocessing pipelines for normalization
+- Integrate with training loops from Module 07
+- Measure throughput and identify bottlenecks
+
+### 3. Analyze
+
+Deep-dive into systems behavior:
+- Profile memory usage patterns with different batch sizes
+- Measure I/O throughput and identify disk bottlenecks
+- Compare streaming vs in-memory loading strategies
+- Analyze the impact of shuffling on training dynamics
+- Understand trade-offs between batch size and memory
+
+## Implementation Guide
+
+### Core Components
+
+**Dataset Abstraction**
+```python
+class Dataset:
+    """Abstract base class for all datasets.
+    
+    Implements Python protocols for indexing and length.
+    Subclasses must implement __getitem__ and __len__.
+    """
+    def __getitem__(self, index: int):
+        """Return (data, label) for given index."""
+        raise NotImplementedError
+    
+    def __len__(self) -> int:
+        """Return total number of samples."""
+        raise NotImplementedError
+```
+
+**DataLoader Implementation**
+```python
+class DataLoader:
+    """Efficient batch loading with shuffling support.
+    
+    Features:
+    - Automatic batching with configurable batch size
+    - Optional shuffling for training randomization
+    - Drop last batch handling for even batch sizes
+    - Memory-efficient iteration without loading all data
+    """
+    def __init__(self, dataset, batch_size=32, shuffle=False, drop_last=False):
+        self.dataset = dataset
+        self.batch_size = batch_size
+        self.shuffle = shuffle
+        self.drop_last = drop_last
+    
+    def __iter__(self):
+        # Generate indices (shuffled or sequential)
+        indices = list(range(len(self.dataset)))
+        if self.shuffle:
+            np.random.shuffle(indices)
+        
+        # Yield batches
+        for i in range(0, len(indices), self.batch_size):
+            batch_indices = indices[i:i + self.batch_size]
+            if len(batch_indices) < self.batch_size and self.drop_last:
+                continue
+            yield self._get_batch(batch_indices)
+```
+
+**CIFAR-10 Dataset Loader**
+```python
+class CIFAR10Dataset(Dataset):
+    """Load CIFAR-10 dataset with automatic download.
+    
+    CIFAR-10: 60,000 32x32 color images in 10 classes
+    - 50,000 training images
+    - 10,000 test images
+    - Classes: airplane, car, bird, cat, deer, dog, frog, horse, ship, truck
+    """
+    def __init__(self, root='./data', train=True, download=True):
+        self.train = train
+        if download:
+            self._download(root)
+        self.data, self.labels = self._load_batch_files(root, train)
+    
+    def __getitem__(self, index):
+        return self.data[index], self.labels[index]
+    
+    def __len__(self):
+        return len(self.data)
+```
+
+**Preprocessing Pipeline**
+```python
+class Normalizer:
+    """Normalize data using fit-transform pattern.
+    
+    Fits statistics on training data, applies to all splits.
+    Ensures consistent preprocessing across train/val/test.
+    """
+    def fit(self, data):
+        """Compute mean and std from training data."""
+        self.mean = data.mean(axis=0)
+        self.std = data.std(axis=0)
+        return self
+    
+    def transform(self, data):
+        """Apply normalization using fitted statistics."""
+        return (data - self.mean) / (self.std + 1e-8)
+    
+    def fit_transform(self, data):
+        """Fit and transform in one step."""
+        return self.fit(data).transform(data)
+```
+
+### Step-by-Step Implementation
+
+1. **Create Dataset Base Class**
+   - Implement `__getitem__` and `__len__` protocols
+   - Define the interface all datasets must follow
+   - Test with simple array-based dataset
+
+2. **Build CIFAR-10 Loader**
+   - Implement download and extraction logic
+   - Parse binary batch files (pickle format)
+   - Reshape data from flat arrays to (3, 32, 32) images
+   - Handle train/test split loading
+
+3. **Implement DataLoader**
+   - Create batching logic with configurable batch size
+   - Add shuffling with random permutation
+   - Implement iterator protocol for Pythonic loops
+   - Handle edge cases (last incomplete batch, empty dataset)
+
+4. **Add Preprocessing**
+   - Build Normalizer with fit-transform pattern
+   - Compute per-channel statistics for RGB images
+   - Apply transformations efficiently across batches
+   - Test normalization correctness (zero mean, unit variance)
+
+5. **Integration Testing**
+   - Load CIFAR-10 and create data loaders
+   - Iterate through batches and verify shapes
+   - Test with actual training loop from Module 07
+   - Measure data loading throughput
+
+## Testing
+
+### Inline Tests (During Development)
+
+Run inline tests while building:
+```bash
+cd modules/08_dataloader
+python dataloader_dev.py
+```
+
+Expected output:
+```
+Unit Test: Dataset abstraction...
+✅ __getitem__ protocol works correctly
+✅ __len__ returns correct size
+✅ Indexing returns (data, label) tuples
+Progress: Dataset Interface ✓
+
+Unit Test: CIFAR-10 loading...
+✅ Downloaded and extracted 170MB dataset
+✅ Loaded 50,000 training samples
+✅ Sample shape: (3, 32, 32), label range: [0, 9]
+Progress: CIFAR-10 Dataset ✓
+
+Unit Test: DataLoader batching...
+✅ Batch shapes correct: (32, 3, 32, 32)
+✅ Shuffling produces different orderings
+✅ Iteration covers all samples exactly once
+Progress: DataLoader ✓
+```
+
+### Export and Validate
+
+After completing the module:
+```bash
+# Export to tinytorch package
+tito export 08_dataloader
+
+# Run integration tests
+tito test 08_dataloader
+```
+
+### Comprehensive Test Coverage
+
+The test suite validates:
+- Dataset interface correctness
+- CIFAR-10 loading and parsing
+- Batch shape consistency
+- Shuffling randomness
+- Memory efficiency
+- Preprocessing accuracy
+
+## Where This Code Lives
+
+```
+tinytorch/
+├── core/
+│   └── dataloader.py          # Your implementation goes here
+└── __init__.py                # Exposes DataLoader, Dataset, etc.
+
+Usage in other modules:
+>>> from tinytorch.core.dataloader import DataLoader, CIFAR10Dataset
+>>> dataset = CIFAR10Dataset(download=True)
+>>> loader = DataLoader(dataset, batch_size=32, shuffle=True)
+```
+
+## Systems Thinking Questions
+
+1. **Memory vs Throughput Trade-off**: Why does increasing batch size improve GPU utilization but increase memory usage? What's the optimal batch size for a 16GB GPU?
+
+2. **Shuffling Impact**: How does shuffling affect training dynamics and convergence? Why is it critical for training but not for evaluation?
+
+3. **I/O Bottlenecks**: Your GPU can process 1000 images/sec but your disk reads at 100 images/sec. Where's the bottleneck? How would you fix it?
+
+4. **Preprocessing Placement**: Should preprocessing happen in the data loader or in the training loop? What are the trade-offs for CPU vs GPU preprocessing?
+
+5. **Distributed Loading**: If you're training on 8 GPUs, how should you partition the dataset? What challenges arise with shuffling across multiple workers?
+
+## Real-World Connections
+
+### Industry Applications
+
+**Netflix (Recommendation Systems)**
+- Processes billions of viewing records through custom data pipelines
+- Uses streaming loaders for datasets that don't fit in memory
+- Implements sophisticated batching strategies for negative sampling
+
+**Autonomous Vehicles (Tesla, Waymo)**
+- Load terabytes of sensor data (camera, LIDAR, radar) for training
+- Use multi-worker data loading to keep GPUs fully utilized
+- Implement real-time preprocessing pipelines for online learning
+
+**Large Language Models (OpenAI, Anthropic)**
+- Stream hundreds of billions of tokens from distributed storage
+- Use custom data loaders optimized for sequence data
+- Implement efficient tokenization and batching for transformers
+
+### Research Impact
+
+This module teaches patterns from:
+- PyTorch DataLoader (2016): The industry-standard data loading API
+- TensorFlow Dataset API (2017): Google's approach to data pipelines
+- NVIDIA DALI (2019): GPU-accelerated preprocessing for peak throughput
+- WebDataset (2020): Efficient loading from cloud storage
+
+## What's Next?
+
+In **Module 09: Spatial (CNNs)**, you'll use these data loaders to train convolutional neural networks on CIFAR-10:
+
+- Apply convolution operations to the RGB images you're loading
+- Use your DataLoader to iterate through 50,000 training samples
+- Achieve >75% accuracy on CIFAR-10 classification
+- Understand how CNNs process spatial data efficiently
+
+The data infrastructure you built here becomes critical—training CNNs requires efficient batch loading of image data with proper preprocessing.
+
+---
+
+**Ready to build production data infrastructure?** Open `modules/08_dataloader/dataloader_dev.py` and start implementing.

modules/09_spatial/ABOUT.md

@@ -0,0 +1,360 @@
+---
+title: "Convolutional Networks"
+description: "Build CNNs from scratch for computer vision and spatial pattern recognition"
+difficulty: 3
+time_estimate: "6-8 hours"
+prerequisites: ["Tensor", "Activations", "Layers", "DataLoader"]
+next_steps: ["Tokenization"]
+learning_objectives:
+  - "Implement convolution as sliding window operations with weight sharing"
+  - "Design CNN architectures with feature extraction and classification components"
+  - "Understand translation invariance and hierarchical feature learning"
+  - "Build pooling operations for spatial downsampling and invariance"
+  - "Apply computer vision principles to image classification tasks"
+---
+
+# 09. Convolutional Networks
+
+**🏛️ ARCHITECTURE TIER** | Difficulty: ⭐⭐⭐ (3/4) | Time: 6-8 hours
+
+## Overview
+
+Implement convolutional neural networks (CNNs) from scratch. This module teaches you how convolution transforms computer vision from hand-crafted features to learned hierarchical representations that power everything from image classification to autonomous driving.
+
+## Learning Objectives
+
+By completing this module, you will be able to:
+
+1. **Implement convolution** as sliding window operations with explicit loops, understanding weight sharing and local connectivity
+2. **Design CNN architectures** by composing convolutional, pooling, and dense layers for image classification
+3. **Understand translation invariance** and why CNNs are superior to dense networks for spatial data
+4. **Build pooling operations** (MaxPool, AvgPool) for spatial downsampling and feature invariance
+5. **Apply computer vision principles** to achieve >75% accuracy on CIFAR-10 image classification
+
+## Why This Matters
+
+### Production Context
+
+CNNs are the backbone of modern computer vision systems:
+
+- **Meta's Vision AI** uses CNN architectures to tag 2 billion photos daily across Facebook and Instagram
+- **Tesla Autopilot** processes camera feeds through CNN backbones for object detection and lane recognition
+- **Google Photos** built a CNN-based system that automatically organizes billions of images
+- **Medical Imaging** systems use CNNs to detect cancer in X-rays and MRIs with superhuman accuracy
+
+### Historical Context
+
+The convolution revolution transformed computer vision:
+
+- **LeNet (1998)**: Yann LeCun's CNN read zip codes on mail; convolution proved viable but limited by compute
+- **AlexNet (2012)**: Won ImageNet with 16% error rate (vs 26% previous); GPUs + convolution = computer vision revolution
+- **ResNet (2015)**: 152-layer CNN achieved 3.6% error (better than human 5%); proved depth matters
+- **Modern Era (2020+)**: CNNs power production vision systems processing trillions of images daily
+
+The patterns you're implementing revolutionized how machines see.
+
+## Pedagogical Pattern: Build → Use → Analyze
+
+### 1. Build
+
+Implement from first principles:
+- Convolution as explicit sliding window operation
+- Conv2D layer with learnable filters and weight sharing
+- MaxPool2D and AvgPool2D for spatial downsampling
+- Flatten layer to connect spatial and dense layers
+- Complete CNN architecture with feature extraction and classification
+
+### 2. Use
+
+Apply to real problems:
+- Build CNN for CIFAR-10 image classification
+- Extract and visualize learned feature maps
+- Compare CNN vs MLP performance on spatial data
+- Achieve >75% accuracy with proper architecture
+- Understand impact of kernel size, stride, and padding
+
+### 3. Analyze
+
+Deep-dive into architectural choices:
+- Why does weight sharing reduce parameters dramatically?
+- How do early vs late layers learn different features?
+- What's the trade-off between depth and width in CNNs?
+- Why are pooling operations crucial for translation invariance?
+- How does spatial structure preservation improve learning?
+
+## Implementation Guide
+
+### Core Components
+
+**Conv2D Layer - The Heart of Computer Vision**
+```python
+class Conv2D:
+    """2D Convolutional layer with learnable filters.
+    
+    Implements sliding window convolution:
+    - Applies same filter across all spatial positions (weight sharing)
+    - Each filter learns to detect different features (edges, textures, objects)
+    - Output is feature map showing where filter activates strongly
+    
+    Args:
+        in_channels: Number of input channels (3 for RGB, 16 for feature maps)
+        out_channels: Number of learned filters (feature detectors)
+        kernel_size: Size of sliding window (typically 3 or 5)
+        stride: Step size when sliding (1 = no downsampling)
+        padding: Border padding to preserve spatial dimensions
+    """
+    def __init__(self, in_channels, out_channels, kernel_size=3, stride=1, padding=0):
+        # Initialize learnable filters
+        self.weight = Tensor(shape=(out_channels, in_channels, kernel_size, kernel_size))
+        self.bias = Tensor(shape=(out_channels,))
+        
+    def forward(self, x):
+        # x shape: (batch, in_channels, height, width)
+        batch, _, H, W = x.shape
+        kh, kw = self.kernel_size, self.kernel_size
+        
+        # Calculate output dimensions
+        out_h = (H + 2 * self.padding - kh) // self.stride + 1
+        out_w = (W + 2 * self.padding - kw) // self.stride + 1
+        
+        # Sliding window convolution
+        output = Tensor(shape=(batch, self.out_channels, out_h, out_w))
+        for b in range(batch):
+            for oc in range(self.out_channels):
+                for i in range(out_h):
+                    for j in range(out_w):
+                        # Extract local patch
+                        i_start = i * self.stride
+                        j_start = j * self.stride
+                        patch = x[b, :, i_start:i_start+kh, j_start:j_start+kw]
+                        
+                        # Convolution: element-wise multiply and sum
+                        output[b, oc, i, j] = (patch * self.weight[oc]).sum() + self.bias[oc]
+        
+        return output
+```
+
+**Pooling Layers - Spatial Downsampling**
+```python
+class MaxPool2D:
+    """Max pooling for spatial downsampling and translation invariance.
+    
+    Takes maximum value in each local region:
+    - Reduces spatial dimensions while preserving important features
+    - Provides invariance to small translations
+    - Reduces computation in later layers
+    """
+    def __init__(self, kernel_size=2, stride=None):
+        self.kernel_size = kernel_size
+        self.stride = stride or kernel_size
+    
+    def forward(self, x):
+        batch, channels, H, W = x.shape
+        kh, kw = self.kernel_size, self.kernel_size
+        
+        out_h = (H - kh) // self.stride + 1
+        out_w = (W - kw) // self.stride + 1
+        
+        output = Tensor(shape=(batch, channels, out_h, out_w))
+        for b in range(batch):
+            for c in range(channels):
+                for i in range(out_h):
+                    for j in range(out_w):
+                        i_start = i * self.stride
+                        j_start = j * self.stride
+                        patch = x[b, c, i_start:i_start+kh, j_start:j_start+kw]
+                        output[b, c, i, j] = patch.max()
+        
+        return output
+```
+
+**Complete CNN Architecture**
+```python
+class SimpleCNN:
+    """CNN for CIFAR-10 classification.
+    
+    Architecture:
+        Conv(3→32, 3x3) → ReLU → MaxPool(2x2)    # 32x32 → 16x16
+        Conv(32→64, 3x3) → ReLU → MaxPool(2x2)   # 16x16 → 8x8
+        Flatten → Dense(64*8*8 → 128) → ReLU
+        Dense(128 → 10) → Softmax
+    """
+    def __init__(self):
+        self.conv1 = Conv2D(3, 32, kernel_size=3, padding=1)
+        self.relu1 = ReLU()
+        self.pool1 = MaxPool2D(kernel_size=2)
+        
+        self.conv2 = Conv2D(32, 64, kernel_size=3, padding=1)
+        self.relu2 = ReLU()
+        self.pool2 = MaxPool2D(kernel_size=2)
+        
+        self.flatten = Flatten()
+        self.fc1 = Linear(64 * 8 * 8, 128)
+        self.relu3 = ReLU()
+        self.fc2 = Linear(128, 10)
+    
+    def forward(self, x):
+        # Feature extraction
+        x = self.pool1(self.relu1(self.conv1(x)))  # (B, 32, 16, 16)
+        x = self.pool2(self.relu2(self.conv2(x)))  # (B, 64, 8, 8)
+        
+        # Classification
+        x = self.flatten(x)                        # (B, 4096)
+        x = self.relu3(self.fc1(x))               # (B, 128)
+        x = self.fc2(x)                           # (B, 10)
+        return x
+```
+
+### Step-by-Step Implementation
+
+1. **Implement Conv2D Forward Pass**
+   - Create sliding window iteration over spatial dimensions
+   - Apply weight sharing: same filter at all positions
+   - Handle batch processing efficiently
+   - Verify output shape calculation
+
+2. **Build Pooling Operations**
+   - Implement MaxPool2D with maximum extraction
+   - Add AvgPool2D for average pooling
+   - Handle stride and kernel size correctly
+   - Test spatial dimension reduction
+
+3. **Create Flatten Layer**
+   - Convert (B, C, H, W) to (B, C*H*W)
+   - Prepare spatial features for dense layers
+   - Preserve batch dimension
+   - Enable gradient flow backward
+
+4. **Design Complete CNN**
+   - Stack Conv → ReLU → Pool blocks for feature extraction
+   - Add Flatten → Dense for classification
+   - Calculate dimensions at each layer
+   - Test end-to-end forward pass
+
+5. **Train on CIFAR-10**
+   - Load CIFAR-10 using Module 08's DataLoader
+   - Train with cross-entropy loss and SGD
+   - Track accuracy on test set
+   - Achieve >75% accuracy
+
+## Testing
+
+### Inline Tests (During Development)
+
+Run inline tests while building:
+```bash
+cd modules/09_spatial
+python spatial_dev.py
+```
+
+Expected output:
+```
+Unit Test: Conv2D implementation...
+✅ Sliding window convolution works correctly
+✅ Weight sharing applied at all positions
+✅ Output shapes match expected dimensions
+Progress: Conv2D ✓
+
+Unit Test: MaxPool2D implementation...
+✅ Maximum extraction works correctly
+✅ Spatial dimensions reduced properly
+✅ Translation invariance verified
+Progress: Pooling ✓
+
+Unit Test: Complete CNN architecture...
+✅ Forward pass through all layers successful
+✅ Output shape: (32, 10) for 10 classes
+✅ Parameter count reasonable: ~500K parameters
+Progress: CNN Architecture ✓
+```
+
+### Export and Validate
+
+After completing the module:
+```bash
+# Export to tinytorch package
+tito export 09_spatial
+
+# Run integration tests
+tito test 09_spatial
+```
+
+### CIFAR-10 Training Test
+
+```bash
+# Train simple CNN on CIFAR-10
+python tests/integration/test_cnn_cifar10.py
+
+Expected results:
+- Epoch 1: 35% accuracy
+- Epoch 5: 60% accuracy
+- Epoch 10: 75% accuracy
+```
+
+## Where This Code Lives
+
+```
+tinytorch/
+├── nn/
+│   └── spatial.py              # Conv2D, MaxPool2D, etc.
+└── __init__.py                 # Exposes CNN components
+
+Usage in other modules:
+>>> from tinytorch.nn import Conv2D, MaxPool2D
+>>> conv = Conv2D(3, 32, kernel_size=3)
+>>> pool = MaxPool2D(kernel_size=2)
+```
+
+## Systems Thinking Questions
+
+1. **Parameter Efficiency**: A Conv2D(3, 32, 3) has ~900 parameters. How many parameters would a Dense layer need to connect a 32x32 image to 32 outputs? Why is this difference critical for scaling?
+
+2. **Translation Invariance**: Why does a CNN detect a cat regardless of whether it's in the top-left or bottom-right of an image? How does weight sharing enable this property?
+
+3. **Hierarchical Features**: Early CNN layers detect edges and textures. Later layers detect objects and faces. How does this emerge from stacking convolutions? Why doesn't this happen in dense networks?
+
+4. **Receptive Field Growth**: A single Conv2D(kernel=3) sees a 3x3 region. After two Conv2D layers, what region does each output see? How do deep CNNs build global context from local operations?
+
+5. **Compute vs Memory Trade-offs**: Large kernel sizes (7x7) have more parameters but fewer operations. Small kernels (3x3) stacked deeply have opposite trade-offs. Which is better and why?
+
+## Real-World Connections
+
+### Industry Applications
+
+**Autonomous Vehicles (Tesla, Waymo)**
+- Multi-camera CNN systems process 30 FPS at 1920x1200 resolution
+- Feature maps from CNNs feed into object detection and segmentation
+- Real-time requirements demand efficient Conv2D implementations
+
+**Medical Imaging (PathAI, Zebra Medical)**
+- CNNs analyze X-rays and CT scans for diagnostic assistance
+- Achieve superhuman performance on specific tasks (diabetic retinopathy detection)
+- Architecture design critical for accuracy-interpretability trade-off
+
+**Face Recognition (Apple Face ID, Facebook DeepFace)**
+- CNN embeddings enable accurate face matching at billion-user scale
+- Lightweight CNN architectures run on mobile devices in real-time
+- Privacy concerns drive on-device processing
+
+### Research Impact
+
+This module implements patterns from:
+- LeNet-5 (1998): First successful CNN for digit recognition
+- AlexNet (2012): Sparked deep learning revolution with CNNs + GPUs
+- VGG (2014): Showed deeper is better with simple 3x3 convolutions
+- ResNet (2015): Enabled training 152-layer CNNs with skip connections
+
+## What's Next?
+
+In **Module 10: Tokenization**, you'll shift from processing images to processing text:
+
+- Learn how to convert text into numerical representations
+- Implement tokenization strategies (character, word, subword)
+- Build vocabulary management systems
+- Prepare text data for transformers in Module 13
+
+This completes the vision half of the Architecture Tier. Next, you'll tackle language!
+
+---
+
+**Ready to build CNNs from scratch?** Open `modules/09_spatial/spatial_dev.py` and start implementing.

modules/10_tokenization/ABOUT.md

@@ -0,0 +1,402 @@
+---
+title: "Tokenization - Text to Numerical Sequences"
+description: "Build tokenizers to convert raw text into sequences for language models"
+difficulty: 2
+time_estimate: "4-5 hours"
+prerequisites: ["Tensor"]
+next_steps: ["Embeddings"]
+learning_objectives:
+  - "Implement character-level and subword tokenization strategies"
+  - "Design efficient vocabulary management systems for language models"
+  - "Understand trade-offs between vocabulary size and sequence length"
+  - "Build BPE tokenizer for optimal subword unit representation"
+  - "Apply text processing optimization for production NLP pipelines"
+---
+
+# 10. Tokenization
+
+**🏛️ ARCHITECTURE TIER** | Difficulty: ⭐⭐ (2/4) | Time: 4-5 hours
+
+## Overview
+
+Build tokenization systems that convert raw text into numerical sequences for language models. This module implements character-level and subword tokenizers (BPE) that balance vocabulary size, sequence length, and computational efficiency.
+
+## Learning Objectives
+
+By completing this module, you will be able to:
+
+1. **Implement character-level and subword tokenization** strategies for converting text to token sequences
+2. **Design efficient vocabulary management** systems with special tokens and encoding/decoding
+3. **Understand trade-offs** between vocabulary size (model parameters) and sequence length (computation)
+4. **Build BPE (Byte Pair Encoding)** tokenizer for optimal subword unit representation
+5. **Apply text processing optimization** techniques for production NLP pipelines at scale
+
+## Why This Matters
+
+### Production Context
+
+Every language model depends on tokenization:
+
+- **GPT-4** uses a 100K-token vocabulary trained on trillions of tokens of text
+- **Google Translate** processes billions of sentences daily through tokenization pipelines
+- **BERT** pioneered WordPiece tokenization that handles 100+ languages efficiently
+- **Code models** like Copilot use specialized tokenizers for programming languages
+
+### Historical Context
+
+Tokenization evolved with language modeling:
+
+- **Word-Level (pre-2016)**: Simple but massive vocabularies (100K+ words); struggles with rare words and typos
+- **Character-Level (2015)**: Small vocabulary but extremely long sequences; computationally expensive
+- **BPE (2016)**: Subword tokenization balances both; enabled GPT and modern transformers
+- **SentencePiece (2018)**: Unified text and multilingual tokenization; powers modern multilingual models
+- **Modern (2020+)**: Specialized tokenizers for code, math, and multimodal content
+
+The tokenizers you're building are the foundation of all modern NLP.
+
+## Pedagogical Pattern: Build → Use → Analyze
+
+### 1. Build
+
+Implement from first principles:
+- Character-level tokenizer with vocab management
+- Special tokens (<PAD>, <UNK>, <BOS>, <EOS>)
+- BPE algorithm for learning subword merges
+- Encode/decode functions for text ↔ tokens
+- Vocabulary serialization for model deployment
+
+### 2. Use
+
+Apply to real problems:
+- Tokenize Shakespeare and modern text datasets
+- Build vocabularies of different sizes (1K, 10K, 50K tokens)
+- Compare character vs BPE on sequence length
+- Handle out-of-vocabulary words gracefully
+- Measure tokenization throughput (tokens/second)
+
+### 3. Analyze
+
+Deep-dive into design trade-offs:
+- How does vocabulary size affect model parameters?
+- Why do longer sequences increase computation quadratically (in transformers)?
+- What's the sweet spot between vocab size and sequence length?
+- How does tokenization affect rare words and morphology?
+- Why do multilingual models need larger vocabularies?
+
+## Implementation Guide
+
+### Core Components
+
+**Character-Level Tokenizer**
+```python
+class CharacterTokenizer:
+    """Simple character-level tokenization.
+    
+    Treats each character as a token. Simple but results in long sequences.
+    Vocab size: typically 100-500 (all ASCII or Unicode characters)
+    """
+    def __init__(self):
+        self.char_to_idx = {}
+        self.idx_to_char = {}
+        self.vocab_size = 0
+        
+        # Special tokens
+        self.PAD_TOKEN = "<PAD>"
+        self.UNK_TOKEN = "<UNK>"
+        self.BOS_TOKEN = "<BOS>"
+        self.EOS_TOKEN = "<EOS>"
+    
+    def build_vocab(self, texts):
+        """Build vocabulary from text corpus."""
+        # Add special tokens first
+        special_tokens = [self.PAD_TOKEN, self.UNK_TOKEN, self.BOS_TOKEN, self.EOS_TOKEN]
+        for token in special_tokens:
+            self.char_to_idx[token] = len(self.char_to_idx)
+        
+        # Add all unique characters
+        unique_chars = set(''.join(texts))
+        for char in sorted(unique_chars):
+            if char not in self.char_to_idx:
+                self.char_to_idx[char] = len(self.char_to_idx)
+        
+        # Create reverse mapping
+        self.idx_to_char = {idx: char for char, idx in self.char_to_idx.items()}
+        self.vocab_size = len(self.char_to_idx)
+    
+    def encode(self, text):
+        """Convert text to token IDs."""
+        return [self.char_to_idx.get(char, self.char_to_idx[self.UNK_TOKEN]) 
+                for char in text]
+    
+    def decode(self, token_ids):
+        """Convert token IDs back to text."""
+        return ''.join([self.idx_to_char[idx] for idx in token_ids])
+```
+
+**BPE (Byte Pair Encoding) Tokenizer**
+```python
+class BPETokenizer:
+    """Byte Pair Encoding for subword tokenization.
+    
+    Iteratively merges most frequent character pairs to create subword units.
+    Balances vocabulary size and sequence length optimally.
+    
+    Example:
+        "unhappiness" → ["un", "happi", "ness"] (3 tokens)
+        vs character-level: ["u","n","h","a","p","p","i","n","e","s","s"] (11 tokens)
+    """
+    def __init__(self, vocab_size=10000):
+        self.vocab_size = vocab_size
+        self.merges = {}  # Learned merge rules
+        self.vocab = {}   # Token to ID mapping
+    
+    def train(self, texts):
+        """Learn BPE merges from corpus.
+        
+        Algorithm:
+        1. Start with character-level vocabulary
+        2. Count all adjacent character pairs
+        3. Merge most frequent pair
+        4. Repeat until vocabulary reaches target size
+        """
+        # Initialize with character-level vocab
+        vocab = self._get_char_vocab(texts)
+        
+        # Learn merges iteratively
+        while len(vocab) < self.vocab_size:
+            # Count pairs
+            pairs = self._count_pairs(texts, vocab)
+            if not pairs:
+                break
+            
+            # Merge most frequent pair
+            best_pair = max(pairs, key=pairs.get)
+            texts = self._merge_pair(texts, best_pair)
+            vocab.add(''.join(best_pair))
+            self.merges[best_pair] = ''.join(best_pair)
+        
+        # Build final vocabulary
+        self.vocab = {token: idx for idx, token in enumerate(sorted(vocab))}
+    
+    def encode(self, text):
+        """Encode text using learned BPE merges."""
+        # Start with characters
+        tokens = list(text)
+        
+        # Apply merges in learned order
+        while True:
+            pairs = [(tokens[i], tokens[i+1]) for i in range(len(tokens)-1)]
+            if not pairs:
+                break
+            
+            # Find first mergeable pair
+            mergeable = [p for p in pairs if p in self.merges]
+            if not mergeable:
+                break
+            
+            # Apply merge
+            pair = mergeable[0]
+            new_tokens = []
+            i = 0
+            while i < len(tokens):
+                if i < len(tokens) - 1 and (tokens[i], tokens[i+1]) == pair:
+                    new_tokens.append(self.merges[pair])
+                    i += 2
+                else:
+                    new_tokens.append(tokens[i])
+                    i += 1
+            tokens = new_tokens
+        
+        # Convert tokens to IDs
+        return [self.vocab.get(token, self.vocab['<UNK>']) for token in tokens]
+```
+
+**Vocabulary Management**
+```python
+class Vocabulary:
+    """Manage token-to-ID mappings with special tokens.
+    
+    Provides clean interface for encoding/decoding and vocab serialization.
+    """
+    def __init__(self):
+        self.token_to_id = {}
+        self.id_to_token = {}
+        
+        # Reserve special token IDs
+        self.PAD_ID = 0
+        self.UNK_ID = 1
+        self.BOS_ID = 2
+        self.EOS_ID = 3
+        
+        self._add_special_tokens()
+    
+    def _add_special_tokens(self):
+        special = [('<PAD>', self.PAD_ID), ('<UNK>', self.UNK_ID),
+                   ('<BOS>', self.BOS_ID), ('<EOS>', self.EOS_ID)]
+        for token, idx in special:
+            self.token_to_id[token] = idx
+            self.id_to_token[idx] = token
+    
+    def add_token(self, token):
+        if token not in self.token_to_id:
+            idx = len(self.token_to_id)
+            self.token_to_id[token] = idx
+            self.id_to_token[idx] = token
+    
+    def save(self, path):
+        """Save vocabulary for deployment."""
+        import json
+        with open(path, 'w') as f:
+            json.dump(self.token_to_id, f)
+    
+    def load(self, path):
+        """Load vocabulary for inference."""
+        import json
+        with open(path, 'r') as f:
+            self.token_to_id = json.load(f)
+            self.id_to_token = {v: k for k, v in self.token_to_id.items()}
+```
+
+### Step-by-Step Implementation
+
+1. **Build Character Tokenizer**
+   - Create vocabulary from unique characters
+   - Add special tokens (PAD, UNK, BOS, EOS)
+   - Implement encode (text → IDs) and decode (IDs → text)
+   - Handle unknown characters gracefully
+
+2. **Implement BPE Algorithm**
+   - Start with character vocabulary
+   - Count adjacent pair frequencies
+   - Merge most frequent pairs iteratively
+   - Build merge rules and final vocabulary
+
+3. **Add Vocabulary Management**
+   - Create token ↔ ID bidirectional mappings
+   - Implement serialization for saving/loading
+   - Handle special tokens consistently
+   - Support vocabulary extension
+
+4. **Optimize for Production**
+   - Cache encode/decode results
+   - Use efficient data structures (tries, hash maps)
+   - Batch process multiple texts
+   - Measure throughput (tokens/second)
+
+5. **Compare Tokenization Strategies**
+   - Measure sequence lengths for same text
+   - Analyze vocabulary size requirements
+   - Test on rare words and typos
+   - Evaluate multilingual performance
+
+## Testing
+
+### Inline Tests (During Development)
+
+Run inline tests while building:
+```bash
+cd modules/10_tokenization
+python tokenization_dev.py
+```
+
+Expected output:
+```
+Unit Test: Character tokenizer...
+✅ Vocabulary built with 89 unique characters
+✅ Encode/decode round-trip successful
+✅ Special tokens handled correctly
+Progress: Character Tokenizer ✓
+
+Unit Test: BPE tokenizer...
+✅ Learned 5000 merge rules from corpus
+✅ Sequence length reduced 3.2x vs character-level
+✅ Handles rare words and typos gracefully
+Progress: BPE Tokenizer ✓
+
+Unit Test: Vocabulary management...
+✅ Token-to-ID mappings bidirectional
+✅ Vocabulary saved and loaded correctly
+✅ Special token IDs reserved
+Progress: Vocabulary ✓
+```
+
+### Export and Validate
+
+After completing the module:
+```bash
+# Export to tinytorch package
+tito export 10_tokenization
+
+# Run integration tests
+tito test 10_tokenization
+```
+
+## Where This Code Lives
+
+```
+tinytorch/
+├── text/
+│   └── tokenization.py         # Your implementation goes here
+└── __init__.py                 # Exposes CharTokenizer, BPETokenizer, etc.
+
+Usage in other modules:
+>>> from tinytorch.text import BPETokenizer
+>>> tokenizer = BPETokenizer(vocab_size=10000)
+>>> tokenizer.train(texts)
+>>> ids = tokenizer.encode("Hello world!")
+```
+
+## Systems Thinking Questions
+
+1. **Vocabulary Size vs Model Size**: GPT-2 has 50K vocabulary with 768-dim embeddings = 38M parameters just for embeddings. How does this scale to GPT-3's 100K vocabulary?
+
+2. **Sequence Length vs Computation**: Transformers have O(n²) attention complexity. If BPE reduces sequence length from 1000 to 300 tokens, how much does this reduce computation?
+
+3. **Rare Word Handling**: A word-level tokenizer marks rare words as <UNK>, losing all information. How does BPE handle rare words like "unhappiness" even if never seen during training?
+
+4. **Multilingual Challenges**: English needs ~30K tokens for good coverage. Chinese needs 50K+. Why? How does this affect multilingual model design?
+
+5. **Tokenization as Compression**: BPE learns common patterns like "ing", "ed", "tion". Why is this similar to data compression? What's the connection to information theory?
+
+## Real-World Connections
+
+### Industry Applications
+
+**OpenAI GPT Series**
+- GPT-2: 50K BPE vocabulary, trained on 8M web pages
+- GPT-3: 100K vocabulary, handles code and multilingual text
+- GPT-4: Advanced tiktoken library with 100K+ tokens
+- Tokenization optimization critical for $700/1M token economics
+
+**Google Multilingual Models**
+- SentencePiece used in BERT, T5, PaLM for 100+ languages
+- Unified tokenization across languages without preprocessing
+- Optimized for fast serving at Google-scale traffic
+
+**Code Models (GitHub Copilot, AlphaCode)**
+- Specialized tokenizers for programming languages
+- Handle indentation, operators, and variable names efficiently
+- Balance natural language and code syntax
+
+### Research Impact
+
+This module implements patterns from:
+- BPE (Sennrich et al., 2016): Subword tokenization for NMT
+- WordPiece (Google, 2016): BERT tokenization strategy
+- SentencePiece (Kudo, 2018): Language-agnostic tokenization
+- tiktoken (OpenAI, 2023): Fast BPE for GPT-3/4
+
+## What's Next?
+
+In **Module 11: Embeddings**, you'll convert these token IDs into dense vector representations:
+
+- Map discrete token IDs to continuous embeddings
+- Learn position encodings for sequence order
+- Implement lookup tables for fast embedding retrieval
+- Understand how embeddings capture semantic similarity
+
+The tokens you create here become the input to every transformer and language model!
+
+---
+
+**Ready to build tokenizers from scratch?** Open `modules/10_tokenization/tokenization_dev.py` and start implementing.

modules/11_embeddings/ABOUT.md

@@ -0,0 +1,402 @@
+---
+title: "Embeddings - Token to Vector Representations"
+description: "Build embedding layers that convert discrete tokens to dense vectors"
+difficulty: 2
+time_estimate: "4-5 hours"
+prerequisites: ["Tensor", "Tokenization"]
+next_steps: ["Attention"]
+learning_objectives:
+  - "Implement embedding layers with efficient lookup table operations"
+  - "Design positional encodings to capture sequence order information"
+  - "Understand memory scaling with vocabulary size and embedding dimensions"
+  - "Optimize embedding lookups for cache efficiency and bandwidth"
+  - "Apply dimensionality principles to semantic vector representations"
+---
+
+# 11. Embeddings
+
+**🏛️ ARCHITECTURE TIER** | Difficulty: ⭐⭐ (2/4) | Time: 4-5 hours
+
+## Overview
+
+Build embedding systems that transform discrete token IDs into dense vector representations. This module implements lookup tables, positional encodings, and optimization techniques that power all modern language models.
+
+## Learning Objectives
+
+By completing this module, you will be able to:
+
+1. **Implement embedding layers** with efficient lookup table operations for token-to-vector conversion
+2. **Design positional encodings** (learned and sinusoidal) to capture sequence order information
+3. **Understand memory scaling** with vocabulary size and embedding dimensions in production models
+4. **Optimize embedding lookups** for cache efficiency and memory bandwidth utilization
+5. **Apply dimensionality principles** to balance expressiveness and computational efficiency
+
+## Why This Matters
+
+### Production Context
+
+Embeddings are the foundation of all modern NLP:
+
+- **GPT-3's embedding table**: 50K vocab × 12K dims = 600M parameters (20% of total model)
+- **BERT's embeddings**: Token + position + segment embeddings enable bidirectional understanding
+- **Word2Vec/GloVe**: Pioneered semantic embeddings; "king - man + woman ≈ queen"
+- **Recommendation systems**: Embedding tables for billions of items (YouTube, Netflix, Spotify)
+
+### Historical Context
+
+Embeddings evolved from sparse to dense representations:
+
+- **One-Hot Encoding (pre-2013)**: Vocabulary-sized vectors; no semantic similarity
+- **Word2Vec (2013)**: Dense embeddings capture semantic relationships; revolutionized NLP
+- **GloVe (2014)**: Global co-occurrence statistics improve quality
+- **Contextual Embeddings (2018)**: BERT/GPT embeddings depend on context; same word, different vectors
+- **Modern Scale (2020+)**: 100K+ vocabulary embeddings in production language models
+
+The embeddings you're building are the input layer of transformers and all modern NLP.
+
+## Pedagogical Pattern: Build → Use → Analyze
+
+### 1. Build
+
+Implement from first principles:
+- Embedding layer with learnable lookup table
+- Sinusoidal positional encoding (Transformer-style)
+- Learned positional embeddings (GPT-style)
+- Combined token + position embeddings
+- Gradient flow through embedding lookups
+
+### 2. Use
+
+Apply to real problems:
+- Convert token sequences to dense vectors
+- Add positional information for sequence order
+- Visualize embedding spaces with t-SNE
+- Measure semantic similarity with cosine distance
+- Integrate with attention mechanisms (Module 12)
+
+### 3. Analyze
+
+Deep-dive into design trade-offs:
+- How does embedding dimension affect model capacity?
+- Why do transformers need positional encodings?
+- What's the memory cost of large vocabularies?
+- How do embeddings capture semantic relationships?
+- Why sinusoidal vs learned position encodings?
+
+## Implementation Guide
+
+### Core Components
+
+**Embedding Layer - Token Lookup Table**
+```python
+class Embedding:
+    """Learnable embedding layer for token-to-vector conversion.
+    
+    Implements efficient lookup table that maps token IDs to dense vectors.
+    The core component of all language models.
+    
+    Args:
+        vocab_size: Size of vocabulary (e.g., 50,000 for GPT-2)
+        embedding_dim: Dimension of dense vectors (e.g., 768 for BERT-base)
+    
+    Memory: vocab_size × embedding_dim parameters
+    Example: 50K vocab × 768 dim = 38M parameters
+    """
+    def __init__(self, vocab_size, embedding_dim):
+        self.vocab_size = vocab_size
+        self.embedding_dim = embedding_dim
+        
+        # Initialize embedding table randomly
+        # Shape: (vocab_size, embedding_dim)
+        self.weight = Tensor.randn(vocab_size, embedding_dim) * 0.02
+    
+    def forward(self, token_ids):
+        """Look up embeddings for token IDs.
+        
+        Args:
+            token_ids: (batch_size, seq_len) tensor of token IDs
+        
+        Returns:
+            embeddings: (batch_size, seq_len, embedding_dim) dense vectors
+        """
+        batch_size, seq_len = token_ids.shape
+        
+        # Lookup operation: index into embedding table
+        embeddings = self.weight[token_ids]  # Advanced indexing
+        
+        return embeddings
+    
+    def backward(self, grad_output):
+        """Gradients accumulate in embedding table.
+        
+        Only embeddings that were looked up receive gradients.
+        This is sparse gradient update - critical for efficiency.
+        """
+        batch_size, seq_len, embed_dim = grad_output.shape
+        
+        # Accumulate gradients for each unique token ID
+        grad_weight = Tensor.zeros_like(self.weight)
+        for b in range(batch_size):
+            for s in range(seq_len):
+                token_id = token_ids[b, s]
+                grad_weight[token_id] += grad_output[b, s]
+        
+        return grad_weight
+```
+
+**Positional Encoding - Sinusoidal (Transformer-Style)**
+```python
+class SinusoidalPositionalEncoding:
+    """Fixed sinusoidal positional encoding.
+    
+    Used in original Transformer (Vaswani et al., 2017).
+    Encodes absolute position using sine/cosine functions of different frequencies.
+    
+    Advantages:
+    - No learned parameters
+    - Can generalize to longer sequences than training length
+    - Mathematically elegant relative position representation
+    """
+    def __init__(self, max_seq_len, embedding_dim):
+        self.max_seq_len = max_seq_len
+        self.embedding_dim = embedding_dim
+        
+        # Pre-compute positional encodings
+        self.encodings = self._compute_encodings()
+    
+    def _compute_encodings(self):
+        """Compute sinusoidal position encodings.
+        
+        PE(pos, 2i) = sin(pos / 10000^(2i/d_model))
+        PE(pos, 2i+1) = cos(pos / 10000^(2i/d_model))
+        """
+        position = np.arange(self.max_seq_len)[:, np.newaxis]
+        div_term = np.exp(np.arange(0, self.embedding_dim, 2) * 
+                         -(np.log(10000.0) / self.embedding_dim))
+        
+        encodings = np.zeros((self.max_seq_len, self.embedding_dim))
+        encodings[:, 0::2] = np.sin(position * div_term)  # Even indices
+        encodings[:, 1::2] = np.cos(position * div_term)  # Odd indices
+        
+        return Tensor(encodings)
+    
+    def forward(self, seq_len):
+        """Return positional encodings for sequence length.
+        
+        Args:
+            seq_len: Length of input sequence
+        
+        Returns:
+            pos_encodings: (seq_len, embedding_dim) positional vectors
+        """
+        return self.encodings[:seq_len]
+```
+
+**Learned Positional Embeddings (GPT-Style)**
+```python
+class LearnedPositionalEmbedding:
+    """Learned positional embeddings.
+    
+    Used in GPT models. Learns absolute position representations during training.
+    
+    Advantages:
+    - Can learn task-specific position patterns
+    - Often performs slightly better than sinusoidal
+    
+    Disadvantages:
+    - Cannot generalize beyond max trained sequence length
+    - Requires additional parameters
+    """
+    def __init__(self, max_seq_len, embedding_dim):
+        self.max_seq_len = max_seq_len
+        self.embedding_dim = embedding_dim
+        
+        # Learnable position embedding table
+        self.weight = Tensor.randn(max_seq_len, embedding_dim) * 0.02
+    
+    def forward(self, seq_len):
+        """Look up learned position embeddings.
+        
+        Args:
+            seq_len: Length of input sequence
+        
+        Returns:
+            pos_embeddings: (seq_len, embedding_dim) learned vectors
+        """
+        return self.weight[:seq_len]
+```
+
+**Combined Token + Position Embeddings**
+```python
+def get_combined_embeddings(token_ids, token_embeddings, pos_embeddings):
+    """Combine token and position embeddings.
+    
+    Used as input to transformer models.
+    
+    Args:
+        token_ids: (batch_size, seq_len) token indices
+        token_embeddings: Embedding layer for tokens
+        pos_embeddings: Positional encoding layer
+    
+    Returns:
+        combined: (batch_size, seq_len, embedding_dim) token + position
+    """
+    batch_size, seq_len = token_ids.shape
+    
+    # Get token embeddings
+    token_vecs = token_embeddings(token_ids)  # (B, L, D)
+    
+    # Get position embeddings
+    pos_vecs = pos_embeddings(seq_len)        # (L, D)
+    
+    # Add them together (broadcasting handles batch dimension)
+    combined = token_vecs + pos_vecs          # (B, L, D)
+    
+    return combined
+```
+
+### Step-by-Step Implementation
+
+1. **Create Embedding Layer**
+   - Initialize weight matrix (vocab_size × embedding_dim)
+   - Implement forward pass with indexing
+   - Add backward pass with sparse gradient accumulation
+   - Test with small vocabulary
+
+2. **Implement Sinusoidal Positions**
+   - Compute sine/cosine encodings
+   - Handle even/odd indices correctly
+   - Verify periodicity properties
+   - Test generalization to longer sequences
+
+3. **Add Learned Positions**
+   - Create learnable position table
+   - Initialize with small random values
+   - Implement forward and backward passes
+   - Compare with sinusoidal encodings
+
+4. **Combine Token + Position**
+   - Add token and position embeddings
+   - Handle batch broadcasting correctly
+   - Verify gradient flow through both
+   - Test with real tokenized sequences
+
+5. **Analyze Embedding Spaces**
+   - Visualize embeddings with t-SNE or PCA
+   - Measure cosine similarity between tokens
+   - Verify semantic relationships emerge
+   - Profile memory and lookup efficiency
+
+## Testing
+
+### Inline Tests (During Development)
+
+Run inline tests while building:
+```bash
+cd modules/11_embeddings
+python embeddings_dev.py
+```
+
+Expected output:
+```
+Unit Test: Embedding layer...
+✅ Lookup table created: 10K vocab × 256 dims = 2.5M parameters
+✅ Forward pass shape correct: (32, 20, 256)
+✅ Backward pass accumulates gradients correctly
+Progress: Embedding Layer ✓
+
+Unit Test: Sinusoidal positional encoding...
+✅ Encodings computed for 512 positions
+✅ Sine/cosine patterns verified
+✅ Generalization to longer sequences works
+Progress: Sinusoidal Positions ✓
+
+Unit Test: Combined embeddings...
+✅ Token + position addition works
+✅ Gradient flows through both components
+✅ Batch broadcasting handled correctly
+Progress: Combined Embeddings ✓
+```
+
+### Export and Validate
+
+After completing the module:
+```bash
+# Export to tinytorch package
+tito export 11_embeddings
+
+# Run integration tests
+tito test 11_embeddings
+```
+
+## Where This Code Lives
+
+```
+tinytorch/
+├── nn/
+│   └── embeddings.py           # Your implementation goes here
+└── __init__.py                 # Exposes Embedding, PositionalEncoding, etc.
+
+Usage in other modules:
+>>> from tinytorch.nn import Embedding, SinusoidalPositionalEncoding
+>>> token_emb = Embedding(vocab_size=50000, embedding_dim=768)
+>>> pos_emb = SinusoidalPositionalEncoding(max_len=512, dim=768)
+```
+
+## Systems Thinking Questions
+
+1. **Memory Scaling**: GPT-3 has 50K vocab × 12K dims = 600M embedding parameters. At FP32 (4 bytes), how much memory? At FP16? Why does this matter for training vs inference?
+
+2. **Sparse Gradients**: During training, only ~1% of vocabulary appears in each batch. How does sparse gradient accumulation save computation compared to dense updates?
+
+3. **Embedding Dimension Choice**: BERT-base uses 768 dims, BERT-large uses 1024. How does dimension affect: (a) model capacity, (b) computation, (c) memory bandwidth?
+
+4. **Position Encoding Trade-offs**: Sinusoidal allows generalization to any length. Learned positions are limited to max training length. When would you choose each?
+
+5. **Semantic Geometry**: Why do word embeddings exhibit linear relationships like "king - man + woman ≈ queen"? What property of the training objective causes this?
+
+## Real-World Connections
+
+### Industry Applications
+
+**Large Language Models (OpenAI, Anthropic, Google)**
+- GPT-4: 100K+ vocabulary embeddings
+- Embedding tables often 20-40% of total model parameters
+- Optimized embedding access critical for inference latency
+- Mixed-precision (FP16) embeddings save memory
+
+**Recommendation Systems (YouTube, Netflix, Spotify)**
+- Billion-scale item embeddings for personalization
+- Embedding retrieval systems for fast nearest-neighbor search
+- Continuous embedding updates with online learning
+- Embedding quantization for serving efficiency
+
+**Multilingual Models (Google Translate, Facebook M2M)**
+- Shared embedding spaces across 100+ languages
+- Cross-lingual embeddings enable zero-shot transfer
+- Vocabulary size optimization for multilingual coverage
+- Embedding alignment techniques for language pairs
+
+### Research Impact
+
+This module implements patterns from:
+- Word2Vec (2013): Pioneered dense semantic embeddings
+- GloVe (2014): Global co-occurrence matrix factorization
+- Transformer (2017): Sinusoidal positional encodings
+- BERT (2018): Contextual embeddings revolutionized NLP
+- GPT (2018): Learned positional embeddings for autoregressive models
+
+## What's Next?
+
+In **Module 12: Attention**, you'll use these embeddings as input to attention mechanisms:
+
+- Query, Key, Value projections from embeddings
+- Scaled dot-product attention over embedded sequences
+- Multi-head attention for different representation subspaces
+- Self-attention that relates all positions in a sequence
+
+The embeddings you built are the foundation input to every transformer!
+
+---
+
+**Ready to build embedding systems from scratch?** Open `modules/11_embeddings/embeddings_dev.py` and start implementing.

modules/12_attention/ABOUT.md

@@ -0,0 +1,403 @@
+---
+title: "Attention - The Mechanism That Powers Modern AI"
+description: "Build scaled dot-product and multi-head attention from scratch"
+difficulty: 3
+time_estimate: "5-6 hours"
+prerequisites: ["Tensor", "Layers", "Embeddings"]
+next_steps: ["Transformers"]
+learning_objectives:
+  - "Implement scaled dot-product attention with query, key, and value matrices"
+  - "Design multi-head attention for parallel attention subspaces"
+  - "Understand masking strategies for causal, padding, and bidirectional attention"
+  - "Build self-attention mechanisms for sequence-to-sequence modeling"
+  - "Apply attention patterns that power GPT, BERT, and modern transformers"
+---
+
+# 12. Attention
+
+**🏛️ ARCHITECTURE TIER** | Difficulty: ⭐⭐⭐ (3/4) | Time: 5-6 hours
+
+## Overview
+
+Implement the attention mechanism that revolutionized AI. This module builds scaled dot-product attention and multi-head attention—the core components of GPT, BERT, and all modern transformer models.
+
+## Learning Objectives
+
+By completing this module, you will be able to:
+
+1. **Implement scaled dot-product attention** with query, key, and value matrices following the Transformer paper formula
+2. **Design multi-head attention** for parallel attention in multiple representation subspaces
+3. **Understand masking strategies** for causal (GPT-style), padding, and bidirectional (BERT-style) attention
+4. **Build self-attention mechanisms** for sequence-to-sequence modeling with global context
+5. **Apply attention patterns** that power all modern transformers from GPT-4 to Claude to Gemini
+
+## Why This Matters
+
+### Production Context
+
+Attention is the core of modern AI:
+
+- **GPT-4** uses 96 attention layers with 128 heads each; attention is 70% of compute
+- **BERT** pioneered bidirectional attention; powers Google Search ranking
+- **AlphaFold2** uses attention over protein sequences; solved 50-year protein folding problem
+- **Vision Transformers** replaced CNNs in production at Google, Meta, OpenAI
+
+### Historical Context
+
+Attention revolutionized machine learning:
+
+- **RNN Era (pre-2017)**: Sequential processing; no parallelism; gradient vanishing in long sequences
+- **Attention is All You Need (2017)**: Pure attention architecture; parallelizable; global context
+- **BERT/GPT (2018)**: Transformers dominate NLP; attention beats all previous approaches
+- **Beyond NLP (2020+)**: Attention powers vision (ViT), biology (AlphaFold), multimodal (CLIP)
+
+The attention mechanism you're implementing sparked the current AI revolution.
+
+## Pedagogical Pattern: Build → Use → Analyze
+
+### 1. Build
+
+Implement from first principles:
+- Scaled dot-product attention: `softmax(QK^T/√d_k)V`
+- Multi-head attention with parallel heads
+- Masking for causal and padding patterns
+- Self-attention wrapper (Q=K=V)
+- Attention visualization and interpretation
+
+### 2. Use
+
+Apply to real problems:
+- Build language model with causal attention
+- Implement BERT-style bidirectional attention
+- Visualize attention patterns on real text
+- Compare single-head vs multi-head performance
+- Measure O(n²) computational scaling
+
+### 3. Analyze
+
+Deep-dive into design choices:
+- Why does attention scale quadratically with sequence length?
+- How do multiple heads capture different linguistic patterns?
+- Why is the 1/√d_k scaling factor critical for training?
+- When would you use causal vs bidirectional attention?
+- What are the memory vs computation trade-offs?
+
+## Implementation Guide
+
+### Core Components
+
+**Scaled Dot-Product Attention - The Heart of Transformers**
+```python
+def scaled_dot_product_attention(Q, K, V, mask=None):
+    """The fundamental attention operation from 'Attention is All You Need'.
+    
+    Attention(Q, K, V) = softmax(QK^T / √d_k) V
+    
+    This exact formula powers GPT, BERT, and all transformers.
+    
+    Args:
+        Q: Query matrix (batch, heads, seq_len_q, d_k)
+        K: Key matrix (batch, heads, seq_len_k, d_k)
+        V: Value matrix (batch, heads, seq_len_v, d_v)
+        mask: Optional mask (batch, 1, seq_len_q, seq_len_k)
+    
+    Returns:
+        output: Attended values (batch, heads, seq_len_q, d_v)
+        attention_weights: Attention probabilities (batch, heads, seq_len_q, seq_len_k)
+    
+    Intuition:
+        Q = "What am I looking for?"
+        K = "What information is available?"
+        V = "What is the actual content?"
+        
+        Attention computes: for each query, how much should I focus on each key?
+        Then uses those weights to mix the values.
+    """
+    # d_k = dimension of keys (and queries)
+    d_k = Q.shape[-1]
+    
+    # Compute attention scores: QK^T
+    # Shape: (batch, heads, seq_len_q, seq_len_k)
+    scores = Q @ K.transpose(-2, -1)
+    
+    # Scale by sqrt(d_k) to prevent extreme softmax saturation
+    scores = scores / math.sqrt(d_k)
+    
+    # Apply mask if provided (for causal or padding masking)
+    if mask is not None:
+        # Set masked positions to large negative value
+        # After softmax, these become ~0
+        scores = scores.masked_fill(mask == 0, -1e9)
+    
+    # Softmax to get attention probabilities
+    # Each row sums to 1: how much attention to pay to each position
+    attention_weights = softmax(scores, dim=-1)
+    
+    # Weighted sum of values based on attention
+    output = attention_weights @ V
+    
+    return output, attention_weights
+```
+
+**Multi-Head Attention - Parallel Attention Subspaces**
+```python
+class MultiHeadAttention:
+    """Multi-head attention from 'Attention is All You Need'.
+    
+    Allows model to jointly attend to information from different
+    representation subspaces at different positions.
+    
+    Architecture:
+        Input (batch, seq_len, d_model)
+          → Project to Q, K, V (each batch, seq_len, d_model)
+          → Split into H heads (batch, H, seq_len, d_model/H)
+          → Attention for each head in parallel
+          → Concatenate heads
+          → Final linear projection
+        Output (batch, seq_len, d_model)
+    
+    Example:
+        d_model = 512, num_heads = 8
+        Each head processes 512/8 = 64 dimensions
+        8 heads learn different attention patterns in parallel
+    """
+    def __init__(self, d_model, num_heads):
+        assert d_model % num_heads == 0
+        
+        self.d_model = d_model
+        self.num_heads = num_heads
+        self.d_k = d_model // num_heads  # Dimension per head
+        
+        # Linear projections for Q, K, V
+        self.W_q = Linear(d_model, d_model)
+        self.W_k = Linear(d_model, d_model)
+        self.W_v = Linear(d_model, d_model)
+        
+        # Output projection
+        self.W_o = Linear(d_model, d_model)
+    
+    def forward(self, query, key, value, mask=None):
+        """Multi-head attention forward pass.
+        
+        Args:
+            query: (batch, seq_len_q, d_model)
+            key: (batch, seq_len_k, d_model)
+            value: (batch, seq_len_v, d_model)
+            mask: Optional mask
+        
+        Returns:
+            output: (batch, seq_len_q, d_model)
+            attention_weights: (batch, num_heads, seq_len_q, seq_len_k)
+        """
+        batch_size = query.shape[0]
+        
+        # 1. Linear projections
+        Q = self.W_q(query)  # (batch, seq_len_q, d_model)
+        K = self.W_k(key)    # (batch, seq_len_k, d_model)
+        V = self.W_v(value)  # (batch, seq_len_v, d_model)
+        
+        # 2. Split into multiple heads
+        # Reshape: (batch, seq_len, d_model) → (batch, seq_len, num_heads, d_k)
+        # Transpose: → (batch, num_heads, seq_len, d_k)
+        Q = Q.reshape(batch_size, -1, self.num_heads, self.d_k).transpose(1, 2)
+        K = K.reshape(batch_size, -1, self.num_heads, self.d_k).transpose(1, 2)
+        V = V.reshape(batch_size, -1, self.num_heads, self.d_k).transpose(1, 2)
+        
+        # 3. Apply attention for each head in parallel
+        attended, attention_weights = scaled_dot_product_attention(Q, K, V, mask)
+        # attended: (batch, num_heads, seq_len_q, d_k)
+        
+        # 4. Concatenate heads
+        # Transpose: (batch, num_heads, seq_len, d_k) → (batch, seq_len, num_heads, d_k)
+        # Reshape: → (batch, seq_len, d_model)
+        attended = attended.transpose(1, 2).reshape(batch_size, -1, self.d_model)
+        
+        # 5. Final linear projection
+        output = self.W_o(attended)
+        
+        return output, attention_weights
+```
+
+**Masking Utilities**
+```python
+def create_causal_mask(seq_len):
+    """Create causal mask for autoregressive (GPT-style) attention.
+    
+    Prevents positions from attending to future positions.
+    Position i can only attend to positions <= i.
+    
+    Returns:
+        mask: (seq_len, seq_len) lower triangular matrix
+        
+    Example (seq_len=4):
+        [[1, 0, 0, 0],     # Position 0 sees only position 0
+         [1, 1, 0, 0],     # Position 1 sees 0,1
+         [1, 1, 1, 0],     # Position 2 sees 0,1,2
+         [1, 1, 1, 1]]     # Position 3 sees all
+    """
+    mask = np.tril(np.ones((seq_len, seq_len)))
+    return Tensor(mask)
+
+def create_padding_mask(lengths, max_length):
+    """Create padding mask to ignore padding tokens.
+    
+    Args:
+        lengths: (batch_size,) actual sequence lengths
+        max_length: maximum sequence length in batch
+    
+    Returns:
+        mask: (batch_size, 1, 1, max_length) where 1=real, 0=padding
+    """
+    batch_size = lengths.shape[0]
+    mask = np.zeros((batch_size, max_length))
+    for i, length in enumerate(lengths):
+        mask[i, :length] = 1
+    return Tensor(mask).reshape(batch_size, 1, 1, max_length)
+```
+
+### Step-by-Step Implementation
+
+1. **Implement Scaled Dot-Product Attention**
+   - Compute QK^T matmul
+   - Apply 1/√d_k scaling
+   - Add masking support
+   - Apply softmax and value weighting
+   - Verify attention weights sum to 1
+
+2. **Build Multi-Head Attention**
+   - Create Q, K, V projection layers
+   - Split embeddings into multiple heads
+   - Apply attention to each head in parallel
+   - Concatenate head outputs
+   - Add final projection layer
+
+3. **Add Masking Utilities**
+   - Implement causal mask for GPT-style models
+   - Create padding mask for variable-length sequences
+   - Test mask shapes and broadcasting
+   - Verify masking prevents information leak
+
+4. **Create Self-Attention Wrapper**
+   - Build convenience class where Q=K=V
+   - Add optional masking parameter
+   - Test with real embedded sequences
+   - Profile computational cost
+
+5. **Visualize Attention Patterns**
+   - Extract attention weights from forward pass
+   - Plot heatmaps for different heads
+   - Analyze what patterns each head learns
+   - Interpret attention on real text examples
+
+## Testing
+
+### Inline Tests (During Development)
+
+Run inline tests while building:
+```bash
+cd modules/12_attention
+python attention_dev.py
+```
+
+Expected output:
+```
+Unit Test: Scaled dot-product attention...
+✅ Attention scores computed correctly
+✅ Softmax normalization verified (sums to 1)
+✅ Output shape matches expected dimensions
+Progress: Attention Mechanism ✓
+
+Unit Test: Multi-head attention...
+✅ 8 heads process 512 dims in parallel
+✅ Head splitting and concatenation correct
+✅ Output projection applied properly
+Progress: Multi-Head Attention ✓
+
+Unit Test: Causal masking...
+✅ Future positions blocked correctly
+✅ Past positions accessible
+✅ Autoregressive property verified
+Progress: Masking ✓
+```
+
+### Export and Validate
+
+After completing the module:
+```bash
+# Export to tinytorch package
+tito export 12_attention
+
+# Run integration tests
+tito test 12_attention
+```
+
+## Where This Code Lives
+
+```
+tinytorch/
+├── nn/
+│   └── attention.py            # Your implementation goes here
+└── __init__.py                 # Exposes MultiHeadAttention, etc.
+
+Usage in other modules:
+>>> from tinytorch.nn import MultiHeadAttention
+>>> attn = MultiHeadAttention(d_model=512, num_heads=8)
+>>> output, weights = attn(query, key, value, mask=causal_mask)
+```
+
+## Systems Thinking Questions
+
+1. **Quadratic Complexity**: Attention is O(n²) in sequence length. For n=1024, we compute ~1M attention scores. For n=4096 (GPT-3 context), how many? Why is this a problem for long documents?
+
+2. **Multi-Head Benefits**: Why 8 heads of 64 dims each instead of 1 head of 512 dims? What different patterns might different heads learn (syntax vs semantics vs coreference)?
+
+3. **Scaling Factor Impact**: Without 1/√d_k scaling, softmax gets extreme values (nearly one-hot). Why? How does this hurt gradient flow? (Hint: softmax derivative)
+
+4. **Memory vs Compute**: Attention weights matrix is (batch × heads × seq × seq). For batch=32, heads=8, seq=1024, this is 256M values. At FP32, how much memory? Why is this a bottleneck?
+
+5. **Causal vs Bidirectional**: GPT uses causal masking (can't see future). BERT uses bidirectional (can see all positions). Why does this architectural choice define fundamentally different models?
+
+## Real-World Connections
+
+### Industry Applications
+
+**Large Language Models (OpenAI, Anthropic, Google)**
+- GPT-4: 96 layers × 128 heads = 12,288 attention computations
+- Attention optimizations (FlashAttention) critical for training at scale
+- Multi-query attention reduces inference cost in production
+- Attention is the primary computational bottleneck
+
+**Machine Translation (Google Translate, DeepL)**
+- Cross-attention aligns source and target languages
+- Attention weights show word alignment (interpretability)
+- Multi-head attention captures different translation patterns
+- Real-time translation requires optimized attention kernels
+
+**Vision Models (Google ViT, Meta DINOv2)**
+- Self-attention over image patches replaces convolution
+- Global receptive field from layer 1 (vs deep CNN stacks)
+- Attention scales better to high-resolution images
+- Now dominant architecture for vision tasks
+
+### Research Impact
+
+This module implements patterns from:
+- Attention is All You Need (Vaswani et al., 2017): The transformer paper
+- BERT (Devlin et al., 2018): Bidirectional attention for NLP
+- GPT-2/3 (Radford et al., 2019): Causal attention for generation
+- ViT (Dosovitskiy et al., 2020): Attention for computer vision
+
+## What's Next?
+
+In **Module 13: Transformers**, you'll compose attention into complete transformer blocks:
+
+- Stack multi-head attention with feedforward networks
+- Add layer normalization and residual connections
+- Build encoder (BERT-style) and decoder (GPT-style) architectures
+- Train full transformer on text generation tasks
+
+The attention mechanism you built is the core component of every transformer!
+
+---
+
+**Ready to build the AI revolution from scratch?** Open `modules/12_attention/attention_dev.py` and start implementing.

modules/13_transformers/ABOUT.md

@@ -0,0 +1,479 @@
+---
+title: "Transformers - Complete Encoder-Decoder Architecture"
+description: "Build full transformer models with encoder and decoder stacks"
+difficulty: 4
+time_estimate: "6-8 hours"
+prerequisites: ["Embeddings", "Attention"]
+next_steps: ["Memoization (Optimization Tier)"]
+learning_objectives:
+  - "Implement complete transformer blocks with attention and feedforward layers"
+  - "Design encoder stacks for bidirectional understanding (BERT-style)"
+  - "Build decoder stacks for autoregressive generation (GPT-style)"
+  - "Understand layer normalization and residual connections for deep networks"
+  - "Apply transformer architectures to language modeling and generation tasks"
+---
+
+# 13. Transformers
+
+**🏛️ ARCHITECTURE TIER** | Difficulty: ⭐⭐⭐⭐ (4/4) | Time: 6-8 hours
+
+## Overview
+
+Build complete transformer models by composing attention, feedforward, and normalization layers. This module implements encoder stacks (BERT-style) and decoder stacks (GPT-style) that power all modern language models.
+
+## Learning Objectives
+
+By completing this module, you will be able to:
+
+1. **Implement complete transformer blocks** with multi-head attention, feedforward networks, and normalization
+2. **Design encoder stacks** for bidirectional understanding using masked self-attention (BERT-style)
+3. **Build decoder stacks** for autoregressive text generation with causal masking (GPT-style)
+4. **Understand layer normalization and residual connections** critical for training deep transformer networks
+5. **Apply transformer architectures** to language modeling, text generation, and sequence-to-sequence tasks
+
+## Why This Matters
+
+### Production Context
+
+Transformers are the architecture of modern AI:
+
+- **GPT-4**: 96-layer decoder-only transformer; powers ChatGPT and GitHub Copilot
+- **BERT**: 12-layer encoder-only transformer; ranks billions of web pages for Google Search
+- **T5**: Encoder-decoder transformer; Google's universal text-to-text model
+- **Claude, Gemini, Llama**: All transformer-based; billions of users daily
+
+### Historical Context
+
+Transformers unified and dominated AI:
+
+- **Pre-Transformer (pre-2017)**: RNNs/LSTMs for sequences; CNNs for vision; separate architectures
+- **Attention is All You Need (2017)**: Pure transformer beats RNNs; parallelizable; scales efficiently
+- **BERT/GPT (2018)**: Transformers dominate NLP; pre-training + fine-tuning paradigm
+- **Transformers Everywhere (2020+)**: Vision (ViT), speech (Whisper), protein folding (AlphaFold), multimodal (GPT-4)
+
+The architecture you're implementing powers virtually all modern AI systems.
+
+## Pedagogical Pattern: Build → Use → Analyze
+
+### 1. Build
+
+Implement from first principles:
+- Feedforward network with two linear layers and activation
+- Layer normalization for training stability
+- Transformer block: attention → residual → norm → FFN → residual → norm
+- Encoder stack (bidirectional, BERT-style)
+- Decoder stack (autoregressive, GPT-style)
+
+### 2. Use
+
+Apply to real problems:
+- Train GPT-style decoder on Shakespeare text generation
+- Build BERT-style encoder for sequence classification
+- Implement encoder-decoder for sequence-to-sequence tasks
+- Generate text autoregressively with sampling
+- Compare encoder-only vs decoder-only architectures
+
+### 3. Analyze
+
+Deep-dive into architectural choices:
+- Why are residual connections critical for deep transformers?
+- How does layer normalization differ from batch normalization?
+- When would you use encoder-only vs decoder-only vs encoder-decoder?
+- Why pre-norm vs post-norm transformer blocks?
+- What's the compute/memory trade-off in stacking many layers?
+
+## Implementation Guide
+
+### Core Components
+
+**Feedforward Network - Position-Wise FFN**
+```python
+class FeedForward:
+    """Position-wise feedforward network in transformer.
+    
+    Two linear transformations with ReLU activation:
+        FFN(x) = ReLU(xW₁ + b₁)W₂ + b₂
+    
+    Applied identically to each position independently.
+    Typically d_ff = 4 × d_model (expansion factor).
+    
+    Args:
+        d_model: Input/output dimension (e.g., 512)
+        d_ff: Hidden dimension (e.g., 2048 = 4 × 512)
+        dropout: Dropout probability for regularization
+    """
+    def __init__(self, d_model, d_ff, dropout=0.1):
+        self.linear1 = Linear(d_model, d_ff)
+        self.linear2 = Linear(d_ff, d_model)
+        self.relu = ReLU()
+        self.dropout = Dropout(dropout)
+    
+    def forward(self, x):
+        # x: (batch, seq_len, d_model)
+        x = self.linear1(x)      # (batch, seq_len, d_ff)
+        x = self.relu(x)          # Nonlinearity
+        x = self.dropout(x)       # Regularization
+        x = self.linear2(x)       # (batch, seq_len, d_model)
+        return x
+```
+
+**Layer Normalization - Training Stability**
+```python
+class LayerNorm:
+    """Layer normalization for transformer training stability.
+    
+    Normalizes across feature dimension for each sample independently.
+    Unlike BatchNorm, works with any batch size including batch=1.
+    
+    Formula: y = γ(x - μ)/√(σ² + ε) + β
+    where μ, σ² computed per sample across features
+    
+    Why not BatchNorm?
+    - Transformers process variable-length sequences
+    - LayerNorm independent of batch size (better for inference)
+    - Empirically works better for NLP tasks
+    """
+    def __init__(self, d_model, eps=1e-6):
+        self.gamma = Parameter(Tensor.ones(d_model))   # Learned scale
+        self.beta = Parameter(Tensor.zeros(d_model))   # Learned shift
+        self.eps = eps
+    
+    def forward(self, x):
+        # x: (batch, seq_len, d_model)
+        mean = x.mean(dim=-1, keepdim=True)
+        std = x.std(dim=-1, keepdim=True)
+        normalized = (x - mean) / (std + self.eps)
+        return self.gamma * normalized + self.beta
+```
+
+**Transformer Block - Complete Layer**
+```python
+class TransformerBlock:
+    """Single transformer layer with attention and feedforward.
+    
+    Architecture (Pre-Norm variant):
+        x → LayerNorm → MultiHeadAttention → Residual
+          → LayerNorm → FeedForward → Residual
+    
+    Pre-Norm (shown above) vs Post-Norm:
+        - Pre-Norm: Normalize before sub-layers; better gradient flow
+        - Post-Norm: Normalize after sub-layers; original Transformer paper
+        - Pre-Norm generally preferred for deep models (>12 layers)
+    """
+    def __init__(self, d_model, num_heads, d_ff, dropout=0.1):
+        # Attention sub-layer
+        self.attention = MultiHeadAttention(d_model, num_heads)
+        self.norm1 = LayerNorm(d_model)
+        self.dropout1 = Dropout(dropout)
+        
+        # Feedforward sub-layer
+        self.feedforward = FeedForward(d_model, d_ff, dropout)
+        self.norm2 = LayerNorm(d_model)
+        self.dropout2 = Dropout(dropout)
+    
+    def forward(self, x, mask=None):
+        """Forward pass with residual connections.
+        
+        Args:
+            x: (batch, seq_len, d_model)
+            mask: Optional attention mask
+        
+        Returns:
+            output: (batch, seq_len, d_model)
+        """
+        # Attention sub-layer with residual
+        normed = self.norm1(x)
+        attended, _ = self.attention(normed, normed, normed, mask)
+        x = x + self.dropout1(attended)  # Residual connection
+        
+        # Feedforward sub-layer with residual
+        normed = self.norm2(x)
+        fed_forward = self.feedforward(normed)
+        x = x + self.dropout2(fed_forward)  # Residual connection
+        
+        return x
+```
+
+**GPT-Style Decoder - Autoregressive Generation**
+```python
+class GPTDecoder:
+    """GPT-style decoder for autoregressive language modeling.
+    
+    Architecture:
+        Input tokens → Embed + PositionalEncoding
+        → TransformerBlocks (with causal masking)
+        → Linear projection to vocabulary
+    
+    Features:
+        - Causal masking: position i can only attend to positions ≤ i
+        - Autoregressive: generates one token at a time
+        - Pre-training objective: predict next token
+    """
+    def __init__(self, vocab_size, d_model, num_layers, num_heads, d_ff, max_len):
+        # Embedding layers
+        self.token_embedding = Embedding(vocab_size, d_model)
+        self.position_embedding = LearnedPositionalEmbedding(max_len, d_model)
+        
+        # Transformer blocks
+        self.blocks = [TransformerBlock(d_model, num_heads, d_ff) 
+                       for _ in range(num_layers)]
+        
+        # Output projection
+        self.norm = LayerNorm(d_model)
+        self.output_proj = Linear(d_model, vocab_size)
+    
+    def forward(self, token_ids):
+        """Forward pass through decoder.
+        
+        Args:
+            token_ids: (batch, seq_len) token indices
+        
+        Returns:
+            logits: (batch, seq_len, vocab_size) unnormalized predictions
+        """
+        batch_size, seq_len = token_ids.shape
+        
+        # Embeddings
+        token_embeds = self.token_embedding(token_ids)
+        pos_embeds = self.position_embedding(seq_len)
+        x = token_embeds + pos_embeds  # (batch, seq_len, d_model)
+        
+        # Create causal mask
+        causal_mask = create_causal_mask(seq_len)
+        
+        # Transformer blocks
+        for block in self.blocks:
+            x = block(x, mask=causal_mask)
+        
+        # Output projection
+        x = self.norm(x)
+        logits = self.output_proj(x)  # (batch, seq_len, vocab_size)
+        
+        return logits
+    
+    def generate(self, start_tokens, max_new_tokens, temperature=1.0):
+        """Autoregressive text generation.
+        
+        Args:
+            start_tokens: (batch, start_len) initial sequence
+            max_new_tokens: Number of tokens to generate
+            temperature: Sampling temperature (higher = more random)
+        
+        Returns:
+            generated: (batch, start_len + max_new_tokens) full sequence
+        """
+        generated = start_tokens
+        
+        for _ in range(max_new_tokens):
+            # Forward pass
+            logits = self.forward(generated)  # (batch, seq_len, vocab_size)
+            
+            # Get logits for last position
+            next_token_logits = logits[:, -1, :] / temperature
+            
+            # Sample from distribution
+            probs = softmax(next_token_logits, dim=-1)
+            next_token = sample(probs)  # (batch, 1)
+            
+            # Append to sequence
+            generated = concat([generated, next_token], dim=1)
+        
+        return generated
+```
+
+**BERT-Style Encoder - Bidirectional Understanding**
+```python
+class BERTEncoder:
+    """BERT-style encoder for bidirectional sequence understanding.
+    
+    Architecture:
+        Input tokens → Embed + PositionalEncoding
+        → TransformerBlocks (no causal masking)
+        → Task-specific head (classification, QA, etc.)
+    
+    Features:
+        - Bidirectional: each position attends to all positions
+        - Pre-training: masked language modeling (MLM)
+        - Fine-tuning: task-specific heads added
+    """
+    def __init__(self, vocab_size, d_model, num_layers, num_heads, d_ff, max_len):
+        self.token_embedding = Embedding(vocab_size, d_model)
+        self.position_embedding = LearnedPositionalEmbedding(max_len, d_model)
+        
+        self.blocks = [TransformerBlock(d_model, num_heads, d_ff) 
+                       for _ in range(num_layers)]
+        
+        self.norm = LayerNorm(d_model)
+    
+    def forward(self, token_ids, attention_mask=None):
+        """Forward pass through encoder.
+        
+        Args:
+            token_ids: (batch, seq_len)
+            attention_mask: Optional mask for padding tokens
+        
+        Returns:
+            embeddings: (batch, seq_len, d_model) contextualized representations
+        """
+        # Embeddings
+        token_embeds = self.token_embedding(token_ids)
+        pos_embeds = self.position_embedding(token_ids.shape[1])
+        x = token_embeds + pos_embeds
+        
+        # Transformer blocks (bidirectional - no causal mask)
+        for block in self.blocks:
+            x = block(x, mask=attention_mask)
+        
+        x = self.norm(x)
+        return x
+```
+
+### Step-by-Step Implementation
+
+1. **Build Feedforward Network**
+   - Two linear layers with expansion factor (4×)
+   - Add ReLU activation between layers
+   - Include dropout for regularization
+   - Test with different d_ff values
+
+2. **Implement Layer Normalization**
+   - Compute mean and std across feature dimension
+   - Add learnable scale (gamma) and shift (beta)
+   - Handle numerical stability with epsilon
+   - Compare with batch normalization
+
+3. **Create Transformer Block**
+   - Add multi-head attention sub-layer
+   - Implement residual connections
+   - Add layer normalization (pre-norm placement)
+   - Include feedforward sub-layer
+   - Test forward and backward passes
+
+4. **Build GPT Decoder**
+   - Stack transformer blocks
+   - Add token and position embeddings
+   - Implement causal masking
+   - Add output projection to vocabulary
+   - Implement autoregressive generation
+
+5. **Build BERT Encoder**
+   - Stack transformer blocks without causal mask
+   - Add bidirectional attention
+   - Implement padding mask handling
+   - Test on classification tasks
+   - Compare with decoder architecture
+
+## Testing
+
+### Inline Tests (During Development)
+
+Run inline tests while building:
+```bash
+cd modules/13_transformers
+python transformers_dev.py
+```
+
+Expected output:
+```
+Unit Test: Transformer block...
+✅ Attention + FFN sub-layers work correctly
+✅ Residual connections preserve gradient flow
+✅ Layer normalization stabilizes training
+Progress: Transformer Block ✓
+
+Unit Test: GPT decoder...
+✅ 12-layer decoder initialized successfully
+✅ Causal masking prevents future information leak
+✅ Text generation produces coherent sequences
+Progress: GPT Decoder ✓
+
+Unit Test: BERT encoder...
+✅ Bidirectional attention accesses all positions
+✅ Padding mask ignores padding tokens correctly
+✅ Encoder outputs contextualized representations
+Progress: BERT Encoder ✓
+```
+
+### Export and Validate
+
+After completing the module:
+```bash
+# Export to tinytorch package
+tito export 13_transformers
+
+# Run integration tests
+tito test 13_transformers
+```
+
+## Where This Code Lives
+
+```
+tinytorch/
+├── models/
+│   ├── transformer.py          # Transformer blocks
+│   ├── gpt.py                  # GPT decoder
+│   └── bert.py                 # BERT encoder
+└── __init__.py                 # Exposes transformer models
+
+Usage in other modules:
+>>> from tinytorch.models import GPTDecoder, BERTEncoder
+>>> gpt = GPTDecoder(vocab_size=50000, d_model=768, num_layers=12, num_heads=12, d_ff=3072, max_len=1024)
+>>> generated_text = gpt.generate(start_tokens, max_new_tokens=100)
+```
+
+## Systems Thinking Questions
+
+1. **Layer Depth Trade-offs**: GPT-3 has 96 layers. What are the benefits? What are the challenges (training stability, memory, gradients)? Why can't we just use 1000 layers?
+
+2. **Residual Connections Necessity**: Remove residual connections from a 12-layer transformer. What happens during training? Why do gradients vanish? How do residuals solve this?
+
+3. **Pre-Norm vs Post-Norm**: Original Transformer used post-norm (norm after sub-layer). Modern transformers use pre-norm (norm before). Why? What's the gradient flow difference?
+
+4. **Encoder vs Decoder Choice**: When would you use encoder-only (BERT), decoder-only (GPT), or encoder-decoder (T5)? What tasks suit each architecture?
+
+5. **Memory Scaling**: A 12-layer transformer with d_model=768 has how many parameters? How does this scale with layers, dimensions, and vocabulary size? What's the memory footprint?
+
+## Real-World Connections
+
+### Industry Applications
+
+**Large Language Models (OpenAI, Anthropic, Google)**
+- GPT-4: 96-layer decoder stack, trained on trillions of tokens
+- Claude: Decoder-only architecture with constitutional AI training
+- PaLM 2: Decoder with 340B parameters across 64 layers
+- Gemini: Multimodal transformer processing text, images, audio
+
+**Search and Understanding (Google, Microsoft)**
+- BERT powers Google Search ranking for billions of queries daily
+- Bing uses transformer encoder for semantic search
+- Question-answering systems built on BERT fine-tuning
+- Document understanding and summarization
+
+**Code Generation (GitHub, Google, Meta)**
+- Copilot: GPT-based decoder trained on GitHub code
+- AlphaCode: Transformer decoder for competitive programming
+- CodeLlama: Specialized decoder for code completion
+- All use decoder-only transformer architecture
+
+### Research Impact
+
+This module implements patterns from:
+- Transformer (Vaswani et al., 2017): The foundational architecture
+- BERT (Devlin et al., 2018): Bidirectional encoder pre-training
+- GPT-2/3 (Radford et al., 2019): Decoder-only scaling
+- T5 (Raffel et al., 2020): Unified encoder-decoder framework
+
+## What's Next?
+
+In **Module 14: Profiling** (Optimization Tier), you'll learn to measure and analyze performance:
+
+- Profile time, memory, and compute for transformer operations
+- Identify bottlenecks in attention, feedforward, and embedding layers
+- Measure FLOPs and memory bandwidth utilization
+- Build the foundation for data-driven optimization
+
+The transformers you built are complete—now it's time to understand their performance characteristics!
+
+---
+
+**Ready to build GPT and BERT from scratch?** Open `modules/13_transformers/transformers_dev.py` and start implementing.

modules/14_profiling/ABOUT.md

@@ -0,0 +1,451 @@
+---
+title: "Profiling - Performance Analysis and Optimization"
+description: "Build profilers to identify bottlenecks and guide optimization decisions"
+difficulty: 3
+time_estimate: "5-6 hours"
+prerequisites: ["All modules 01-13"]
+next_steps: ["Memoization"]
+learning_objectives:
+  - "Implement timing profilers with statistical rigor for accurate measurements"
+  - "Design memory profilers to track allocation patterns and identify leaks"
+  - "Build FLOP counters to measure computational complexity"
+  - "Understand performance bottlenecks across different architectures"
+  - "Apply data-driven analysis to guide optimization priorities"
+---
+
+# 14. Profiling
+
+**⚡ OPTIMIZATION TIER** | Difficulty: ⭐⭐⭐ (3/4) | Time: 5-6 hours
+
+## Overview
+
+Build comprehensive profiling tools to measure where time and memory go in your ML systems. This module implements timing profilers, memory trackers, and FLOP counters that reveal bottlenecks and guide optimization decisions.
+
+## Learning Objectives
+
+By completing this module, you will be able to:
+
+1. **Implement timing profilers** with statistical rigor (multiple runs, confidence intervals) for accurate measurements
+2. **Design memory profilers** to track allocation patterns, peak usage, and identify memory leaks
+3. **Build FLOP counters** to measure theoretical computational complexity of different operations
+4. **Understand performance bottlenecks** by comparing MLPs, CNNs, and Transformers systematically
+5. **Apply data-driven analysis** to prioritize optimization efforts based on actual impact
+
+## Why This Matters
+
+### Production Context
+
+Profiling is mandatory for production ML systems:
+
+- **Google TPU teams** profile every operation to optimize hardware utilization
+- **OpenAI** profiles GPT training to identify $millions in compute savings
+- **Meta** profiles inference to serve billions of requests per day efficiently  
+- **NVIDIA** uses profiling to optimize cuDNN kernels for peak performance
+
+### Historical Context
+
+Profiling evolved with ML scale:
+
+- **Early ML (pre-2012)**: Ad-hoc timing with `time.time()`; no systematic profiling
+- **Deep Learning Era (2012-2017)**: NVIDIA profiler, TensorBoard timing; focus on GPU utilization
+- **Production Scale (2018+)**: Comprehensive profiling (compute, memory, I/O, network); optimization critical for economics
+- **Modern Systems (2020+)**: Automated profiling and optimization; ML compilers use profiling data
+
+Without profiling, you're optimizing blind—profiling shows you where to focus.
+
+## Pedagogical Pattern: Build → Use → Optimize
+
+### 1. Build
+
+Implement from first principles:
+- High-precision timing with multiple runs
+- Statistical analysis (mean, std, confidence intervals)
+- Memory profiler tracking allocations and deallocations
+- FLOP counter for theoretical complexity
+- Comparative profiler across architectures
+
+### 2. Use
+
+Apply to real problems:
+- Profile attention vs feedforward in transformers
+- Compare MLP vs CNN vs Transformer efficiency
+- Identify memory bottlenecks in training loops
+- Measure impact of batch size on throughput
+- Analyze scaling behavior with model size
+
+### 3. Optimize
+
+Production insights:
+- Prioritize optimizations by impact (80/20 rule)
+- Measure before/after optimization
+- Understand hardware utilization (CPU vs GPU)
+- Identify memory bandwidth vs compute bottlenecks
+- Build optimization roadmap based on data
+
+## Implementation Guide
+
+### Core Components
+
+**High-Precision Timer**
+```python
+class Timer:
+    """High-precision timing with statistical analysis.
+    
+    Performs multiple runs to account for variance and noise.
+    Reports mean, std, and confidence intervals.
+    
+    Example:
+        timer = Timer()
+        with timer:
+            model.forward(x)
+        print(f"Time: {timer.mean:.3f}ms ± {timer.std:.3f}ms")
+    """
+    def __init__(self, num_runs=10, warmup_runs=3):
+        self.num_runs = num_runs
+        self.warmup_runs = warmup_runs
+        self.times = []
+    
+    def __enter__(self):
+        # Warmup runs (not counted)
+        for _ in range(self.warmup_runs):
+            start = time.perf_counter()
+            # Operation happens in with block
+        
+        # Timed runs
+        self.start_time = time.perf_counter()
+        return self
+    
+    def __exit__(self, *args):
+        elapsed = time.perf_counter() - self.start_time
+        self.times.append(elapsed * 1000)  # Convert to ms
+    
+    @property
+    def mean(self):
+        return np.mean(self.times)
+    
+    @property
+    def std(self):
+        return np.std(self.times)
+    
+    @property
+    def confidence_interval(self, confidence=0.95):
+        """95% confidence interval using t-distribution."""
+        from scipy import stats
+        ci = stats.t.interval(confidence, len(self.times)-1,
+                              loc=self.mean, scale=stats.sem(self.times))
+        return ci
+    
+    def report(self):
+        ci = self.confidence_interval()
+        return f"{self.mean:.3f}ms ± {self.std:.3f}ms (95% CI: [{ci[0]:.3f}, {ci[1]:.3f}])"
+```
+
+**Memory Profiler**
+```python
+class MemoryProfiler:
+    """Track memory allocations and peak usage.
+    
+    Monitors memory throughout execution to identify:
+    - Peak memory usage
+    - Memory leaks
+    - Allocation patterns
+    - Memory bandwidth bottlenecks
+    """
+    def __init__(self):
+        self.snapshots = []
+        self.peak_memory = 0
+    
+    def snapshot(self, label=""):
+        """Take memory snapshot at current point."""
+        import psutil
+        process = psutil.Process()
+        mem_info = process.memory_info()
+        
+        snapshot = {
+            'label': label,
+            'rss': mem_info.rss / 1024**2,  # MB
+            'vms': mem_info.vms / 1024**2,  # MB
+            'timestamp': time.time()
+        }
+        self.snapshots.append(snapshot)
+        self.peak_memory = max(self.peak_memory, snapshot['rss'])
+        
+        return snapshot
+    
+    def report(self):
+        """Generate memory usage report."""
+        print(f"Peak Memory: {self.peak_memory:.2f} MB")
+        print("\nMemory Timeline:")
+        for snap in self.snapshots:
+            print(f"  {snap['label']:30s}: {snap['rss']:8.2f} MB")
+        
+        # Calculate memory growth
+        if len(self.snapshots) >= 2:
+            growth = self.snapshots[-1]['rss'] - self.snapshots[0]['rss']
+            print(f"\nTotal Growth: {growth:+.2f} MB")
+            
+            # Check for potential memory leak
+            if growth > 100:  # Arbitrary threshold
+                print("⚠️  Potential memory leak detected!")
+```
+
+**FLOP Counter**
+```python
+class FLOPCounter:
+    """Count floating-point operations for complexity analysis.
+    
+    Provides theoretical computational complexity independent of hardware.
+    Useful for comparing different architectural choices.
+    """
+    def __init__(self):
+        self.total_flops = 0
+        self.op_counts = {}
+    
+    def count_matmul(self, A_shape, B_shape):
+        """Count FLOPs for matrix multiplication.
+        
+        C = A @ B where A is (m, k) and B is (k, n)
+        FLOPs = 2*m*k*n (multiply-add for each output element)
+        """
+        m, k = A_shape
+        k2, n = B_shape
+        assert k == k2, "Invalid matmul dimensions"
+        
+        flops = 2 * m * k * n
+        self.total_flops += flops
+        self.op_counts['matmul'] = self.op_counts.get('matmul', 0) + flops
+        return flops
+    
+    def count_attention(self, batch, seq_len, d_model, num_heads):
+        """Count FLOPs for multi-head attention.
+        
+        Components:
+        - Q,K,V projections: 3 * (batch * seq_len * d_model * d_model)
+        - Attention scores: batch * heads * seq_len * seq_len * d_k
+        - Attention weighting: batch * heads * seq_len * seq_len * d_v
+        - Output projection: batch * seq_len * d_model * d_model
+        """
+        d_k = d_model // num_heads
+        
+        # QKV projections
+        qkv_flops = 3 * self.count_matmul((batch * seq_len, d_model), (d_model, d_model))
+        
+        # Attention computation
+        scores_flops = batch * num_heads * seq_len * seq_len * d_k * 2
+        weights_flops = batch * num_heads * seq_len * seq_len * d_k * 2
+        attention_flops = scores_flops + weights_flops
+        
+        # Output projection
+        output_flops = self.count_matmul((batch * seq_len, d_model), (d_model, d_model))
+        
+        total = qkv_flops + attention_flops + output_flops
+        self.op_counts['attention'] = self.op_counts.get('attention', 0) + total
+        return total
+    
+    def report(self):
+        """Generate FLOP report with breakdown."""
+        print(f"Total FLOPs: {self.total_flops / 1e9:.2f} GFLOPs")
+        print("\nBreakdown by operation:")
+        for op, flops in sorted(self.op_counts.items(), key=lambda x: x[1], reverse=True):
+            percentage = (flops / self.total_flops) * 100
+            print(f"  {op:20s}: {flops/1e9:8.2f} GFLOPs ({percentage:5.1f}%)")
+```
+
+**Architecture Profiler - Comparative Analysis**
+```python
+class ArchitectureProfiler:
+    """Compare performance across different architectures.
+    
+    Profiles MLP, CNN, and Transformer on same task to understand
+    compute/memory trade-offs.
+    """
+    def __init__(self):
+        self.results = {}
+    
+    def profile_model(self, model, input_data, model_name):
+        """Profile a model comprehensively."""
+        result = {
+            'model_name': model_name,
+            'parameters': count_parameters(model),
+            'timing': {},
+            'memory': {},
+            'flops': {}
+        }
+        
+        # Timing profile
+        timer = Timer(num_runs=10)
+        for _ in range(timer.num_runs + timer.warmup_runs):
+            with timer:
+                output = model.forward(input_data)
+        result['timing']['forward'] = timer.mean
+        
+        # Memory profile
+        mem = MemoryProfiler()
+        mem.snapshot("Before forward")
+        output = model.forward(input_data)
+        mem.snapshot("After forward")
+        result['memory']['peak'] = mem.peak_memory
+        
+        # FLOP count
+        flop_counter = FLOPCounter()
+        # Count FLOPs based on model architecture
+        result['flops']['total'] = flop_counter.total_flops
+        
+        self.results[model_name] = result
+        return result
+    
+    def compare(self):
+        """Generate comparative report."""
+        print("\nArchitecture Comparison")
+        print("=" * 80)
+        
+        for name, result in self.results.items():
+            print(f"\n{name}:")
+            print(f"  Parameters: {result['parameters']/1e6:.2f}M")
+            print(f"  Forward time: {result['timing']['forward']:.3f}ms")
+            print(f"  Peak memory: {result['memory']['peak']:.2f}MB")
+            print(f"  FLOPs: {result['flops']['total']/1e9:.2f}GFLOPs")
+```
+
+### Step-by-Step Implementation
+
+1. **Build High-Precision Timer**
+   - Use `time.perf_counter()` for nanosecond precision
+   - Implement multiple runs with warmup
+   - Calculate mean, std, confidence intervals
+   - Test with known delays
+
+2. **Implement Memory Profiler**
+   - Track memory at key points (before/after operations)
+   - Calculate peak memory usage
+   - Identify memory growth patterns
+   - Detect potential leaks
+
+3. **Create FLOP Counter**
+   - Count operations for matmul, convolution, attention
+   - Build hierarchical counting (operation → layer → model)
+   - Compare theoretical vs actual performance
+   - Identify compute-bound vs memory-bound operations
+
+4. **Build Architecture Profiler**
+   - Profile MLP on MNIST/CIFAR
+   - Profile CNN on CIFAR
+   - Profile Transformer on text
+   - Generate comparative reports
+
+5. **Analyze Results**
+   - Identify bottleneck operations (Pareto principle)
+   - Compare efficiency across architectures
+   - Understand scaling behavior
+   - Prioritize optimization opportunities
+
+## Testing
+
+### Inline Tests
+
+Run inline tests while building:
+```bash
+cd modules/15_profiling
+python profiling_dev.py
+```
+
+Expected output:
+```
+Unit Test: Timer with statistical analysis...
+✅ Multiple runs produce consistent results
+✅ Confidence intervals computed correctly
+✅ Warmup runs excluded from statistics
+Progress: Timing Profiler ✓
+
+Unit Test: Memory profiler...
+✅ Snapshots capture memory correctly
+✅ Peak memory tracked accurately
+✅ Memory growth detected
+Progress: Memory Profiler ✓
+
+Unit Test: FLOP counter...
+✅ Matmul FLOPs: 2*m*k*n verified
+✅ Attention FLOPs match theoretical
+✅ Operation breakdown correct
+Progress: FLOP Counter ✓
+```
+
+### Export and Validate
+
+```bash
+tito export 15_profiling
+tito test 15_profiling
+```
+
+## Where This Code Lives
+
+```
+tinytorch/
+├── profiler/
+│   └── profiling.py            # Your implementation goes here
+└── __init__.py                 # Exposes Timer, MemoryProfiler, etc.
+
+Usage:
+>>> from tinytorch.profiler import Timer, MemoryProfiler, FLOPCounter
+>>> timer = Timer()
+>>> with timer:
+>>>     model.forward(x)
+>>> print(timer.report())
+```
+
+## Systems Thinking Questions
+
+1. **Amdahl's Law**: If attention is 70% of compute and you optimize it 2×, what's the overall speedup? Why can't you get 2× end-to-end speedup?
+
+2. **Memory vs Compute Bottlenecks**: Your GPU can do 100 TFLOPs/s but memory bandwidth is 900 GB/s. For FP32 operations needing 4 bytes/FLOP, what's the bottleneck? When?
+
+3. **Batch Size Impact**: Doubling batch size doesn't double throughput. Why? What's the relationship between batch size, memory, and throughput?
+
+4. **Profiling Overhead**: Your profiler adds 5% overhead. Is this acceptable? When would you use sampling profilers vs instrumentation profilers?
+
+5. **Hardware Differences**: Your code runs 10× slower on CPU than GPU for large matrices, but only 2× slower for small ones. Why? What's the crossover point?
+
+## Real-World Connections
+
+### Industry Applications
+
+**Google TPU Optimization**
+- Profile every kernel to maximize TPU utilization
+- Optimize for both FLOPs and memory bandwidth
+- Use profiling to guide hardware design decisions
+- Achieve 40-50% utilization (very high for accelerators)
+
+**OpenAI Training Optimization**
+- Profile GPT training to find $millions in savings
+- Identify gradient checkpointing opportunities
+- Optimize data loading pipelines
+- Achieve 50%+ MFU (model FLOPs utilization)
+
+**Meta Inference Serving**
+- Profile PyTorch models for production deployment
+- Identify operator fusion opportunities
+- Optimize for latency (p50, p99) not just throughput
+- Serve billions of requests per day efficiently
+
+### Research Impact
+
+This module implements patterns from:
+- TensorBoard Profiler (Google, 2019): Visual profiling for TensorFlow
+- PyTorch Profiler (Meta, 2020): Comprehensive profiling for PyTorch
+- NVIDIA Nsight (2021): GPU-specific profiling and optimization
+- MLPerf (2022): Standardized benchmarking and profiling
+
+## What's Next?
+
+In **Module 15: Quantization**, you'll use your profiling data to compress models:
+
+- Reduce precision from FP32 to INT8 for 4× memory savings
+- Implement calibration strategies to minimize accuracy loss
+- Measure memory and speed improvements
+- Apply quantization based on profiling insights
+
+Profiling shows you *what* to optimize—the next modules show you *how* to optimize it!
+
+---
+
+**Ready to become a performance detective?** Open `modules/14_profiling/profiling_dev.py` and start implementing.

modules/15_quantization/ABOUT.md

@@ -3,7 +3,7 @@ title: "Quantization - Reduced Precision for Efficiency"
 description: "INT8 quantization, calibration, and mixed-precision strategies"
 difficulty: 3
 time_estimate: "5-6 hours"
-prerequisites: ["Profiling", "Memoization"]
+prerequisites: ["Profiling"]
 next_steps: ["Compression"]
 learning_objectives:
   - "Implement INT8 quantization for weights and activations"
@@ -13,7 +13,7 @@ learning_objectives:
   - "Measure memory and speed improvements from reduced precision"
 ---
 
-# 16. Quantization
+# 15. Quantization
 
 **⚡ OPTIMIZATION TIER** | Difficulty: ⭐⭐⭐ (3/4) | Time: 5-6 hours
 
@@ -74,8 +74,8 @@ Dequantization: x_fp32 = (x_int8 - zero_point) * scale
 ## Testing
 
 ```bash
-tito export 17_quantization
-tito test 17_quantization
+tito export 15_quantization
+tito test 15_quantization
 ```
 
 ## Where This Code Lives
@@ -103,11 +103,11 @@ tinytorch/
 
 ## What's Next?
 
-In **Module 18: Compression**, you'll combine quantization with pruning:
+In **Module 16: Compression**, you'll combine quantization with pruning:
 - Remove unimportant weights (pruning)
 - Quantize remaining weights (INT8)
 - Achieve 10-50× compression with minimal accuracy loss
 
 ---
 
-**Ready to quantize models?** Open `modules/17_quantization/quantization_dev.py` and start implementing.
+**Ready to quantize models?** Open `modules/15_quantization/quantization_dev.py` and start implementing.

modules/16_compression/ABOUT.md

@@ -13,7 +13,7 @@ learning_objectives:
   - "Measure compression ratios and inference speedups"
 ---
 
-# 17. Compression
+# 16. Compression
 
 **⚡ OPTIMIZATION TIER** | Difficulty: ⭐⭐⭐ (3/4) | Time: 5-6 hours
 
@@ -81,8 +81,8 @@ Compression is now standard for deployment, not optional.
 ## Testing
 
 ```bash
-tito export 18_compression
-tito test 18_compression
+tito export 16_compression
+tito test 16_compression
 ```
 
 ## Where This Code Lives
@@ -110,12 +110,12 @@ tinytorch/
 
 ## What's Next?
 
-In **Module 19: Benchmarking**, you'll measure everything you've built:
-- Fair comparison across optimizations
-- Statistical significance testing
-- MLPerf-style benchmarking protocols
-- Comprehensive performance reports
+In **Module 17: Memoization**, you'll learn computational reuse:
+- KV-caching for transformers
+- Eliminate redundant computation
+- 10-15× speedup for autoregressive generation
+- Memory-compute trade-offs
 
 ---
 
-**Ready to compress models?** Open `modules/18_compression/compression_dev.py` and start implementing.
+**Ready to compress models?** Open `modules/16_compression/compression_dev.py` and start implementing.

modules/17_memoization/ABOUT.md

@@ -3,8 +3,8 @@ title: "Memoization - Computational Reuse for Inference"
 description: "Apply memoization pattern to transformers through KV caching for 10-15x faster generation"
 difficulty: 2
 time_estimate: "4-5 hours"
-prerequisites: ["Profiling", "Transformers"]
-next_steps: ["Quantization"]
+prerequisites: ["Profiling", "Transformers", "Quantization", "Compression"]
+next_steps: ["Acceleration"]
 learning_objectives:
   - "Understand memoization as a fundamental optimization pattern"
   - "Apply memoization to transformers through KV caching"
@@ -13,7 +13,7 @@ learning_objectives:
   - "Recognize when computational reuse applies to other problems"
 ---
 
-# 15. Memoization
+# 17. Memoization
 
 **⚡ OPTIMIZATION TIER** | Difficulty: ⭐⭐ (2/4) | Time: 4-5 hours
 
@@ -368,10 +368,10 @@ Progress: Cached Generation ✓
 After completing the module:
 ```bash
 # Export to tinytorch package
-tito export 14_kvcaching
+tito export 17_memoization
 
 # Run integration tests
-tito test 14_kvcaching
+tito test 17_memoization
 ```
 
 ## Where This Code Lives
@@ -432,15 +432,15 @@ This module implements patterns from:
 
 ## What's Next?
 
-In **Module 14: Profiling**, you measured where time goes in your transformer. Now you'll fix the bottleneck:
+In **Module 18: Acceleration**, you'll learn hardware-aware optimization:
 
-- Profile attention, feedforward, and embedding operations
-- Identify computational bottlenecks beyond caching
-- Measure FLOPs, memory bandwidth, and latency
-- Understand performance characteristics across architectures
+- Vectorization and SIMD operations
+- Batch processing for GPU efficiency
+- Hardware-specific optimizations
+- Parallel computation strategies
 
-The caching you implemented solves the biggest inference bottleneck—now let's find what else to optimize!
+You've compressed models (Quantization + Compression) and now you're learning computational reuse (Memoization). Next, you'll accelerate computation through parallelism!
 
 ---
 
-**Ready to implement production-critical caching?** Open `modules/14_kvcaching/kvcaching_dev.py` and start implementing.
+**Ready to implement production-critical caching?** Open `modules/17_memoization/memoization_dev.py` and start implementing.

modules/18_acceleration/ABOUT.md

@@ -0,0 +1,149 @@
+---
+title: "Acceleration - Hardware-Aware Optimization"
+description: "Optimize ML operations with SIMD, cache-friendly algorithms, and parallel computing"
+difficulty: 3
+time_estimate: "6-8 hours"
+prerequisites: ["Profiling", "Compression"]
+next_steps: ["Benchmarking"]
+learning_objectives:
+  - "Implement cache-friendly algorithms for matrix operations"
+  - "Apply SIMD vectorization for parallel data processing"
+  - "Design multi-core parallelization strategies for batch operations"
+  - "Understand hardware bottlenecks (compute vs memory bandwidth)"
+  - "Optimize ML kernels based on profiling data from Module 14"
+---
+
+# 18. Acceleration
+
+**⚡ OPTIMIZATION TIER** | Difficulty: ⭐⭐⭐ (3/4) | Time: 6-8 hours
+
+## Overview
+
+Optimize ML operations through hardware-aware programming. This module implements cache-friendly algorithms, SIMD vectorization, and multi-core parallelization to achieve significant speedups based on profiling insights from Module 14.
+
+## Learning Objectives
+
+By completing this module, you will be able to:
+
+1. **Implement cache-friendly algorithms** for matrix multiplication and convolution using blocked algorithms
+2. **Apply SIMD vectorization** to parallelize element-wise operations across data
+3. **Design multi-core parallelization strategies** for batch processing and data parallelism
+4. **Understand hardware bottlenecks** (compute-bound vs memory-bound operations)
+5. **Optimize ML kernels** based on actual profiling data, achieving measurable speedups
+
+## Why This Matters
+
+### Production Context
+
+Hardware optimization is critical for production ML:
+
+- **PyTorch** uses custom CUDA kernels and CPU vectorization; 100× faster than naive Python
+- **TensorFlow XLA** compiles models to optimized machine code; reduces latency by 2-5×
+- **ONNX Runtime** applies hardware-specific optimizations; powers Microsoft/Azure ML serving
+- **Apple Neural Engine** uses custom accelerators; enables on-device ML on iPhones
+
+### Historical Context
+
+Hardware optimization evolved with ML scale:
+
+- **Pre-Deep Learning (pre-2010)**: Hand-written assembly for critical loops; library implementations
+- **GPU Era (2010-2017)**: CUDA kernels dominate; cuDNN becomes standard; 10-100× speedups
+- **Specialized Hardware (2018+)**: TPUs, custom ASICs; compiler-based optimization
+- **Modern Systems (2020+)**: ML compilers (TVM, XLA); automated kernel generation and tuning
+
+Understanding hardware optimization separates production engineers from researchers.
+
+## Pedagogical Pattern: Build → Use → Optimize
+
+### 1. Build
+
+Implement from first principles:
+- Blocked matrix multiplication for cache efficiency
+- SIMD-vectorized element-wise operations
+- Multi-threaded batch processing
+- Memory-aligned data structures
+- Profiling integration
+
+### 2. Use
+
+Apply to real problems:
+- Optimize bottlenecks identified in Module 14
+- Accelerate attention computation
+- Speed up convolutional operations
+- Parallelize data loading pipelines
+- Measure actual speedups
+
+### 3. Optimize
+
+Production techniques:
+- Auto-tuning for different hardware
+- Mixed-precision computation (FP16/FP32)
+- Operator fusion to reduce memory traffic
+- Batch processing for amortized overhead
+- Hardware-specific code paths
+
+## Implementation Guide
+
+### Core Patterns
+
+**Cache-Friendly Matrix Multiplication**
+- Block matrices into cache-sized tiles
+- Reuse data while in cache (temporal locality)
+- Access memory sequentially (spatial locality)
+- Typical speedup: 2-5× over naive implementation
+
+**SIMD Vectorization**
+- Process multiple data elements simultaneously
+- Use Numba/Cython for automatic vectorization
+- Align data to vector boundaries (16/32/64 bytes)
+- Typical speedup: 2-8× for element-wise ops
+
+**Multi-Core Parallelization**
+- Divide work across CPU cores
+- Use thread pools for batch processing
+- Minimize synchronization overhead
+- Typical speedup: 0.5-0.8× number of cores (due to overhead)
+
+## Testing
+
+```bash
+cd modules/16_acceleration
+python acceleration_dev.py
+tito export 16_acceleration
+tito test 16_acceleration
+```
+
+## Where This Code Lives
+
+```
+tinytorch/
+├── acceleration/
+│   └── kernels.py              # Optimized implementations
+└── __init__.py
+```
+
+## Systems Thinking Questions
+
+1. **Roofline Model**: Your operation needs 1000 FLOPs and 100 bytes. At 100 GFLOPs/s compute and 10 GB/s bandwidth, what's the bottleneck?
+
+2. **Amdahl's Law Applied**: You parallelize 90% of code perfectly across 8 cores. What's max speedup? Why not 8×?
+
+3. **Cache Hierarchy**: L1 cache is 10× faster than L2, which is 10× faster than RAM. How does blocking matrix multiplication exploit this?
+
+## Real-World Connections
+
+**PyTorch/TensorFlow**: Custom CUDA kernels for all operations
+**ONNX Runtime**: Hardware-specific optimization for production serving
+**Apple ML**: Metal shaders and Neural Engine for on-device inference
+
+## What's Next?
+
+In **Module 19: Benchmarking**, you'll rigorously measure all optimizations:
+- Fair comparison across optimization techniques
+- Statistical significance testing
+- MLPerf-style benchmarking protocols
+- Comprehensive performance reports
+
+---
+
+**Ready to optimize for hardware?** Open `modules/18_acceleration/acceleration_dev.py` and start implementing.

modules/19_benchmarking/ABOUT.md

@@ -0,0 +1,118 @@
+---
+title: "Benchmarking - Fair Performance Comparison"
+description: "MLPerf-style benchmarking with statistical rigor and standardized metrics"
+difficulty: 3
+time_estimate: "5-6 hours"
+prerequisites: ["Profiling", "All optimization techniques"]
+next_steps: ["Competition (Capstone)"]
+learning_objectives:
+  - "Implement MLPerf-inspired benchmarking frameworks"
+  - "Design fair comparison protocols across different hardware"
+  - "Apply statistical significance testing to performance claims"
+  - "Build normalized metrics for hardware-independent comparison"
+  - "Generate comprehensive performance reports with visualizations"
+---
+
+# 19. Benchmarking
+
+**⚡ OPTIMIZATION TIER** | Difficulty: ⭐⭐⭐ (3/4) | Time: 5-6 hours
+
+## Overview
+
+Build rigorous benchmarking systems following MLPerf principles. This module implements fair comparison protocols, statistical testing, and normalized metrics for evaluating all the optimizations you've built in the Optimization Tier.
+
+## Learning Objectives
+
+By completing this module, you will be able to:
+
+1. **Implement MLPerf-inspired benchmarking** frameworks with standardized scenarios
+2. **Design fair comparison protocols** accounting for hardware differences
+3. **Apply statistical significance testing** to validate performance claims
+4. **Build normalized metrics** (speedup, compression ratio, efficiency scores)
+5. **Generate comprehensive reports** with visualizations and actionable insights
+
+## Why This Matters
+
+### Production Context
+
+Benchmarking drives ML systems decisions:
+
+- **MLPerf** standardizes ML benchmarking; companies compete on leaderboards
+- **Google TPU** teams use rigorous benchmarking to justify hardware investments
+- **Meta PyTorch** benchmarks every optimization before merging to production
+- **OpenAI** benchmarks training efficiency to optimize $millions in compute costs
+
+### Historical Context
+
+- **Pre-2018**: Ad-hoc benchmarking; inconsistent metrics; hard to compare
+- **MLPerf Launch (2018)**: Standardized benchmarks; reproducible results
+- **2019-2021**: MLPerf Training and Inference; industry adoption
+- **2021+**: MLPerf Tiny, Mobile; benchmarking for edge deployment
+
+Without rigorous benchmarking, optimization claims are meaningless.
+
+## Implementation Guide
+
+### Core Components
+
+**MLPerf Principles**
+1. **Reproducibility**: Fixed random seeds, documented environment
+2. **Fairness**: Same workload, measured on same hardware
+3. **Realism**: Representative tasks (ResNet, BERT, etc.)
+4. **Transparency**: Open-source code and results
+
+**Normalized Metrics**
+- **Speedup**: baseline_time / optimized_time
+- **Compression Ratio**: baseline_size / compressed_size
+- **Accuracy Delta**: optimized_accuracy - baseline_accuracy
+- **Efficiency Score**: (speedup × compression) / (1 + accuracy_loss)
+
+**Statistical Rigor**
+- Multiple runs (typically 10+)
+- Confidence intervals
+- Significance testing (t-test, Mann-Whitney)
+- Report variance, not just mean
+
+## Testing
+
+```bash
+tito export 19_benchmarking
+tito test 19_benchmarking
+```
+
+## Where This Code Lives
+
+```
+tinytorch/
+├── benchmarking/
+│   └── benchmark.py
+└── __init__.py
+```
+
+## Systems Thinking Questions
+
+1. **Hardware Normalization**: How do you compare optimizations across M1 Mac vs Intel vs AMD? What metrics are fair?
+
+2. **Statistical Power**: You measure 5% speedup with p=0.06. Is this significant? How many runs do you need?
+
+3. **Benchmark Selection**: MLPerf uses ResNet-50. Does this represent all workloads? What about transformers, GANs, RL?
+
+## Real-World Connections
+
+**MLPerf**: Industry-standard benchmarking consortium
+**SPEC**: Hardware benchmarking standards
+**TensorFlow/PyTorch**: Continuous benchmarking in CI/CD
+
+## What's Next?
+
+In **Module 20: TinyMLPerf Competition** (Capstone), you'll apply everything:
+- Use all Optimization Tier techniques
+- Compete on a standardized benchmark
+- Submit results to a leaderboard
+- Demonstrate complete ML systems skills
+
+This is your capstone—show what you've learned!
+
+---
+
+**Ready to benchmark rigorously?** Open `modules/19_benchmarking/benchmarking_dev.py` and start implementing.

modules/20_capstone/ABOUT.md

@@ -0,0 +1,279 @@
+---
+title: "MLPerf® Edu Competition - Your Capstone Challenge"
+description: "Apply all optimizations in a standardized MLPerf-inspired educational competition"
+difficulty: 5
+time_estimate: "10-20 hours"
+prerequisites: ["All modules 01-19"]
+next_steps: []
+learning_objectives:
+  - "Apply all Optimization Tier techniques to a standardized benchmark"
+  - "Implement either Closed Division (optimize given model) or Open Division (innovate architecture)"
+  - "Generate validated submission with normalized metrics"
+  - "Demonstrate complete ML systems engineering skills"
+  - "Compete fairly across different hardware platforms"
+---
+
+# 20. MLPerf® Edu Competition
+
+**🏆 CAPSTONE** | Difficulty: ⭐⭐⭐⭐⭐ (5/5 - Ninja Level) | Time: 10-20 hours
+
+## Overview
+
+Your capstone challenge: optimize a CIFAR-10 CNN using everything you've learned. Choose between Closed Division (optimize our CNN) or Open Division (design your own). Compete on a level playing field with normalized metrics that account for hardware differences.
+
+## Learning Objectives
+
+By completing this capstone, you will be able to:
+
+1. **Apply all Optimization Tier techniques** (profiling, memoization, quantization, compression, acceleration, benchmarking)
+2. **Implement either Closed Division** (optimize given CNN; pure optimization challenge) or **Open Division** (design novel architecture; innovation challenge)
+3. **Generate validated submission** with standardized metrics, honor code attestation, and GitHub repo
+4. **Demonstrate complete ML systems skills** from implementation through optimization to deployment
+5. **Compete fairly** using normalized metrics (speedup, compression ratio) that work across hardware
+
+## Why This Matters
+
+### Production Context
+
+This competition simulates real ML systems engineering:
+
+- **MLPerf** is the industry standard for ML benchmarking; this follows the same principles
+- **Production optimization** requires choosing what to optimize and measuring impact
+- **Hardware diversity** in production demands normalized comparison metrics
+- **Documentation** of optimization choices matters for team collaboration
+
+### Competition Philosophy
+
+This capstone teaches:
+- **Optimization discipline**: Profile first, optimize bottlenecks, measure impact
+- **Trade-off analysis**: Speed vs accuracy vs memory - what matters for your use case?
+- **Fair comparison**: Normalized metrics ensure your M1 MacBook competes fairly with AWS GPU
+- **Real constraints**: Must maintain >70% accuracy; actual production requirement
+
+## Competition Structure
+
+### Two Tracks
+
+**Closed Division - Optimization Challenge**
+- **Task**: Optimize provided CNN architecture
+- **Rules**: Cannot change model architecture, training, or dataset
+- **Focus**: Pure systems optimization (caching, quantization, pruning, acceleration)
+- **Goal**: Maximum speedup with minimal accuracy loss
+
+**Open Division - Innovation Challenge**  
+- **Task**: Design your own architecture
+- **Rules**: Can change anything (architecture, training, data augmentation)
+- **Focus**: Novel approaches, architectural innovations, creative solutions
+- **Goal**: Best efficiency score balancing speed, size, and accuracy
+
+### Metrics (Both Divisions)
+
+**Normalized for Fair Hardware Comparison:**
+- **Speedup**: your_inference_time / baseline_inference_time (on YOUR hardware)
+- **Compression Ratio**: baseline_params / your_params
+- **Accuracy Delta**: your_accuracy - baseline_accuracy (must be ≥ -5%)
+- **Efficiency Score**: (speedup × compression) / (1 + |accuracy_loss|)
+
+## Implementation Guide
+
+### Step 1: Validate Your Installation
+
+```bash
+tito setup --validate
+# Ensures all modules work before starting
+```
+
+### Step 2: Generate Baseline
+
+```python
+from tinytorch.competition import generate_baseline
+
+# This runs the unoptimized CNN and records your baseline
+baseline = generate_baseline()
+# Saves: baseline_submission.json with your hardware specs
+```
+
+### Step 3: Choose Your Track
+
+**Option A: Closed Division (Recommended for first-time)**
+```python
+from tinytorch.competition import optimize_closed_division
+
+# Optimize the provided CNN
+optimized_model = optimize_closed_division(
+    baseline_model,
+    techniques=['kvcaching', 'quantization', 'pruning']
+)
+```
+
+**Option B: Open Division (For advanced students)**
+```python
+from tinytorch.competition import design_open_division
+
+# Design your own architecture
+my_model = MyCustomCNN(...)
+# Train it
+trained_model = train(my_model, train_loader)
+```
+
+### Step 4: Generate Submission
+
+```python
+from tinytorch.competition import generate_submission
+
+submission = generate_submission(
+    model=optimized_model,
+    division='closed',  # or 'open'
+    github_repo='https://github.com/yourname/tinytorch-submission',
+    techniques_used=['INT8 quantization', '90% magnitude pruning', 'KV caching'],
+    athlete_name='Your Name'
+)
+
+# This creates: submission.json with all required fields
+```
+
+### Step 5: Validate and Submit
+
+```bash
+# Local validation
+tito submit --file submission.json --validate-only
+
+# Official submission (when ready)
+tito submit --file submission.json
+```
+
+## Submission Requirements
+
+### Required Fields
+
+- **division**: 'closed' or 'open'
+- **athlete_name**: Your name
+- **github_repo**: Link to your code (public or private with access)
+- **baseline_metrics**: From Step 2
+- **optimized_metrics**: From Step 4
+- **normalized_scores**: Speedup, compression, accuracy delta
+- **techniques_used**: List of optimizations applied
+- **honor_code**: "I certify that this submission follows the rules" 
+- **hardware**: CPU/GPU specs, RAM (for reference, not ranking)
+- **tinytorch_version**: Automatically captured
+- **timestamp**: Automatically captured
+
+### Validation Checks
+
+The submission system performs sanity checks:
+- ✅ Speedup between 0.5× and 100× (realistic range)
+- ✅ Compression between 1× and 100× (realistic range)
+- ✅ Accuracy drop < 10% (must maintain reasonable performance)
+- ✅ GitHub repo exists and contains code
+- ✅ Techniques used are documented
+- ✅ No training modifications in Closed Division
+
+### Honor Code
+
+This is an honor-based system with light validation:
+- We trust you followed the rules
+- Automated checks catch accidental errors
+- If something seems wrong, we may ask for clarification
+- GitHub repo allows others to learn from your work
+
+## Example Optimizations (Closed Division)
+
+**Beginner**: 
+- Apply INT8 quantization: ~4× compression, ~2× speedup
+- Result: Speedup=2×, Compression=4×, Efficiency≈8
+
+**Intermediate**:
+- Quantization + 50% pruning: ~8× compression, ~3× speedup
+- Result: Speedup=3×, Compression=8×, Efficiency≈24
+
+**Advanced**:
+- Quantization + 90% pruning + operator fusion: ~40× compression, ~5× speedup
+- Result: Speedup=5×, Compression=40×, Efficiency≈200
+
+## Testing
+
+```bash
+# Run everything end-to-end
+cd modules/20_competition
+python competition_dev.py
+
+# Export and test
+tito export 20_competition
+tito test 20_competition
+
+# Generate baseline
+python -c "from tinytorch.competition import generate_baseline; generate_baseline()"
+
+# Validate submission
+tito submit --file submission.json --validate-only
+```
+
+## Where This Code Lives
+
+```
+tinytorch/
+├── competition/
+│   ├── baseline.py             # Baseline model
+│   ├── submission.py           # Submission generation
+│   └── validate.py             # Validation logic
+└── __init__.py
+
+Generated files:
+- baseline_submission.json      # Your baseline metrics
+- submission.json               # Your final submission
+```
+
+## Systems Thinking Questions
+
+1. **Optimization Priority**: You have limited time. Profile shows attention=40%, FFN=35%, embedding=15%, other=10%. Where do you start and why?
+
+2. **Accuracy Trade-off**: Closed Division allows up to 5% accuracy loss. How do you decide what's acceptable? What if you could get 10× speedup for 6% loss?
+
+3. **Hardware Fairness**: Student A has M1 Max, Student B has i5 laptop. Normalized metrics show both achieved 3× speedup. Who optimized better?
+
+4. **Open Division Strategy**: You could design a tiny 100K-param model (fast but potentially less accurate) or optimize a 1M-param model. What's your strategy?
+
+5. **Verification Challenge**: How would you verify submissions without running everyone's code? What checks are sufficient?
+
+## Real-World Connections
+
+### MLPerf
+
+This competition mirrors MLPerf principles:
+- Closed Division = MLPerf Closed (fixed model/training)
+- Open Division = MLPerf Open (anything goes)
+- Normalized metrics for fair hardware comparison
+- Honor-based with validation checks
+
+### Industry Applications
+
+**Model Deployment Engineer** (your future job):
+- Given: Slow model from research team
+- Goal: Deploy at production scale
+- Constraints: Latency SLA, accuracy requirements, hardware budget
+- Skills: Profiling, optimization, trade-off analysis (this capstone!)
+
+**ML Competition Platforms**: Kaggle, DrivenData use similar structures
+- Leaderboards drive innovation
+- Standardized metrics ensure fairness
+- Open sharing advances the field
+
+## What's Next?
+
+**You've completed TinyTorch!** You've built:
+- **Foundation Tier**: All ML building blocks from scratch
+- **Architecture Tier**: Vision and language systems
+- **Optimization Tier**: Production optimization techniques
+- **Capstone**: Real-world ML systems engineering
+
+**Where to go from here:**
+- Deploy your optimized model to production
+- Contribute to open-source ML frameworks
+- Join ML systems research or engineering teams
+- Build the next generation of ML infrastructure
+
+---
+
+**Ready for your capstone challenge?** Open `modules/20_competition/competition_dev.py` and start optimizing!
+
+**Compete. Optimize. Dominate.** 🏆

modules/LEARNING_PATH.md

@@ -0,0 +1,512 @@
+# TinyTorch Learning Journey
+**From Zero to Transformer: A 20-Module Adventure**
+
+```
+┌─────────────────────────────────────────────────────────────────────┐
+│                    🎯 YOUR LEARNING DESTINATION                      │
+│                                                                       │
+│  Start: "What's a tensor?"                                           │
+│    ↓                                                                  │
+│  Finish: "I built a transformer from scratch using only NumPy!"      │
+│                                                                       │
+│  🏆 North Star Achievement: Train CNNs on CIFAR-10 to 75%+ accuracy │
+└─────────────────────────────────────────────────────────────────────┘
+```
+
+## Overview: 4 Phases, 20 Modules, 6 Milestones
+
+**Total Time**: 60-80 hours (3-4 weeks at 20 hrs/week)
+**Prerequisites**: Python, NumPy basics, basic linear algebra
+**Tools**: Just Python + NumPy + Jupyter notebooks
+
+---
+
+## Phase 1: FOUNDATION (Modules 01-04)
+**Goal**: Build the fundamental data structures and operations
+**Time**: 10-12 hours | **Difficulty**: ⭐⭐ Beginner-friendly
+
+```
+┌──────────┐      ┌──────────────┐      ┌─────────┐      ┌─────────┐
+│    01    │─────▶│      02      │─────▶│   03    │─────▶│   04    │
+│  Tensor  │      │ Activations  │      │ Layers  │      │ Losses  │
+│          │      │              │      │         │      │         │
+│ • Shape  │      │ • ReLU       │      │ • Linear│      │ • MSE   │
+│ • Data   │      │ • Sigmoid    │      │ • Module│      │ • Cross │
+│ • Ops    │      │ • Softmax    │      │ • Params│      │   Entropy│
+└──────────┘      └──────────────┘      └─────────┘      └─────────┘
+  2-3 hrs           1.5-2 hrs            2-3 hrs          2-3 hrs
+   ⭐⭐              ⭐⭐                  ⭐⭐⭐            ⭐⭐⭐
+```
+
+### Module Details
+
+**Module 01: Tensor** (2-3 hours, ⭐⭐)
+- Build the foundation: n-dimensional arrays with operations
+- Implement: shape, reshape, indexing, broadcasting
+- Operations: add, multiply, matmul, transpose
+- Why it matters: Everything in ML is tensor operations
+
+**Module 02: Activations** (1.5-2 hours, ⭐⭐)
+- Add non-linearity: ReLU, Sigmoid, Softmax
+- Understand: Why neural networks need activations
+- Implement: Forward passes for each activation
+- Why it matters: Without activations, networks are just linear algebra
+
+**Module 03: Layers** (2-3 hours, ⭐⭐⭐)
+- Build neural network components: Linear layers
+- Implement: nn.Module system, Parameter class
+- Create: Weight initialization, layer composition
+- Why it matters: Foundation for all network architectures
+
+**Module 04: Losses** (2-3 hours, ⭐⭐⭐)
+- Measure performance: MSE and CrossEntropy
+- Understand: How to quantify model errors
+- Implement: Loss calculation and aggregation
+- Why it matters: Without loss, we can't train networks
+
+### Milestone Checkpoint 1: 1957 Perceptron
+**Unlock After**: Module 04
+```
+🏆 CHECKPOINT: Train Rosenblatt's Original Perceptron
+├─ Dataset: Linearly separable binary classification
+├─ Architecture: Single layer, no hidden units
+├─ Achievement: First trainable neural network in history!
+└─ Test: Can your implementation learn AND/OR logic?
+```
+
+---
+
+## Phase 2: TRAINING SYSTEMS (Modules 05-08)
+**Goal**: Make your networks learn from data
+**Time**: 14-18 hours | **Difficulty**: ⭐⭐⭐ Core ML concepts
+
+```
+┌──────────┐      ┌────────────┐      ┌──────────┐      ┌────────────┐
+│    05    │─────▶│     06     │─────▶│    07    │─────▶│     08     │
+│ Autograd │      │ Optimizers │      │ Training │      │ DataLoader │
+│          │      │            │      │          │      │            │
+│ • Graph  │      │ • SGD      │      │ • Loops  │      │ • Batching │
+│ • Forward│      │ • Momentum │      │ • Epochs │      │ • Shuffling│
+│ • Backward│     │ • Adam     │      │ • Eval   │      │ • Pipeline │
+└──────────┘      └────────────┘      └──────────┘      └────────────┘
+  3-4 hrs          3-4 hrs             4-5 hrs           3-4 hrs
+  ⭐⭐⭐⭐          ⭐⭐⭐⭐             ⭐⭐⭐⭐           ⭐⭐⭐
+     │                 │                  │                  │
+     └─────────────────┴──────────────────┴──────────────────┘
+                    ALL BUILD ON TENSOR (Module 01)
+```
+
+### Module Details
+
+**Module 05: Autograd** (3-4 hours, ⭐⭐⭐⭐) **CRITICAL MODULE**
+- Implement automatic differentiation: The magic of modern ML
+- Build: Computational graph, gradient tracking
+- Implement: backward() for all operations
+- Why it matters: This IS machine learning - without gradients, no training
+
+**Module 06: Optimizers** (3-4 hours, ⭐⭐⭐⭐)
+- Update weights intelligently: SGD, Momentum, Adam
+- Understand: Learning rates, momentum, adaptive methods
+- Implement: Parameter updates, state management
+- Why it matters: How networks actually improve over time
+
+**Module 07: Training** (4-5 hours, ⭐⭐⭐⭐) **CRITICAL MODULE**
+- Complete training loops: The full ML pipeline
+- Implement: Epochs, batches, forward/backward passes
+- Add: Metrics tracking, model evaluation
+- Why it matters: This is where everything comes together
+
+**Module 08: DataLoader** (3-4 hours, ⭐⭐⭐)
+- Efficient data handling: Batching, shuffling, pipelines
+- Implement: Batch creation, data iteration
+- Optimize: Memory efficiency, preprocessing
+- Why it matters: Real ML needs to handle millions of examples
+
+### Milestone Checkpoint 2: 1969 XOR Crisis & Solution
+**Unlock After**: Module 07
+```
+🏆 CHECKPOINT: Solve the Problem That Nearly Killed AI
+├─ Dataset: XOR (the "impossible" problem for single-layer networks)
+├─ Architecture: Multi-layer perceptron with hidden units
+├─ Achievement: Prove Minsky wrong - MLPs can learn XOR!
+└─ Test: 100% accuracy on XOR with your backpropagation
+```
+
+### Milestone Checkpoint 3: 1986 MLP Revival
+**Unlock After**: Module 08
+```
+🏆 CHECKPOINT: Recognize Handwritten Digits (MNIST)
+├─ Dataset: MNIST (60,000 handwritten digits)
+├─ Architecture: 2-3 layer MLP with ReLU activations
+├─ Achievement: 95%+ accuracy on real computer vision!
+└─ Test: Your network recognizes digits you draw yourself
+```
+
+---
+
+## Phase 3: ADVANCED ARCHITECTURES (Modules 09-13)
+**Goal**: Build modern CV and NLP architectures
+**Time**: 20-25 hours | **Difficulty**: ⭐⭐⭐⭐ Advanced concepts
+
+```
+┌──────────┐      ┌───────────────┐      ┌─────────────┐
+│    09    │─────▶│      10       │─────▶│     11      │
+│ Spatial  │      │ Tokenization  │      │ Embeddings  │
+│          │      │               │      │             │
+│ • Conv2d │      │ • BPE         │      │ • Token Emb │
+│ • Pool2d │      │ • Vocab       │      │ • Position  │
+│ • CNNs   │      │ • Encoding    │      │ • Learned   │
+└──────────┘      └───────────────┘      └─────────────┘
+  5-6 hrs          4-5 hrs                3-4 hrs
+ ⭐⭐⭐⭐⭐         ⭐⭐⭐⭐               ⭐⭐⭐⭐
+     │                  │                      │
+     │                  └──────────┬───────────┘
+     │                             ▼
+     │            ┌──────────┐      ┌──────────────┐
+     │            │    12    │─────▶│      13      │
+     │            │Attention │      │Transformers  │
+     │            │          │      │              │
+     │            │ • Q,K,V  │      │ • Encoder    │
+     │            │ • Multi  │      │ • Decoder    │
+     │            │   -Head  │      │ • Complete   │
+     │            └──────────┘      └──────────────┘
+     │              4-5 hrs           6-8 hrs
+     │             ⭐⭐⭐⭐⭐          ⭐⭐⭐⭐⭐
+     │                  │                  │
+     └──────────────────┴──────────────────┘
+              ALL USE AUTOGRAD (Module 05)
+```
+
+### Module Details
+
+**Module 09: Spatial Operations** (5-6 hours, ⭐⭐⭐⭐⭐) **CRITICAL MODULE**
+- Convolutional Neural Networks: Modern computer vision
+- Implement: Conv2d (with 6 nested loops!), MaxPool2d
+- Understand: Why CNNs revolutionized image processing
+- Why it matters: The foundation of modern computer vision
+
+**Module 10: Tokenization** (4-5 hours, ⭐⭐⭐⭐)
+- Text preprocessing: From strings to numbers
+- Implement: Byte-Pair Encoding (BPE), vocabulary building
+- Understand: How transformers see language
+- Why it matters: Can't process text without tokenization
+
+**Module 11: Embeddings** (3-4 hours, ⭐⭐⭐⭐)
+- Convert tokens to vectors: Token and positional embeddings
+- Implement: Embedding lookup, sinusoidal position encoding
+- Understand: How models represent meaning
+- Why it matters: Foundation for all language models
+
+**Module 12: Attention** (4-5 hours, ⭐⭐⭐⭐⭐) **CRITICAL MODULE**
+- The transformer revolution: Multi-head self-attention
+- Implement: Q, K, V projections, scaled dot-product attention
+- Understand: Why attention changed everything
+- Why it matters: The core of GPT, BERT, and all modern LLMs
+
+**Module 13: Transformers** (6-8 hours, ⭐⭐⭐⭐⭐) **CRITICAL MODULE**
+- Complete transformer architecture: GPT-style models
+- Implement: Encoder/decoder blocks, layer norm, residuals
+- Build: Full transformer from components
+- Why it matters: You're building GPT from scratch!
+
+### Milestone Checkpoint 4: 1998 CNN Revolution
+**Unlock After**: Module 09
+```
+🏆 CHECKPOINT: CIFAR-10 Image Classification (North Star!)
+├─ Dataset: CIFAR-10 (50,000 color images, 10 classes)
+├─ Architecture: LeNet-inspired CNN with Conv2d + MaxPool
+├─ Achievement: 75%+ accuracy on real-world images!
+├─ Test: Classify airplanes, cars, birds, cats, etc.
+└─ Impact: This is where your framework becomes REAL
+```
+
+### Milestone Checkpoint 5: 2017 Transformer Era
+**Unlock After**: Module 13
+```
+🏆 CHECKPOINT: Build a Language Model
+├─ Dataset: Text corpus (Shakespeare, WikiText, etc.)
+├─ Architecture: GPT-style decoder with multi-head attention
+├─ Achievement: Generate coherent text character-by-character
+├─ Test: Your model completes sentences meaningfully
+└─ Impact: You've built the architecture behind ChatGPT!
+```
+
+---
+
+## Phase 4: PRODUCTION SYSTEMS (Modules 14-20)
+**Goal**: Optimize and deploy ML systems at scale
+**Time**: 18-22 hours | **Difficulty**: ⭐⭐⭐⭐⭐ Systems engineering
+
+```
+┌──────────┐      ┌──────────────┐      ┌──────────────┐
+│    14    │─────▶│      15      │─────▶│      16      │
+│Profiling │      │ Quantization │      │ Compression  │
+│          │      │              │      │              │
+│ • Time   │      │ • INT8       │      │ • Pruning    │
+│ • Memory │      │ • Calibrate  │      │ • Distill    │
+│ • FLOPs  │      │ • Compress   │      │ • Sparse     │
+└──────────┘      └──────────────┘      └──────────────┘
+  3-4 hrs          5-6 hrs                4-5 hrs
+  ⭐⭐⭐⭐          ⭐⭐⭐⭐⭐              ⭐⭐⭐⭐⭐
+
+       ▼                 ▼                     ▼
+
+┌──────────┐      ┌──────────────┐      ┌──────────┐      ┌──────────┐
+│    17    │─────▶│      18      │─────▶│    19    │─────▶│    20    │
+│Memoization│    │Acceleration  │      │Benchmark │      │ Capstone │
+│          │      │              │      │          │      │          │
+│ • KV-Cache│     │ • Vectorize  │      │ • Compare│      │ • Full   │
+│ • Reuse  │      │ • Hardware   │      │ • Report │      │   System │
+│ • Speedup│      │ • Parallel   │      │ • Analyze│      │ • Deploy │
+└──────────┘      └──────────────┘      └──────────┘      └──────────┘
+  3-4 hrs          3-4 hrs               3-4 hrs          4-6 hrs
+  ⭐⭐⭐⭐          ⭐⭐⭐⭐              ⭐⭐⭐⭐          ⭐⭐⭐⭐⭐
+```
+
+### Module Details
+
+**Module 14: Profiling** (3-4 hours, ⭐⭐⭐⭐)
+- Measure everything: Time, memory, FLOPs
+- Implement: Profiling decorators, bottleneck analysis
+- Understand: Where computation actually happens
+- Why it matters: Can't optimize what you don't measure
+
+**Module 15: Quantization** (5-6 hours, ⭐⭐⭐⭐⭐)
+- Compress models: Float32 → INT8
+- Implement: Quantization, calibration, dequantization
+- Achieve: 4× smaller models, faster inference
+- Why it matters: Deploy models on edge devices
+
+**Module 16: Compression** (4-5 hours, ⭐⭐⭐⭐⭐)
+- Shrink models: Pruning and distillation
+- Implement: Weight pruning, knowledge distillation
+- Achieve: 10× smaller models with minimal accuracy loss
+- Why it matters: Mobile ML and resource-constrained deployment
+
+**Module 17: Memoization** (3-4 hours, ⭐⭐⭐⭐)
+- Cache computations: KV-cache for transformers
+- Implement: Memoization decorators, cache management
+- Optimize: 10-100× speedup for inference
+- Why it matters: How production LLMs run efficiently
+
+**Module 18: Acceleration** (3-4 hours, ⭐⭐⭐⭐)
+- Hardware optimization: Vectorization, parallelization
+- Implement: NumPy tricks, batch processing
+- Achieve: 10-100× speedups
+- Why it matters: Production systems need speed
+
+**Module 19: Benchmarking** (3-4 hours, ⭐⭐⭐⭐)
+- Compare implementations: Rigorous performance testing
+- Implement: Benchmark suite, statistical analysis
+- Report: Scientific measurements
+- Why it matters: Engineering decisions need data
+
+**Module 20: Capstone** (4-6 hours, ⭐⭐⭐⭐⭐) **FINAL PROJECT**
+- Build complete system: End-to-end ML pipeline
+- Integrate: All 19 modules into production-ready system
+- Deploy: Real application with optimization
+- Why it matters: This is your portfolio piece!
+
+### Milestone Checkpoint 6: 2024 Systems Age
+**Unlock After**: Module 20
+```
+🏆 FINAL CHECKPOINT: Production-Optimized ML System
+├─ Challenge: Take any milestone and make it production-ready
+├─ Requirements:
+│   ├─ 10× faster inference (profiling + acceleration)
+│   ├─ 4× smaller model (quantization + compression)
+│   ├─ <100ms latency (memoization + optimization)
+│   └─ Rigorous benchmarks (statistical significance)
+├─ Achievement: You're now an ML systems engineer!
+└─ Test: Deploy your system, measure everything, compare to PyTorch
+```
+
+---
+
+## Dependency Map: How Modules Connect
+
+```
+CORE FOUNDATION
+├─ Module 01 (Tensor)
+│   ├─▶ Module 02 (Activations)
+│   ├─▶ Module 03 (Layers)
+│   ├─▶ Module 04 (Losses)
+│   └─▶ Module 08 (DataLoader)
+│
+TRAINING ENGINE
+├─ Module 05 (Autograd) ← Enhances Module 01
+│   ├─▶ Module 06 (Optimizers)
+│   └─▶ Module 07 (Training)
+│
+COMPUTER VISION BRANCH
+├─ Module 09 (Spatial) ← Uses 01,02,03,05
+│   └─▶ Module 20 (Capstone)
+│
+NLP BRANCH
+├─ Module 10 (Tokenization) ← Uses 01
+│   ├─▶ Module 11 (Embeddings)
+│   └─▶ Module 12 (Attention) ← Uses 01,03,05,11
+│       └─▶ Module 13 (Transformers) ← Uses 02,11,12
+│
+OPTIMIZATION BRANCH
+├─ Module 14 (Profiling) ← Measures any module
+│   ├─▶ Module 15 (Quantization) ← Compresses any module
+│   ├─▶ Module 16 (Compression) ← Shrinks any module
+│   ├─▶ Module 17 (Memoization) ← Optimizes 12,13
+│   ├─▶ Module 18 (Acceleration) ← Speeds up any module
+│   └─▶ Module 19 (Benchmarking) ← Measures optimizations
+│       └─▶ Module 20 (Capstone)
+```
+
+---
+
+## Time Estimates by Experience Level
+
+```
+┌──────────────────┬──────────┬──────────┬──────────┬──────────┐
+│ Experience Level │ Phase 1  │ Phase 2  │ Phase 3  │ Phase 4  │
+├──────────────────┼──────────┼──────────┼──────────┼──────────┤
+│ Beginner         │ 12-15h   │ 18-22h   │ 25-30h   │ 22-26h   │
+│ (New to ML)      │          │          │          │          │
+├──────────────────┼──────────┼──────────┼──────────┼──────────┤
+│ Intermediate     │ 10-12h   │ 14-18h   │ 20-25h   │ 18-22h   │
+│ (Used PyTorch)   │          │          │          │          │
+├──────────────────┼──────────┼──────────┼──────────┼──────────┤
+│ Advanced         │  8-10h   │ 12-15h   │ 18-22h   │ 16-20h   │
+│ (Built models)   │          │          │          │          │
+└──────────────────┴──────────┴──────────┴──────────┴──────────┘
+
+Total Time: 60-80 hours (Intermediate) | 3-4 weeks at 20 hrs/week
+```
+
+---
+
+## Difficulty Ratings Explained
+
+```
+⭐⭐         │ Beginner-friendly
+            │ - Follow clear instructions
+            │ - Build intuition for concepts
+            │ - ~2 hours per module
+            │
+⭐⭐⭐       │ Core ML concepts
+            │ - Implement fundamental algorithms
+            │ - Connect multiple concepts
+            │ - ~3 hours per module
+            │
+⭐⭐⭐⭐     │ Advanced implementation
+            │ - Complex algorithms
+            │ - Systems thinking required
+            │ - ~4 hours per module
+            │
+⭐⭐⭐⭐⭐   │ Expert-level systems
+            │ - Multi-layered complexity
+            │ - Production considerations
+            │ - ~5-6 hours per module
+```
+
+---
+
+## Suggested Learning Paths
+
+### Fast Track (Core ML Only) - 40 hours
+Focus on the essentials to build and train networks:
+```
+01 → 02 → 03 → 04 → 05 → 06 → 07 → 08 → 09
+(Tensor through Spatial for CNNs)
+
+Milestones: Perceptron → XOR → MNIST → CIFAR-10
+```
+
+### NLP Focus - 55 hours
+Core + Language models:
+```
+01 → 02 → 03 → 04 → 05 → 06 → 07 → 08
+          ↓
+10 → 11 → 12 → 13
+(Add Tokenization through Transformers)
+
+Milestones: All ML history + Transformer Era
+```
+
+### Systems Engineering Path - Full 75 hours
+Everything + optimization:
+```
+Complete all 20 modules
+(Tensor → Transformers → Optimization → Capstone)
+
+Milestones: All 6 checkpoints + Production Systems
+```
+
+---
+
+## Success Metrics: What "Done" Looks Like
+
+```
+✅ Module Complete When:
+├─ All unit tests pass (test_unit_* functions)
+├─ Module integration test passes (test_module())
+├─ You can explain the algorithm to someone else
+└─ Code matches PyTorch API (but implemented from scratch)
+
+✅ Phase Complete When:
+├─ All modules in phase pass tests
+├─ Milestone checkpoint achieved
+└─ You understand connections between modules
+
+✅ Course Complete When:
+├─ All 20 modules implemented
+├─ All 6 milestones achieved
+├─ Capstone project deployed
+└─ You can confidently say: "I built a transformer from scratch!"
+```
+
+---
+
+## Common Questions
+
+**Q: Do I need to complete modules in order?**
+A: YES! Each module builds on previous ones. Module 05 (Autograd) enhances Module 01 (Tensor), Module 12 (Attention) uses Modules 01, 03, 05, and 11. The dependency chain is strict.
+
+**Q: Can I skip modules?**
+A: Modules 01-08 are REQUIRED. Modules 09-13 split into CV (09) and NLP (10-13) tracks - you can choose one. Modules 14-20 are optimization - recommended but optional for core understanding.
+
+**Q: How do I know if I'm ready for the next module?**
+A: Run `test_module()` - if all tests pass, you're ready! Each module has comprehensive integration tests.
+
+**Q: What if I get stuck?**
+A: Each module has reference solutions, detailed scaffolding, and clear error messages. Plus milestone checkpoints validate your progress.
+
+**Q: How is this different from online courses?**
+A: You BUILD everything from scratch. No black boxes. No "just import PyTorch." You implement every line of a production ML framework.
+
+---
+
+## Your Journey Starts Now
+
+```
+┌─────────────────────────────────────────────┐
+│  📍 YOU ARE HERE                             │
+│                                              │
+│  Next Step: cd modules/01_tensor/    │
+│             jupyter notebook tensor_dev.py   │
+│                                              │
+│  First Goal: Understand what a tensor is    │
+│  First Win: Implement your first matmul     │
+│  First Checkpoint: Train a perceptron       │
+│                                              │
+│  🎯 Final Destination (60-80 hours ahead):  │
+│     "I built a transformer from scratch!"   │
+└─────────────────────────────────────────────┘
+```
+
+**Remember**: Every expert was once a beginner. Every line of PyTorch was written by someone who understood these fundamentals. Now it's your turn.
+
+**Ready to start building?**
+
+```bash
+cd modules/01_tensor
+jupyter notebook tensor_dev.py
+```
+
+Let's build something amazing! 🚀

site/.nojekyll

@@ -0,0 +1,2 @@
+# This file tells GitHub Pages not to use Jekyll processing
+# Required for Jupyter Book deployment 

site/Makefile

@@ -0,0 +1,58 @@
+# TinyTorch Book Build Makefile
+# Convenient shortcuts for building HTML and PDF versions
+
+.PHONY: help html pdf pdf-simple clean install test
+
+help:
+	@echo "TinyTorch Book Build Commands"
+	@echo "=============================="
+	@echo ""
+	@echo "  make html        - Build HTML version (default website)"
+	@echo "  make pdf         - Build PDF via LaTeX (requires LaTeX installation)"
+	@echo "  make pdf-simple  - Build PDF via HTML (no LaTeX required)"
+	@echo "  make clean       - Remove all build artifacts"
+	@echo "  make install     - Install Python dependencies"
+	@echo "  make install-pdf - Install dependencies for PDF building"
+	@echo "  make test        - Test build configuration"
+	@echo ""
+	@echo "Quick start for PDF:"
+	@echo "  make install-pdf && make pdf-simple"
+	@echo ""
+
+html:
+	@echo "🌐 Building HTML version..."
+	jupyter-book build .
+
+pdf:
+	@echo "📚 Building PDF via LaTeX..."
+	@./build_pdf.sh
+
+pdf-simple:
+	@echo "📚 Building PDF via HTML..."
+	@./build_pdf_simple.sh
+
+clean:
+	@echo "🧹 Cleaning build artifacts..."
+	jupyter-book clean . --all
+	rm -rf _build/
+
+install:
+	@echo "📦 Installing base dependencies..."
+	pip install -U pip
+	pip install "jupyter-book<1.0"
+	pip install -r requirements.txt
+
+install-pdf:
+	@echo "📦 Installing PDF dependencies..."
+	pip install -U pip
+	pip install "jupyter-book<1.0" pyppeteer
+	pip install -r requirements.txt
+
+test:
+	@echo "🧪 Testing build configuration..."
+	jupyter-book config sphinx .
+	@echo "✅ Configuration valid"
+
+# Default target
+.DEFAULT_GOAL := help
+

site/README.md

@@ -0,0 +1,162 @@
+# TinyTorch Course Book
+
+This directory contains the TinyTorch course content built with [Jupyter Book](https://jupyterbook.org/).
+
+## 🌐 View Online
+
+**Live website:** https://mlsysbook.github.io/TinyTorch/
+
+## 📚 Build Options
+
+### Option 1: HTML (Default Website)
+
+```bash
+cd book
+jupyter-book build .
+```
+
+Output: `_build/html/index.html`
+
+### Option 2: PDF (Simple Method - Recommended)
+
+No LaTeX installation required!
+
+```bash
+cd book
+make install-pdf    # Install dependencies
+make pdf-simple     # Build PDF
+```
+
+Output: `_build/tinytorch-course.pdf`
+
+### Option 3: PDF (LaTeX Method - Professional Quality)
+
+Requires LaTeX installation (texlive, mactex, etc.)
+
+```bash
+cd book
+make pdf
+```
+
+Output: `_build/latex/tinytorch-course.pdf`
+
+## 🚀 Quick Commands
+
+Using the Makefile (recommended):
+
+```bash
+make html        # Build website
+make pdf-simple  # Build PDF (no LaTeX needed)
+make pdf         # Build PDF via LaTeX
+make clean       # Remove build artifacts
+make install     # Install dependencies
+make install-pdf # Install PDF dependencies
+```
+
+Using scripts directly:
+
+```bash
+./build_pdf_simple.sh  # PDF without LaTeX
+./build_pdf.sh         # PDF with LaTeX
+```
+
+## 📖 Detailed Documentation
+
+See **[PDF_BUILD_GUIDE.md](PDF_BUILD_GUIDE.md)** for:
+- Complete setup instructions
+- Troubleshooting guide
+- Configuration options
+- Build performance details
+
+## 🏗️ Structure
+
+```
+book/
+├── _config.yml              # Jupyter Book configuration
+├── _toc.yml                 # Table of contents
+├── chapters/                # Course chapters (01-20)
+├── _static/                 # Images, CSS, JavaScript
+├── intro.md                 # Book introduction
+├── quickstart-guide.md      # Quick start for students
+├── tito-essentials.md       # CLI reference
+└── ...                      # Additional course pages
+```
+
+## 🎯 Content Overview
+
+### 📚 20 Technical Chapters
+
+**Foundation Tier (01-07):**
+- Tensor operations, activations, layers, losses, autograd, optimizers, training
+
+**Architecture Tier (08-13):**
+- DataLoader, convolutional networks (CNNs), tokenization, embeddings, attention, transformers
+
+**Optimization Tier (14-19):**
+- Profiling, memoization (KV caching), quantization, compression, acceleration, benchmarking
+
+**Capstone (20):**
+- MLPerf® Edu Competition project
+
+## 🔧 Development
+
+### Local Development Server
+
+```bash
+jupyter-book build . --path-output ./_build-dev
+python -m http.server 8000 -d _build-dev/html
+```
+
+Visit: http://localhost:8000
+
+### Auto-rebuild on Changes
+
+```bash
+pip install sphinx-autobuild
+sphinx-autobuild book book/_build/html
+```
+
+## 🤝 Contributing
+
+To contribute to the course content:
+
+1. Edit chapter files in `chapters/`
+2. Test your changes: `jupyter-book build .`
+3. Preview in browser: Open `_build/html/index.html`
+4. Submit PR with your improvements
+
+## 📦 Dependencies
+
+Core dependencies are in `requirements.txt`:
+- jupyter-book
+- numpy, matplotlib
+- sphinxcontrib-mermaid
+- rich (for CLI output)
+
+PDF dependencies (optional):
+- `pyppeteer` (HTML-to-PDF, no LaTeX)
+- LaTeX distribution (for pdflatex method)
+
+## 🎓 For Instructors
+
+**Using this book for teaching:**
+
+1. **Host locally:** Build and serve on your institution's server
+2. **Customize content:** Modify chapters for your course
+3. **Generate PDFs:** Distribute offline reading material
+4. **Track progress:** Use the checkpoint system for assessment
+
+See [instructor guide](instructor-guide.md) for more details.
+
+## 📝 License
+
+MIT License - see LICENSE file in repository root
+
+## 🐛 Issues
+
+Report issues: https://github.com/mlsysbook/TinyTorch/issues
+
+---
+
+**Build ML systems from scratch. Understand how things work.**
+

site/_config.yml

@@ -0,0 +1,90 @@
+# TinyTorch: Build ML Systems from Scratch
+# Interactive Jupyter Book Configuration
+
+title: "TinyTorch"
+author: "Prof. Vijay Janapa Reddi (Harvard University)"
+copyright: "2025"
+logo: _static/logos/logo-tinytorch-white.png
+
+# Book description and metadata
+description: >-
+  An interactive course for building machine learning systems from the ground up.
+  Learn by implementing your own PyTorch-style framework with hands-on coding,
+  real datasets, and production-ready practices.
+
+# Execution settings for interactive notebooks  
+execute:
+  execute_notebooks: "cache"
+  allow_errors: true
+  timeout: 300
+
+# Exclude patterns - don't scan these directories/files
+exclude_patterns:
+  - _build
+  - .venv
+  - appendices
+  - "**/.venv/**"
+  - "**/__pycache__/**"
+  - "**/.DS_Store"
+
+# GitHub repository configuration for GitHub Pages
+repository:
+  url: https://github.com/mlsysbook/TinyTorch
+  path_to_book: book
+  branch: main
+
+# HTML output configuration
+html:
+  use_issues_button: true
+  use_repository_button: true
+  use_edit_page_button: true
+  use_download_button: true
+  use_fullscreen_button: true
+  
+  # Custom styling
+  extra_css:
+    - _static/custom.css
+
+  # Custom JavaScript
+  extra_js:
+    - _static/wip-banner.js
+  
+  # Favicon configuration
+  favicon: "_static/favicon.svg"
+  
+  # Binder integration for executable notebooks
+  launch_buttons:
+    binderhub_url: "https://mybinder.org"
+    colab_url: "https://colab.research.google.com"
+
+# LaTeX/PDF output
+latex:
+  latex_documents:
+    targetname: tinytorch-course.tex
+
+# Bibliography support
+bibtex_bibfiles:
+  - references.bib
+
+# Sphinx extensions for enhanced functionality  
+sphinx:
+  extra_extensions:
+    - sphinxcontrib.mermaid
+  config:
+    mermaid_version: "10.6.1"
+
+# Parse configuration for MyST Markdown
+parse:
+  myst_enable_extensions:
+    - "colon_fence"
+    - "deflist" 
+    - "html_admonition"
+    - "html_image"
+    - "linkify"
+    - "replacements"
+    - "smartquotes"
+    - "substitution"
+    - "tasklist"
+
+# Advanced options
+only_build_toc_files: true

site/_static/custom.css

@@ -0,0 +1,249 @@
+/* Mermaid diagram styling - remove grey backgrounds ONLY from Mermaid containers */
+pre.mermaid {
+    background: transparent !important;
+    border: none !important;
+    margin: 1.5rem auto !important;
+    text-align: center !important;
+    padding: 0 !important;
+}
+
+/* Ensure Mermaid diagrams are properly centered and clean */
+pre.mermaid.align-center {
+    display: block;
+    margin-left: auto;
+    margin-right: auto;
+    background: transparent !important;
+}
+
+/* Fix oversized navigation links at bottom of pages */
+.prev-next-area .prev-next-info .prev-next-label {
+    font-size: 0.9rem !important;
+    font-weight: normal !important;
+}
+
+.prev-next-area .prev-next-info a {
+    font-size: 1rem !important;
+    font-weight: 500 !important;
+    line-height: 1.4 !important;
+}
+
+/* Ensure consistent navigation styling */
+.prev-next-area {
+    border-top: 1px solid #dee2e6;
+    padding-top: 1.5rem;
+    margin-top: 3rem;
+}
+
+.prev-next-area .prev-next-info {
+    max-width: none !important;
+}
+
+/* Work-in-Progress Banner Styles - Construction Theme */
+.wip-banner {
+    background: linear-gradient(135deg, #ffc107 0%, #ffb300 25%, #ff9800 50%, #ffc107 100%);
+    border-bottom: 3px solid #ff6f00;
+    color: #000000;
+    padding: 0.75rem 1rem;
+    text-align: center;
+    position: relative;
+    box-shadow: 0 4px 12px rgba(255, 152, 0, 0.25);
+    z-index: 1000;
+    font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
+    animation: attention-pulse 4s ease-in-out infinite;
+}
+
+/* Add spacing after the banner */
+.wip-banner + * {
+    margin-top: 2.5rem !important;
+}
+
+.wip-banner.collapsed {
+    padding: 0.5rem 1rem;
+}
+
+.wip-banner-content {
+    max-width: 1200px;
+    margin: 0 auto;
+    position: relative;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    flex-direction: column;  /* Stack title and description vertically */
+    flex-wrap: nowrap;
+    gap: 0.25rem;
+    padding-right: 3.5rem;  /* Space for buttons */
+}
+
+.wip-banner-title {
+    font-size: 1rem;
+    font-weight: 700;
+    margin: 0;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    gap: 0.3rem;  /* Reduced gap to prevent wrapping */
+    color: #000000;
+    text-transform: uppercase;
+    letter-spacing: 0.5px;
+    white-space: nowrap;  /* Prevent text wrapping */
+    flex-shrink: 0;  /* Don't shrink the title */
+}
+
+.wip-banner-title .icon {
+    font-size: 1.2rem;
+    animation: construction-blink 2s infinite ease-in-out;
+}
+
+.wip-banner-description {
+    font-size: 0.85rem;
+    margin: 0;
+    line-height: 1.4;
+    transition: all 0.3s ease;
+    color: #212121;
+    font-weight: 500;
+    max-width: 600px;
+    flex-shrink: 1;  /* Allow description to shrink if needed */
+    text-align: center;
+}
+
+
+.wip-banner.collapsed .wip-banner-description {
+    display: none;
+}
+
+.wip-banner-toggle {
+    position: absolute;
+    right: 2.5rem;
+    top: 50%;
+    transform: translateY(-50%);
+    background: rgba(0, 0, 0, 0.1);
+    border: 2px solid rgba(0, 0, 0, 0.2);
+    color: #000000;
+    font-size: 0.875rem;
+    cursor: pointer;
+    padding: 0.25rem 0.375rem;
+    border-radius: 4px;
+    transition: all 0.2s ease;
+    opacity: 0.8;
+    width: 1.75rem;
+    height: 1.75rem;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+}
+
+.wip-banner-toggle:hover {
+    background: rgba(0, 0, 0, 0.2);
+    border-color: rgba(0, 0, 0, 0.4);
+    opacity: 1;
+    transform: translateY(-50%) scale(1.05);
+}
+
+.wip-banner-close {
+    position: absolute;
+    right: 0.5rem;
+    top: 50%;
+    transform: translateY(-50%);
+    background: rgba(0, 0, 0, 0.1);
+    border: 2px solid rgba(0, 0, 0, 0.2);
+    color: #000000;
+    font-size: 1rem;
+    cursor: pointer;
+    padding: 0.25rem;
+    border-radius: 4px;
+    transition: all 0.2s ease;
+    opacity: 0.8;
+    width: 1.75rem;
+    height: 1.75rem;
+    display: flex;
+    align-items: center;
+    justify-content: center;
+    font-weight: 600;
+}
+
+.wip-banner-close:hover {
+    background: rgba(0, 0, 0, 0.2);
+    border-color: rgba(0, 0, 0, 0.4);
+    opacity: 1;
+    transform: translateY(-50%) scale(1.1);
+}
+
+.wip-banner.hidden {
+    display: none;
+}
+
+@keyframes attention-pulse {
+    0%, 100% {
+        box-shadow: 0 4px 12px rgba(255, 152, 0, 0.25);
+    }
+    50% {
+        box-shadow: 0 6px 20px rgba(255, 152, 0, 0.4);
+    }
+}
+
+@keyframes construction-blink {
+    0%, 100% { opacity: 1; transform: scale(1) rotate(0deg); }
+    25% { transform: scale(1.1) rotate(-5deg); }
+    50% { opacity: 0.8; transform: scale(0.95) rotate(0deg); }
+    75% { transform: scale(1.1) rotate(5deg); }
+}
+
+/* Adjust banner when sidebar is expanded or on smaller screens */
+@media (max-width: 1200px) {
+    .wip-banner-title {
+        font-size: 0.9rem;
+    }
+
+    .wip-banner-title .icon {
+        font-size: 1rem;
+    }
+
+    .wip-banner-title span:nth-child(2),  /* Hide second icon */
+    .wip-banner-title span:nth-child(4) {  /* Hide fourth icon */
+        display: none;
+    }
+}
+
+/* Mobile responsiveness for banner */
+@media (max-width: 768px) {
+    .wip-banner {
+        padding: 0.625rem 0.75rem;
+    }
+
+    .wip-banner-content {
+        flex-direction: column;  /* Stack vertically on mobile */
+        gap: 0.25rem;
+        padding-right: 3rem;
+    }
+
+    .wip-banner-title {
+        font-size: 0.8rem;
+        flex-wrap: nowrap;
+    }
+
+    .wip-banner-title span:nth-child(2),  /* Hide warning icon on mobile */
+    .wip-banner-title span:nth-child(4),  /* Hide hammer icon on mobile */
+    .wip-banner-title span:nth-child(5) {  /* Hide last construction icon */
+        display: none;
+    }
+
+    .wip-banner-description {
+        font-size: 0.7rem;
+        margin: 0;
+        line-height: 1.2;
+    }
+
+    .wip-banner-toggle {
+        right: 2rem;
+        width: 1.5rem;
+        height: 1.5rem;
+        font-size: 0.75rem;
+    }
+
+    .wip-banner-close {
+        right: 0.375rem;
+        width: 1.5rem;
+        height: 1.5rem;
+        font-size: 0.75rem;
+    }
+}

site/_static/favicon.ico

@@ -0,0 +1 @@
+🔥

site/_static/favicon.svg

@@ -0,0 +1,3 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 100 100">
+  <text x="50" y="75" font-size="60" text-anchor="middle" font-family="Arial, sans-serif">🔥</text>
+</svg>

site/_static/wip-banner.js

@@ -0,0 +1,59 @@
+/**
+ * Work-in-Progress Banner JavaScript
+ * Handles banner toggle, collapse, and dismiss functionality
+ */
+
+document.addEventListener('DOMContentLoaded', function() {
+    const banner = document.getElementById('wip-banner');
+    const toggleBtn = document.getElementById('wip-banner-toggle');
+    const closeBtn = document.getElementById('wip-banner-close');
+
+    if (!banner) return;
+
+    // Check if banner was previously dismissed
+    const dismissed = localStorage.getItem('wip-banner-dismissed');
+    if (dismissed === 'true') {
+        banner.style.display = 'none';
+        return;
+    }
+
+    // Check if banner was previously collapsed
+    const collapsed = localStorage.getItem('wip-banner-collapsed');
+    if (collapsed === 'true') {
+        banner.classList.add('collapsed');
+        if (toggleBtn) {
+            toggleBtn.innerHTML = '<i class="fas fa-chevron-down"></i>';
+            toggleBtn.title = 'Expand banner';
+        }
+    }
+
+    // Toggle collapse/expand
+    if (toggleBtn) {
+        toggleBtn.addEventListener('click', function() {
+            const isCollapsed = banner.classList.contains('collapsed');
+
+            if (isCollapsed) {
+                banner.classList.remove('collapsed');
+                toggleBtn.innerHTML = '<i class="fas fa-chevron-up"></i>';
+                toggleBtn.title = 'Collapse banner';
+                localStorage.setItem('wip-banner-collapsed', 'false');
+            } else {
+                banner.classList.add('collapsed');
+                toggleBtn.innerHTML = '<i class="fas fa-chevron-down"></i>';
+                toggleBtn.title = 'Expand banner';
+                localStorage.setItem('wip-banner-collapsed', 'true');
+            }
+        });
+    }
+
+    // Dismiss banner completely
+    if (closeBtn) {
+        closeBtn.addEventListener('click', function() {
+            banner.style.display = 'none';
+            localStorage.setItem('wip-banner-dismissed', 'true');
+        });
+    }
+
+    // Add smooth transitions
+    banner.style.transition = 'all 0.3s ease';
+});

site/_toc.yml

@@ -0,0 +1,92 @@
+# TinyTorch: Build ML Systems from Scratch
+# Table of Contents Structure
+
+format: jb-book
+root: intro
+title: "TinyTorch Course"
+
+parts:
+- caption: 🚀 Getting Started
+  chapters:
+  - file: quickstart-guide
+    title: "Quick Start Guide"
+  - file: usage-paths/classroom-use
+    title: "For Instructors"
+
+- caption: 🛠️ Using TinyTorch
+  chapters:
+  - file: tito-essentials
+    title: "Essential Commands"
+  - file: learning-progress
+    title: "Track Your Progress"
+
+- caption: 🧭 Course Orientation
+  chapters:
+  - file: chapters/00-introduction
+    title: "Introduction"
+
+- caption: 🏗️ Foundation Tier (01-07)
+  chapters:
+  - file: ../modules/01_tensor/ABOUT
+    title: "01. Tensor"
+  - file: ../modules/02_activations/ABOUT
+    title: "02. Activations"
+  - file: ../modules/03_layers/ABOUT
+    title: "03. Layers"
+  - file: ../modules/04_losses/ABOUT
+    title: "04. Losses"
+  - file: ../modules/05_autograd/ABOUT
+    title: "05. Autograd"
+  - file: ../modules/06_optimizers/ABOUT
+    title: "06. Optimizers"
+  - file: ../modules/07_training/ABOUT
+    title: "07. Training"
+
+- caption: 🏛️ Architecture Tier (08-13)
+  chapters:
+  - file: ../modules/08_dataloader/ABOUT
+    title: "08. DataLoader"
+  - file: ../modules/09_spatial/ABOUT
+    title: "09. Convolutions"
+  - file: ../modules/10_tokenization/ABOUT
+    title: "10. Tokenization"
+  - file: ../modules/11_embeddings/ABOUT
+    title: "11. Embeddings"
+  - file: ../modules/12_attention/ABOUT
+    title: "12. Attention"
+  - file: ../modules/13_transformers/ABOUT
+    title: "13. Transformers"
+
+- caption: ⚡ Optimization Tier (14-19)
+  chapters:
+  - file: ../modules/14_profiling/ABOUT
+    title: "14. Profiling"
+  - file: ../modules/15_quantization/ABOUT
+    title: "15. Quantization"
+  - file: ../modules/16_compression/ABOUT
+    title: "16. Compression"
+  - file: ../modules/17_memoization/ABOUT
+    title: "17. Memoization"
+  - file: ../modules/18_acceleration/ABOUT
+    title: "18. Acceleration"
+  - file: ../modules/19_benchmarking/ABOUT
+    title: "19. Benchmarking"
+
+- caption: 🏅 Capstone Project
+  chapters:
+  - file: ../modules/20_capstone/ABOUT
+    title: "20. MLPerf® Edu Competition"
+
+- caption: 🌍 Community
+  chapters:
+  - file: community
+    title: "Ecosystem"
+
+- caption: 🛠️ Resources & Tools
+  chapters:
+  - file: checkpoint-system
+    title: "Progress Tracking"
+  - file: testing-framework
+    title: "Testing Guide"
+  - file: resources
+    title: "Additional Resources"

site/build_pdf.sh

@@ -0,0 +1,69 @@
+#!/bin/bash
+# Build PDF version of TinyTorch book
+# This script builds the LaTeX/PDF version using jupyter-book
+
+set -e  # Exit on error
+
+echo "🔥 Building TinyTorch PDF..."
+echo ""
+
+# Check if we're in the book directory
+if [ ! -f "_config.yml" ]; then
+    echo "❌ Error: Must run from book/ directory"
+    echo "Usage: cd book && ./build_pdf.sh"
+    exit 1
+fi
+
+# Check dependencies
+echo "📋 Checking dependencies..."
+if ! command -v jupyter-book &> /dev/null; then
+    echo "❌ Error: jupyter-book not installed"
+    echo "Install with: pip install jupyter-book"
+    exit 1
+fi
+
+if ! command -v pdflatex &> /dev/null; then
+    echo "⚠️  Warning: pdflatex not found"
+    echo "PDF build requires LaTeX installation:"
+    echo "  - macOS: brew install --cask mactex-no-gui"
+    echo "  - Ubuntu: sudo apt-get install texlive-latex-extra texlive-fonts-recommended"
+    echo "  - Windows: Install MiKTeX from miktex.org"
+    echo ""
+    echo "Alternatively, use HTML-to-PDF build (doesn't require LaTeX):"
+    echo "  jupyter-book build . --builder pdfhtml"
+    exit 1
+fi
+
+echo "✅ Dependencies OK"
+echo ""
+
+# Clean previous builds
+echo "🧹 Cleaning previous builds..."
+jupyter-book clean . --all || true
+echo ""
+
+# Build PDF via LaTeX
+echo "📚 Building LaTeX/PDF (this may take a few minutes)..."
+jupyter-book build . --builder pdflatex
+
+# Check if build succeeded
+if [ -f "_build/latex/tinytorch-course.pdf" ]; then
+    PDF_SIZE=$(du -h "_build/latex/tinytorch-course.pdf" | cut -f1)
+    echo ""
+    echo "✅ PDF build complete!"
+    echo "📄 Output: book/_build/latex/tinytorch-course.pdf"
+    echo "📊 Size: ${PDF_SIZE}"
+    echo ""
+    echo "To view the PDF:"
+    echo "  open _build/latex/tinytorch-course.pdf    # macOS"
+    echo "  xdg-open _build/latex/tinytorch-course.pdf  # Linux"
+    echo "  start _build/latex/tinytorch-course.pdf     # Windows"
+else
+    echo ""
+    echo "❌ PDF build failed - check errors above"
+    echo ""
+    echo "📝 Build artifacts in: _build/latex/"
+    echo "Check _build/latex/tinytorch-course.log for detailed errors"
+    exit 1
+fi
+

site/build_pdf_simple.sh

@@ -0,0 +1,66 @@
+#!/bin/bash
+# Build PDF version of TinyTorch book (Simple HTML-to-PDF method)
+# This script builds PDF via HTML conversion - no LaTeX installation required
+
+set -e  # Exit on error
+
+echo "🔥 Building TinyTorch PDF (Simple Method - No LaTeX Required)..."
+echo ""
+
+# Check if we're in the book directory
+if [ ! -f "_config.yml" ]; then
+    echo "❌ Error: Must run from book/ directory"
+    echo "Usage: cd book && ./build_pdf_simple.sh"
+    exit 1
+fi
+
+# Check dependencies
+echo "📋 Checking dependencies..."
+if ! command -v jupyter-book &> /dev/null; then
+    echo "❌ Error: jupyter-book not installed"
+    echo "Install with: pip install jupyter-book pyppeteer"
+    exit 1
+fi
+
+# Check if pyppeteer is installed
+python3 -c "import pyppeteer" 2>/dev/null || {
+    echo "❌ Error: pyppeteer not installed"
+    echo "Install with: pip install pyppeteer"
+    echo ""
+    echo "Note: First run will download Chromium (~170MB)"
+    exit 1
+}
+
+echo "✅ Dependencies OK"
+echo ""
+
+# Clean previous builds
+echo "🧹 Cleaning previous builds..."
+jupyter-book clean . --all || true
+echo ""
+
+# Build PDF via HTML
+echo "📚 Building PDF from HTML (this may take a few minutes)..."
+echo "ℹ️  First run will download Chromium browser (~170MB)"
+jupyter-book build . --builder pdfhtml
+
+# Check if build succeeded
+if [ -f "_build/pdf/book.pdf" ]; then
+    # Copy to standard location with better name
+    cp "_build/pdf/book.pdf" "_build/tinytorch-course.pdf"
+    PDF_SIZE=$(du -h "_build/tinytorch-course.pdf" | cut -f1)
+    echo ""
+    echo "✅ PDF build complete!"
+    echo "📄 Output: book/_build/tinytorch-course.pdf"
+    echo "📊 Size: ${PDF_SIZE}"
+    echo ""
+    echo "To view the PDF:"
+    echo "  open _build/tinytorch-course.pdf           # macOS"
+    echo "  xdg-open _build/tinytorch-course.pdf       # Linux"
+    echo "  start _build/tinytorch-course.pdf          # Windows"
+else
+    echo ""
+    echo "❌ PDF build failed - check errors above"
+    exit 1
+fi
+

site/chapters/00-introduction.md

@@ -0,0 +1,440 @@
+# Course Introduction: ML Systems Engineering Through Implementation
+
+**Transform from ML user to ML systems engineer by building everything yourself.**
+
+---
+
+## The Origin Story: Why TinyTorch Exists
+
+### The Problem We're Solving
+
+There's a critical gap in ML engineering today. Plenty of people can use ML frameworks (PyTorch, TensorFlow, JAX, etc.), but very few understand the systems underneath. This creates real problems:
+
+- **Engineers deploy models** but can't debug when things go wrong
+- **Teams hit performance walls** because no one understands the bottlenecks
+- **Companies struggle to scale** - whether to tiny edge devices or massive clusters
+- **Innovation stalls** when everyone is limited to existing framework capabilities
+
+### How TinyTorch Began
+
+TinyTorch started as exercises for the [MLSysBook.ai](https://mlsysbook.ai) textbook - students needed hands-on implementation experience. But it quickly became clear this addressed a much bigger problem:
+
+**The industry desperately needs engineers who can BUILD ML systems, not just USE them.**
+
+Deploying ML systems at scale is hard. Scale means both directions:
+- **Small scale**: Running models on edge devices with 1MB of RAM
+- **Large scale**: Training models across thousands of GPUs
+- **Production scale**: Serving millions of requests with <100ms latency
+
+We need more engineers who understand memory hierarchies, computational graphs, kernel optimization, distributed communication - the actual systems that make ML work.
+
+### Our Solution: Learn By Building
+
+TinyTorch teaches ML systems the only way that really works: **by building them yourself**.
+
+When you implement your own tensor operations, write your own autograd, build your own optimizer - you gain understanding that's impossible to achieve by just calling APIs. You learn not just what these systems do, but HOW they do it and WHY they're designed that way.
+
+---
+
+## 🎯 Core Learning Concepts
+
+<div style="background: #f7fafc; border: 1px solid #e2e8f0; padding: 2rem; border-radius: 0.5rem; margin: 2rem 0;">
+
+**Concept 1: Systems Memory Analysis**
+```python
+# Learning objective: Understand memory usage patterns
+# Framework user: "torch.optim.Adam()" - black box
+# TinyTorch student: Implements Adam and discovers why it needs 3x parameter memory
+# Result: Deep understanding of optimizer trade-offs applicable to any framework
+```
+
+**Concept 2: Computational Complexity**
+```python
+# Learning objective: Analyze algorithmic scaling behavior
+# Framework user: "Attention mechanism" - abstract concept
+# TinyTorch student: Implements attention from scratch, measures O(n²) scaling
+# Result: Intuition for sequence modeling limits across PyTorch, TensorFlow, JAX
+```
+
+**Concept 3: Automatic Differentiation**
+```python
+# Learning objective: Understand gradient computation
+# Framework user: "loss.backward()" - mysterious process
+# TinyTorch student: Builds autograd engine with computational graphs
+# Result: Knowledge of how all modern ML frameworks enable learning
+```
+
+</div>
+
+---
+
+## What Makes TinyTorch Different
+
+Most ML education teaches you to **use** frameworks (PyTorch, TensorFlow, JAX, etc.). TinyTorch teaches you to **build** them.
+
+This fundamental difference creates engineers who understand systems deeply, not just APIs superficially.
+
+### The Learning Philosophy: Build → Use → Reflect
+
+**Traditional Approach:**
+```python
+import torch
+model = torch.nn.Linear(784, 10)  # Use someone else's implementation
+output = model(input)             # Trust it works, don't understand how
+```
+
+**TinyTorch Approach:**
+```python
+# 1. BUILD: You implement Linear from scratch
+class Linear:
+    def forward(self, x):
+        return x @ self.weight + self.bias  # You write this
+        
+# 2. USE: Your implementation in action
+from tinytorch.core.layers import Linear  # YOUR code
+model = Linear(784, 10)                  # YOUR implementation
+output = model(input)                    # YOU know exactly how this works
+
+# 3. REFLECT: Systems thinking
+# "Why does matrix multiplication dominate compute time?"
+# "How does this scale with larger models?"
+# "What memory optimizations are possible?"
+```
+
+---
+
+## Who This Course Serves
+
+### Perfect For:
+
+**🎓 Computer Science Students**
+- Want to understand ML systems beyond high-level APIs
+- Need to implement custom operations for research
+- Preparing for ML engineering roles that require systems knowledge
+
+**👩‍💻 Software Engineers → ML Engineers**
+- Transitioning into ML engineering roles
+- Need to debug and optimize production ML systems
+- Want to understand what happens "under the hood" of ML frameworks
+
+**🔬 ML Practitioners & Researchers**
+- Debug performance issues in production systems
+- Implement novel architectures and custom operations
+- Optimize training and inference for resource constraints
+
+**🧠 Anyone Curious About ML Systems**
+- Understand how PyTorch, TensorFlow actually work
+- Build intuition for ML systems design and optimization
+- Appreciate the engineering behind modern AI breakthroughs
+
+### Prerequisites
+
+**Required:**
+- **Python Programming**: Comfortable with classes, functions, basic NumPy
+- **Linear Algebra Basics**: Matrix multiplication, gradients (we review as needed)
+- **Learning Mindset**: Willingness to implement rather than just use
+
+**Not Required:**
+- Prior ML framework experience (we build our own!)
+- Deep learning theory (we learn through implementation)
+- Advanced math (we focus on practical systems implementation)
+
+---
+
+## What You'll Achieve: Tier-by-Tier Mastery
+
+### 🏗️ After Foundation Tier (Modules 01-07)
+Build a complete neural network framework from mathematical first principles:
+
+```python
+# YOUR implementation training real networks on real data
+model = Sequential([
+    Linear(784, 128),    # Your linear algebra implementation
+    ReLU(),              # Your activation function
+    Linear(128, 64),     # Your gradient-aware layers
+    ReLU(),              # Your nonlinearity
+    Linear(64, 10)       # Your classification head
+])
+
+# YOUR complete training system
+optimizer = Adam(model.parameters(), lr=0.001)  # Your optimization algorithm
+for batch in dataloader:  # Your data management
+    output = model(batch.x)                     # Your forward computation
+    loss = CrossEntropyLoss()(output, batch.y)  # Your loss calculation
+    loss.backward()                             # YOUR backpropagation engine
+    optimizer.step()                            # Your parameter updates
+```
+
+**🎯 Foundation Achievement**: 95%+ accuracy on MNIST using 100% your own mathematical implementations
+
+### 🏛️ After Architecture Tier (Modules 08-13)
+- **Computer Vision Mastery**: CNNs achieving 75%+ accuracy on CIFAR-10 with YOUR convolution implementations
+- **Language Understanding**: Transformers generating coherent text using YOUR attention mechanisms
+- **Universal Architecture**: Discover why the SAME mathematical principles work for vision AND language
+- **AI Breakthrough Recreation**: Implement the architectures that created the modern AI revolution
+
+### ⚡ After Optimization Tier (Modules 14-20)
+- **Production Performance**: Systems optimized for <100ms inference latency using YOUR profiling tools
+- **Memory Efficiency**: Models compressed to 25% original size with YOUR quantization implementations
+- **Hardware Acceleration**: Kernels achieving 10x speedups through YOUR vectorization techniques
+- **Competition Ready**: TinyMLPerf submissions competitive with industry implementations
+
+---
+
+## The ML Evolution Story You'll Experience
+
+TinyTorch's three-tier structure follows the actual historical progression of machine learning breakthroughs:
+
+### 🏗️ Foundation Era (1980s-1990s) → Foundation Tier
+**The Beginning**: Mathematical foundations that started it all
+- **1986 Breakthrough**: Backpropagation enables multi-layer networks
+- **Your Implementation**: Build automatic differentiation and gradient-based optimization
+- **Historical Milestone**: Train MLPs to 95%+ accuracy on MNIST using YOUR autograd engine
+
+### 🏛️ Architecture Era (1990s-2010s) → Architecture Tier
+**The Revolution**: Specialized architectures for vision and language
+- **1998 Breakthrough**: CNNs revolutionize computer vision (LeCun's LeNet)
+- **2017 Breakthrough**: Transformers unify vision and language ("Attention is All You Need")
+- **Your Implementation**: Build CNNs achieving 75%+ on CIFAR-10, then transformers for text generation
+- **Historical Milestone**: Recreate both revolutions using YOUR spatial and attention implementations
+
+### ⚡ Optimization Era (2010s-Present) → Optimization Tier
+**The Engineering**: Production systems that scale to billions of users
+- **2020s Breakthrough**: Efficient inference enables real-time LLMs (GPT, ChatGPT)
+- **Your Implementation**: Build KV-caching, quantization, and production optimizations
+- **Historical Milestone**: Deploy systems competitive in TinyMLPerf benchmarks
+
+**Why This Progression Matters**: You'll understand not just modern AI, but WHY it evolved this way. Each tier builds essential capabilities that inform the next, just like ML history itself.
+
+---
+
+## Systems Engineering Focus: Why Tiers Matter
+
+Traditional ML courses teach algorithms in isolation. TinyTorch's tier structure teaches **systems thinking** - how components interact to create production ML systems.
+
+### Traditional Linear Approach:
+```
+Module 1: Tensors → Module 2: Layers → Module 3: Training → ...
+```
+**Problem**: Students learn components but miss system interactions
+
+### TinyTorch Tier Approach:
+```
+🏗️ Foundation Tier: Build mathematical infrastructure
+🏛️ Architecture Tier: Compose intelligent architectures
+⚡ Optimization Tier: Deploy at production scale
+```
+**Advantage**: Each tier builds complete, working systems with clear progression
+
+### What Traditional Courses Teach vs. TinyTorch Tiers:
+
+**Traditional**: "Use `torch.optim.Adam` for optimization"
+**Foundation Tier**: "Why Adam needs 3× more memory than SGD and how to implement both from mathematical first principles"
+
+**Traditional**: "Transformers use attention mechanisms"
+**Architecture Tier**: "How attention creates O(N²) scaling, why this limits context windows, and how to implement efficient attention yourself"
+
+**Traditional**: "Deploy models with TensorFlow Serving"
+**Optimization Tier**: "How to profile bottlenecks, implement KV-caching for 10× speedup, and compete in production benchmarks"
+
+### Career Impact by Tier
+After each tier, you become the team member who:
+
+**🏗️ Foundation Tier Graduate**:
+- Debugs gradient flow issues: "Your ReLU is causing dead neurons"
+- Implements custom optimizers: "I'll build a variant of Adam for this use case"
+- Understands memory patterns: "Batch size 64 hits your GPU memory limit here"
+
+**🏛️ Architecture Tier Graduate**:
+- Designs novel architectures: "We can adapt transformers for this computer vision task"
+- Optimizes attention patterns: "This attention bottleneck is why your model won't scale to longer sequences"
+- Bridges vision and language: "The same mathematical principles work for both domains"
+
+**⚡ Optimization Tier Graduate**:
+- Deploys production systems: "I can get us from 500ms to 50ms inference latency"
+- Leads performance optimization: "Here's our memory bottleneck and my 3-step plan to fix it"
+- Competes at industry scale: "Our optimizations achieve TinyMLPerf benchmark performance"
+
+---
+
+## Learning Support & Community
+
+### Comprehensive Infrastructure
+- **Automated Testing**: Every component includes comprehensive test suites
+- **Progress Tracking**: 16-checkpoint capability assessment system
+- **CLI Tools**: `tito` command-line interface for development workflow
+- **Visual Progress**: Real-time tracking of learning milestones
+
+### Multiple Learning Paths
+- **Quick Exploration** (5 min): Browser-based exploration, no setup required
+- **Serious Development** (8+ weeks): Full local development environment
+- **Classroom Use**: Complete course infrastructure with automated grading
+
+### Professional Development Practices
+- **Version Control**: Git-based workflow with feature branches
+- **Testing Culture**: Test-driven development for all implementations
+- **Code Quality**: Professional coding standards and review processes
+- **Documentation**: Comprehensive guides and system architecture documentation
+
+---
+
+## 🚀 Start Your Journey
+
+<div style="background: #f8f9fa; padding: 2rem; border-radius: 0.5rem; margin: 2rem 0; text-align: center;">
+<h3 style="margin: 0 0 1rem 0; color: #495057;">Begin Building ML Systems</h3>
+<p style="margin: 0 0 1.5rem 0; color: #6c757d;">Choose your starting point based on your goals and time commitment</p>
+<a href="../quickstart-guide.html" style="display: inline-block; background: #007bff; color: white; padding: 0.75rem 1.5rem; border-radius: 0.25rem; text-decoration: none; font-weight: 500; margin-right: 1rem;">15-Minute Start →</a>
+<a href="01-setup.html" style="display: inline-block; background: #28a745; color: white; padding: 0.75rem 1.5rem; border-radius: 0.25rem; text-decoration: none; font-weight: 500;">Foundation Tier →</a>
+</div>
+
+**Next Steps**:
+- **New to TinyTorch**: Start with [Quick Start Guide](../quickstart-guide.html) for immediate hands-on experience
+- **Ready to Commit**: Begin [Module 01: Setup](01-setup.html) to configure your development environment
+- **Teaching a Course**: Review [Instructor Guide](../usage-paths/classroom-use.html) for classroom integration
+
+```{admonition} Your Three-Tier Journey Awaits
+:class: tip
+By completing all three tiers, you'll have built a complete ML framework that rivals production implementations:
+
+**🏗️ Foundation Tier Achievement**: 95%+ accuracy on MNIST with YOUR mathematical implementations
+**🏛️ Architecture Tier Achievement**: 75%+ accuracy on CIFAR-10 AND coherent text generation
+**⚡ Optimization Tier Achievement**: Production systems competitive in TinyMLPerf benchmarks
+
+All using code you wrote yourself, from mathematical first principles to production optimization.
+```
+
+---
+
+### 🏗️ FOUNDATION TIER (Modules 01-07)
+**Building Blocks of ML Systems • 6-8 weeks • All Prerequisites for Neural Networks**
+
+<div style="background: #f8f9fd; border: 1px solid #e0e7ff; padding: 2rem; border-radius: 0.5rem; margin: 2rem 0;">
+
+**What You'll Learn**: Build the mathematical and computational infrastructure that powers all neural networks. Master tensor operations, gradient computation, and optimization algorithms.
+
+**Prerequisites**: Python programming, basic linear algebra (matrix multiplication)
+
+**Career Connection**: Foundation skills required for ML Infrastructure Engineer, Research Engineer, Framework Developer roles
+
+**Time Investment**: ~20 hours total (3 hours/week for 6-8 weeks)
+
+</div>
+
+| Module | Component | Core Capability | Real-World Connection |
+|--------|-----------|-----------------|----------------------|
+| **01** | **Tensor** | Data structures and operations | NumPy, PyTorch tensors |
+| **02** | **Activations** | Nonlinear functions | ReLU, attention activations |
+| **03** | **Layers** | Linear transformations | `nn.Linear`, dense layers |
+| **04** | **Losses** | Optimization objectives | CrossEntropy, MSE loss |
+| **05** | **Autograd** | Automatic differentiation | PyTorch autograd engine |
+| **06** | **Optimizers** | Parameter updates | Adam, SGD optimizers |
+| **07** | **Training** | Complete training loops | Model.fit(), training scripts |
+
+**🎯 Tier Milestone**: Train neural networks achieving **95%+ accuracy on MNIST** using 100% your own implementations!
+
+**Skills Gained**:
+- Understand memory layout and computational graphs
+- Debug gradient flow and numerical stability issues
+- Implement any optimization algorithm from research papers
+- Build custom neural network architectures from scratch
+
+---
+
+### 🧠 INTELLIGENCE TIER (Modules 08-13)
+**Modern AI Algorithms • 4-6 weeks • Vision + Language Architectures**
+
+<div style="background: #fef7ff; border: 1px solid #f3e8ff; padding: 2rem; border-radius: 0.5rem; margin: 2rem 0;">
+
+**What You'll Learn**: Implement the architectures powering modern AI: convolutional networks for vision and transformers for language. Discover why the same mathematical principles work across domains.
+
+**Prerequisites**: Foundation Tier complete (Modules 01-07)
+
+**Career Connection**: Computer Vision Engineer, NLP Engineer, AI Research Scientist, ML Product Manager roles
+
+**Time Investment**: ~25 hours total (4-6 hours/week for 4-6 weeks)
+
+</div>
+
+| Module | Component | Core Capability | Real-World Connection |
+|--------|-----------|-----------------|----------------------|
+| **08** | **Spatial** | Convolutions and regularization | CNNs, ResNet, computer vision |
+| **09** | **DataLoader** | Batch processing | PyTorch DataLoader, tf.data |
+| **10** | **Tokenization** | Text preprocessing | BERT tokenizer, GPT tokenizer |
+| **11** | **Embeddings** | Representation learning | Word2Vec, positional encodings |
+| **12** | **Attention** | Information routing | Multi-head attention, self-attention |
+| **13** | **Transformers** | Modern architectures | GPT, BERT, Vision Transformer |
+
+**🎯 Tier Milestone**: Achieve **75%+ accuracy on CIFAR-10** with CNNs AND generate coherent text with transformers!
+
+**Skills Gained**:
+- Understand why convolution works for spatial data
+- Implement attention mechanisms from scratch
+- Build transformer architectures for any domain
+- Debug sequence modeling and attention patterns
+
+---
+
+### ⚡ OPTIMIZATION TIER (Modules 14-20)
+**Production & Performance • 4-6 weeks • Deploy and Scale ML Systems**
+
+<div style="background: #f0fdfa; border: 1px solid #a7f3d0; padding: 2rem; border-radius: 0.5rem; margin: 2rem 0;">
+
+**What You'll Learn**: Transform research models into production systems. Master profiling, optimization, and deployment techniques used by companies like OpenAI, Google, and Meta.
+
+**Prerequisites**: Architecture Tier complete (Modules 08-13)
+
+**Career Connection**: ML Systems Engineer, Performance Engineer, MLOps Engineer, Senior ML Engineer roles
+
+**Time Investment**: ~30 hours total (5-7 hours/week for 4-6 weeks)
+
+</div>
+
+| Module | Component | Core Capability | Real-World Connection |
+|--------|-----------|-----------------|----------------------|
+| **14** | **Profiling** | Performance analysis | PyTorch Profiler, TensorBoard |
+| **15** | **Quantization** | Memory efficiency | INT8 inference, model compression |
+| **16** | **Compression** | Model optimization | Pruning, distillation, ONNX |
+| **17** | **Memoization** | Memory management | KV-cache for generation |
+| **18** | **Acceleration** | Speed improvements | CUDA kernels, vectorization |
+| **19** | **Benchmarking** | Measurement systems | MLPerf, production monitoring |
+| **20** | **Capstone** | Full system integration | End-to-end ML pipeline |
+
+**🎯 Tier Milestone**: Build **production-ready systems** competitive in TinyMLPerf benchmarks!
+
+**Skills Gained**:
+- Profile memory usage and identify bottlenecks
+- Implement efficient inference optimizations
+- Deploy models with <100ms latency requirements
+- Design scalable ML system architectures
+
+---
+
+## 🎯 Learning Path Recommendations
+
+### Choose Your Learning Style
+
+<div style="display: grid; grid-template-columns: repeat(3, 1fr); gap: 1.5rem; margin: 2rem 0;">
+
+<div style="background: #fff7ed; border: 1px solid #fdba74; padding: 1.5rem; border-radius: 0.5rem;">
+<h4 style="margin: 0 0 1rem 0; color: #c2410c;">🚀 Complete Builder</h4>
+<p style="margin: 0 0 1rem 0; font-size: 0.9rem;">Implement every component from scratch</p>
+<p style="margin: 0; font-size: 0.85rem; color: #6b7280;"><strong>Time:</strong> 14-18 weeks<br><strong>Ideal for:</strong> CS students, aspiring ML engineers</p>
+</div>
+
+<div style="background: #f0f9ff; border: 1px solid #7dd3fc; padding: 1.5rem; border-radius: 0.5rem;">
+<h4 style="margin: 0 0 1rem 0; color: #0284c7;">⚡ Focused Explorer</h4>
+<p style="margin: 0 0 1rem 0; font-size: 0.9rem;">Pick one tier based on your goals</p>
+<p style="margin: 0; font-size: 0.85rem; color: #6b7280;"><strong>Time:</strong> 4-8 weeks<br><strong>Ideal for:</strong> Working professionals, specific skill gaps</p>
+</div>
+
+<div style="background: #f0fdf4; border: 1px solid #86efac; padding: 1.5rem; border-radius: 0.5rem;">
+<h4 style="margin: 0 0 1rem 0; color: #166534;">📚 Guided Learner</h4>
+<p style="margin: 0 0 1rem 0; font-size: 0.9rem;">Study implementations with hands-on exercises</p>
+<p style="margin: 0; font-size: 0.85rem; color: #6b7280;"><strong>Time:</strong> 8-12 weeks<br><strong>Ideal for:</strong> Self-directed learners, bootcamp graduates</p>
+</div>
+
+</div>
+
+---
+
+Welcome to ML systems engineering!

site/chapters/docs/README.md

@@ -0,0 +1,73 @@
+# TinyTorch PDF Book Generation
+
+This directory contains the configuration for generating the TinyTorch course as a PDF book.
+
+## Building the PDF
+
+To build the PDF version of the TinyTorch course:
+
+```bash
+# Install Jupyter Book if not already installed
+pip install jupyter-book
+
+# Build the PDF (from the docs/ directory)
+jupyter-book build . --builder pdflatex
+
+# Or from the repository root:
+jupyter-book build docs --builder pdflatex
+```
+
+The generated PDF will be in `docs/_build/latex/tinytorch-course.pdf`.
+
+## Structure
+
+- `_config_pdf.yml` - Jupyter Book configuration optimized for PDF output
+- `_toc_pdf.yml` - Linear table of contents for the PDF book
+- `cover.md` - Cover page for the PDF
+- `preface.md` - Preface explaining the book's approach and philosophy
+
+## Content Sources
+
+The PDF pulls content from:
+- **Module ABOUT.md files**: `../modules/XX_*/ABOUT.md` - Core technical content
+- **Site files**: `../site/*.md` - Introduction, quick start guide, resources
+- **Site chapters**: `../site/chapters/*.md` - Course overview and milestones
+
+All content is sourced from a single location and reused for both the website and PDF, ensuring consistency.
+
+## Customization
+
+### PDF-Specific Settings
+
+The `_config_pdf.yml` includes PDF-specific settings:
+- Disabled notebook execution (`execute_notebooks: "off"`)
+- LaTeX engine configuration
+- Custom page headers and formatting
+- Paper size and typography settings
+
+### Chapter Ordering
+
+The `_toc_pdf.yml` provides linear chapter ordering suitable for reading cover-to-cover, unlike the website's multi-section structure.
+
+## Dependencies
+
+Building the PDF requires:
+- `jupyter-book`
+- `pyppeteer` (for HTML to PDF conversion)
+- LaTeX distribution (e.g., TeX Live, MiKTeX)
+- `latexmk` (usually included with LaTeX distributions)
+
+## Troubleshooting
+
+**LaTeX errors**: Ensure you have a complete LaTeX distribution installed
+**Missing fonts**: Install the required fonts for the logo and styling
+**Build timeouts**: Increase the timeout in `_config_pdf.yml` if needed
+
+## Future Enhancements
+
+Planned improvements for the PDF:
+- Custom LaTeX styling for code blocks
+- Better figure placement and captions
+- Index generation
+- Cross-reference optimization
+- Improved table formatting

site/chapters/milestones.md

@@ -0,0 +1,315 @@
+# 🏆 Journey Through ML History
+
+**Experience the evolution of AI by rebuilding history's most important breakthroughs with YOUR TinyTorch implementations!**
+
+---
+
+## 🎯 What Are Milestones?
+
+Milestones are **proof-of-mastery demonstrations** that showcase what you can build after completing specific modules. Each milestone recreates a historically significant ML achievement using YOUR implementations.
+
+### Why This Approach?
+
+- 🧠 **Deep Understanding**: Experience the actual challenges researchers faced
+- 📈 **Progressive Learning**: Each milestone builds on previous foundations
+- 🏆 **Real Achievements**: Not toy examples - these are historically significant breakthroughs
+- 🔧 **Systems Thinking**: Understand WHY each innovation mattered for ML systems
+
+---
+
+## 📅 The Timeline
+
+### 🧠 01. Perceptron (1957) - Rosenblatt
+
+**After Modules 02-04**
+
+```
+Input → Linear → Sigmoid → Output
+```
+
+**The Beginning**: The first trainable neural network! Frank Rosenblatt proved machines could learn from data.
+
+**What You'll Build**:
+- Binary classification with gradient descent
+- Simple but revolutionary architecture
+- YOUR Linear layer recreates history
+
+**Systems Insights**:
+- Memory: O(n) parameters
+- Compute: O(n) operations
+- Limitation: Only linearly separable problems
+
+```bash
+cd milestones/01_1957_perceptron
+python perceptron_trained.py
+```
+
+**Expected Results**: 95%+ accuracy on linearly separable data
+
+---
+
+### ⚡ 02. XOR Crisis (1969) - Minsky & Papert
+
+**After Modules 02-06**
+
+```
+Input → Linear → ReLU → Linear → Output
+```
+
+**The Challenge**: Minsky proved perceptrons couldn't solve XOR. This crisis nearly ended AI research!
+
+**What You'll Build**:
+- Hidden layers enable non-linear solutions
+- Multi-layer networks break through limitations
+- YOUR autograd makes it possible
+
+**Systems Insights**:
+- Memory: O(n²) with hidden layers
+- Compute: O(n²) operations
+- Breakthrough: Hidden representations
+
+```bash
+cd milestones/02_1969_xor_crisis
+python xor_solved.py
+```
+
+**Expected Results**: 90%+ accuracy solving XOR
+
+---
+
+### 🔢 03. MLP Revival (1986) - Backpropagation Era
+
+**After Modules 02-08**
+
+```
+Images → Flatten → Linear → ReLU → Linear → ReLU → Linear → Classes
+```
+
+**The Revolution**: Backpropagation enabled training deep networks on real datasets like MNIST.
+
+**What You'll Build**:
+- Multi-class digit recognition
+- Complete training pipelines
+- YOUR optimizers achieve 95%+ accuracy
+
+**Systems Insights**:
+- Memory: ~100K parameters for MNIST
+- Compute: Dense matrix operations
+- Architecture: Multi-layer feature learning
+
+```bash
+cd milestones/03_1986_mlp_revival
+python mlp_digits.py      # 8x8 digits (quick)
+python mlp_mnist.py       # Full MNIST
+```
+
+**Expected Results**: 95%+ accuracy on MNIST
+
+---
+
+### 🖼️ 04. CNN Revolution (1998) - LeCun's Breakthrough
+
+**After Modules 02-09** • **🎯 North Star Achievement**
+
+```
+Images → Conv → ReLU → Pool → Conv → ReLU → Pool → Flatten → Linear → Classes
+```
+
+**The Game-Changer**: CNNs exploit spatial structure for computer vision. This enabled modern AI!
+
+**What You'll Build**:
+- Convolutional feature extraction
+- Natural image classification (CIFAR-10)
+- YOUR Conv2d + MaxPool2d unlock spatial intelligence
+
+**Systems Insights**:
+- Memory: ~1M parameters (weight sharing reduces vs dense)
+- Compute: Convolution is intensive but parallelizable
+- Architecture: Local connectivity + translation invariance
+
+```bash
+cd milestones/04_1998_cnn_revolution
+python cnn_digits.py          # Spatial features on digits
+python lecun_cifar10.py       # CIFAR-10 @ 75%+ accuracy
+```
+
+**Expected Results**: **75%+ accuracy on CIFAR-10** ✨
+
+---
+
+### 🤖 05. Transformer Era (2017) - Attention Revolution
+
+**After Modules 02-13**
+
+```
+Tokens → Embeddings → Attention → FFN → ... → Attention → Output
+```
+
+**The Modern Era**: Transformers + attention launched the LLM revolution (GPT, BERT, ChatGPT).
+
+**What You'll Build**:
+- Self-attention mechanisms
+- Autoregressive text generation
+- YOUR attention implementation generates language
+
+**Systems Insights**:
+- Memory: O(n²) attention requires careful management
+- Compute: Highly parallelizable
+- Architecture: Long-range dependencies
+
+```bash
+cd milestones/05_2017_transformer_era
+python vaswani_shakespeare.py
+```
+
+**Expected Results**: Coherent text generation
+
+---
+
+### ⚡ 06. Systems Age (2024) - Modern ML Engineering
+
+**After Modules 02-19**
+
+```
+Profile → Analyze → Optimize → Benchmark → Compete
+```
+
+**The Present**: Modern ML is systems engineering - profiling, optimization, and production deployment.
+
+**What You'll Build**:
+- Performance profiling tools
+- Memory optimization techniques
+- Competitive benchmarking
+
+**Systems Insights**:
+- Full ML systems pipeline
+- Production optimization patterns
+- Real-world engineering trade-offs
+
+```bash
+cd milestones/06_2024_systems_age
+python optimize_models.py
+```
+
+**Expected Results**: Production-grade optimized models
+
+---
+
+## 🎓 Learning Philosophy
+
+### Progressive Capability Building
+
+| Stage | Era | Capability | Your Tools |
+|-------|-----|-----------|-----------|
+| **1957** | Foundation | Binary classification | Linear + Sigmoid |
+| **1969** | Depth | Non-linear problems | Hidden layers + Autograd |
+| **1986** | Scale | Multi-class vision | Optimizers + Training |
+| **1998** | Structure | Spatial understanding | Conv2d + Pooling |
+| **2017** | Attention | Sequence modeling | Transformers + Attention |
+| **2024** | Systems | Production deployment | Profiling + Optimization |
+
+### Systems Engineering Progression
+
+Each milestone teaches critical systems thinking:
+
+1. **Memory Management**: From O(n) → O(n²) → O(n²) with optimizations
+2. **Computational Trade-offs**: Accuracy vs efficiency
+3. **Architectural Patterns**: How structure enables capability
+4. **Production Deployment**: What it takes to scale
+
+---
+
+## 🚀 How to Use Milestones
+
+### 1. Complete Prerequisites
+
+```bash
+# Check which modules you've completed
+tito checkpoint status
+
+# Complete required modules
+tito module complete 02_tensor
+tito module complete 03_activations
+# ... and so on
+```
+
+### 2. Run the Milestone
+
+```bash
+cd milestones/01_1957_perceptron
+python perceptron_trained.py
+```
+
+### 3. Understand the Systems
+
+Each milestone includes:
+- 📊 **Memory profiling**: See actual memory usage
+- ⚡ **Performance metrics**: FLOPs, parameters, timing
+- 🧠 **Architectural analysis**: Why this design matters
+- 📈 **Scaling insights**: How performance changes with size
+
+### 4. Reflect and Compare
+
+**Questions to ask:**
+- How does this compare to modern architectures?
+- What were the computational constraints in that era?
+- How would you optimize this for production?
+- What patterns appear in PyTorch/TensorFlow?
+
+---
+
+## 🎯 Quick Reference
+
+### Milestone Prerequisites
+
+| Milestone | After Module | Key Requirements |
+|-----------|-------------|-----------------|
+| 01. Perceptron (1957) | 04 | Tensor, Activations, Layers |
+| 02. XOR (1969) | 06 | + Losses, Autograd |
+| 03. MLP (1986) | 08 | + Optimizers, Training |
+| 04. CNN (1998) | 09 | + Spatial, DataLoader |
+| 05. Transformer (2017) | 13 | + Tokenization, Embeddings, Attention |
+| 06. Systems (2024) | 19 | Full optimization suite |
+
+### What Each Milestone Proves
+
+✅ **Your implementations work** - Not just toy code  
+✅ **Historical significance** - These breakthroughs shaped modern AI  
+✅ **Systems understanding** - You know memory, compute, scaling  
+✅ **Production relevance** - Patterns used in real ML frameworks
+
+---
+
+## 📚 Further Learning
+
+After completing milestones, explore:
+
+- **TinyMLPerf Competition**: Optimize your implementations
+- **Leaderboard**: Compare with other students
+- **Capstone Projects**: Build your own ML applications
+- **Research Papers**: Read the original papers for each milestone
+
+---
+
+## 🌟 Why This Matters
+
+**Most courses teach you to USE frameworks.**  
+**TinyTorch teaches you to UNDERSTAND them.**
+
+By rebuilding ML history, you gain:
+- 🧠 Deep intuition for how neural networks work
+- 🔧 Systems thinking for production ML
+- 🏆 Portfolio projects demonstrating mastery
+- 💼 Preparation for ML systems engineering roles
+
+---
+
+**Ready to start your journey through ML history?**
+
+```bash
+cd milestones/01_1957_perceptron
+python perceptron_trained.py
+```
+
+**Build the future by understanding the past.** 🚀
+

site/checkpoint-system.md

@@ -0,0 +1,295 @@
+# 🎯 TinyTorch Checkpoint System
+
+<div style="background: #f8f9fa; border: 1px solid #dee2e6; padding: 2rem; border-radius: 0.5rem; text-align: center; margin: 2rem 0;">
+<h2 style="margin: 0 0 1rem 0; color: #495057;">Technical Implementation Guide</h2>
+<p style="margin: 0; color: #6c757d;">Capability validation system architecture and implementation details</p>
+</div>
+
+**Purpose**: Technical documentation for the checkpoint validation system. Understand the architecture and implementation details of capability-based learning assessment.
+
+The TinyTorch checkpoint system provides technical infrastructure for capability validation and progress tracking. This system transforms traditional module completion into measurable skill assessment through automated testing and validation.
+
+<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(250px, 1fr)); gap: 1rem; margin: 2rem 0;">
+
+<div style="background: #f8f9fa; border-left: 4px solid #007bff; padding: 1rem; border-radius: 0.25rem;">
+<h4 style="margin: 0 0 0.5rem 0; color: #0056b3;">Progress Markers</h4>
+<p style="margin: 0; font-size: 0.9rem; color: #6c757d;">Academic milestones marking concrete learning achievements</p>
+</div>
+
+<div style="background: #f8f9fa; border-left: 4px solid #28a745; padding: 1rem; border-radius: 0.25rem;">
+<h4 style="margin: 0 0 0.5rem 0; color: #1e7e34;">Capability-Based</h4>
+<p style="margin: 0; font-size: 0.9rem; color: #6c757d;">Unlock actual ML systems engineering capabilities</p>
+</div>
+
+<div style="background: #f8f9fa; border-left: 4px solid #ffc107; padding: 1rem; border-radius: 0.25rem;">
+<h4 style="margin: 0 0 0.5rem 0; color: #856404;">Cumulative Learning</h4>
+<p style="margin: 0; font-size: 0.9rem; color: #6c757d;">Each checkpoint builds comprehensive expertise</p>
+</div>
+
+<div style="background: #f8f9fa; border-left: 4px solid #6f42c1; padding: 1rem; border-radius: 0.25rem;">
+<h4 style="margin: 0 0 0.5rem 0; color: #4e2b80;">Visual Progress</h4>
+<p style="margin: 0; font-size: 0.9rem; color: #6c757d;">Rich CLI tools with achievement visualization</p>
+</div>
+
+</div>
+
+---
+
+## 🚀 The Five Major Checkpoints
+
+### 🎯 Foundation
+*Core ML primitives and environment setup*
+
+**Modules**: Setup • Tensors • Activations  
+**Capability Unlocked**: "Can build mathematical operations and ML primitives"
+
+**What You Build:**
+- Working development environment with all tools
+- Multi-dimensional tensor operations (the foundation of all ML)
+- Mathematical functions that enable neural network learning
+- Core computational primitives that power everything else
+
+---
+
+### 🎯 Neural Architecture
+*Building complete neural network architectures*
+
+**Modules**: Layers • Dense • Spatial • Attention  
+**Capability Unlocked**: "Can design and construct any neural network architecture"
+
+**What You Build:**
+- Fundamental layer abstractions for all neural networks
+- Dense (fully-connected) networks for classification
+- Convolutional layers for spatial pattern recognition
+- Attention mechanisms for sequence and vision tasks
+- Complete architectural building blocks
+
+---
+
+### 🎯 Training 
+*Complete model training pipeline*
+
+**Modules**: DataLoader • Autograd • Optimizers • Training  
+**Capability Unlocked**: "Can train neural networks on real datasets"
+
+**What You Build:**
+- CIFAR-10 data loading and preprocessing pipeline
+- Automatic differentiation engine (the "magic" behind PyTorch)
+- SGD and Adam optimizers with memory profiling
+- Complete training orchestration system
+- Real model training on real datasets
+
+---
+
+### 🎯 Inference Deployment
+*Optimized model deployment and serving*
+
+**Modules**: Compression • Kernels • Benchmarking • MLOps  
+**Capability Unlocked**: "Can deploy optimized models for production inference"
+
+**What You Build:**
+- Model compression techniques (75% size reduction achievable)
+- High-performance kernel optimizations
+- Systematic performance benchmarking
+- Production monitoring and deployment systems
+- Real-world inference optimization
+
+---
+
+### 🔥 Language Models
+*Framework generalization across modalities*
+
+**Modules**: TinyGPT  
+**Capability Unlocked**: "Can build unified frameworks that support both vision and language"
+
+**What You Build:**
+- GPT-style transformer using your framework components
+- Character-level tokenization and text generation
+- 95% component reuse from vision to language
+- Understanding of universal ML foundations
+
+---
+
+## 📊 Tracking Your Progress
+
+### Visual Timeline
+See your journey through the ML systems engineering pipeline:
+
+```
+Foundation → Architecture → Training → Inference → Language Models
+```
+
+Each checkpoint represents a major learning milestone and capability unlock in your unified vision+language framework.
+
+### Rich Progress Tracking
+Within each checkpoint, track granular progress through individual modules with enhanced Rich CLI visualizations:
+
+```
+🎯 Neural Architecture ████████▓▓▓▓ 66%
+   ✅ Layers ──── ✅ Dense ──── 🔄 Spatial ──── ⏳ Attention
+     │              │            │              │
+   100%           100%          33%            0%
+```
+
+### Capability Statements
+Every checkpoint completion unlocks a concrete capability:
+- ✅ "I can build mathematical operations and ML primitives"
+- ✅ "I can design and construct any neural network architecture"  
+- 🔄 "I can train neural networks on real datasets"
+- ⏳ "I can deploy optimized models for production inference"
+- 🔥 "I can build unified frameworks supporting vision and language"
+
+---
+
+## 🛠️ Technical Usage
+
+The checkpoint system provides comprehensive progress tracking and capability validation through automated testing infrastructure.
+
+**📖 See [Essential Commands](tito-essentials.html)** for complete command reference and usage examples.
+
+### Integration with Development
+The checkpoint system connects directly to your actual development work:
+
+#### Automatic Module-to-Checkpoint Mapping
+Each module automatically maps to its corresponding checkpoint for seamless testing integration.
+
+#### Real Capability Validation
+- **Not just code completion**: Tests verify actual functionality works
+- **Import testing**: Ensures modules export correctly to package
+- **Functionality testing**: Validates capabilities like tensor operations, neural layers
+- **Integration testing**: Confirms components work together
+
+#### Rich Visual Feedback
+- **Achievement celebrations**: 🎉 when checkpoints are completed
+- **Progress visualization**: Rich CLI progress bars and timelines
+- **Next step guidance**: Suggests the next module to work on
+- **Capability statements**: Clear "I can..." statements for each achievement
+
+---
+
+## 🏗️ Implementation Architecture
+
+### 16 Individual Test Files
+Each checkpoint is implemented as a standalone Python test file in `tests/checkpoints/`:
+```
+tests/checkpoints/
+├── checkpoint_00_environment.py   # "Can I configure my environment?"
+├── checkpoint_01_foundation.py    # "Can I create ML building blocks?"
+├── checkpoint_02_intelligence.py  # "Can I add nonlinearity?"
+├── ...
+└── checkpoint_15_capstone.py      # "Can I build complete end-to-end ML systems?"
+```
+
+### Rich CLI Integration
+The command-line interface provides:
+- **Visual progress tracking** with progress bars and timelines
+- **Capability testing** with immediate feedback
+- **Achievement celebrations** with next step guidance
+- **Detailed status reporting** with module-level information
+
+### Automated Module Completion
+The module completion workflow:
+1. **Exports module** using existing export functionality
+2. **Maps module to checkpoint** using predefined mapping table
+3. **Runs capability test** with Rich progress visualization
+4. **Shows results** with achievement celebration or guidance
+
+### Agent Team Implementation
+This system was successfully implemented by coordinated AI agents:
+- **Module Developer**: Built checkpoint tests and CLI integration
+- **QA Agent**: Tested all 21 checkpoints and CLI functionality
+- **Package Manager**: Validated integration with package system
+- **Documentation Publisher**: Created this documentation and usage guides
+
+---
+
+## 🧠 Why This Approach Works
+
+### Systems Thinking Over Task Completion
+Traditional approach: *"I finished Module 3"*  
+Checkpoint approach: *"My framework can now build neural networks"
+
+### Clear Learning Goals
+Every module contributes to a **concrete system capability** rather than abstract completion.
+
+### Academic Progress Markers
+- **Rich CLI visualizations** with progress bars and connecting lines show your growing ML framework
+- **Capability unlocks** feel like real learning milestones achieved in academic progression
+- **Clear direction** toward complete ML systems mastery through structured checkpoints
+- **Visual timeline** similar to academic transcripts tracking completed coursework
+
+### Real-World Relevance
+The checkpoint progression **Foundation → Architecture → Training → Inference → Language Models** mirrors both academic learning progression and the evolution from specialized to unified ML frameworks.
+
+---
+
+## 🐛 Debugging Checkpoint Failures
+
+**When checkpoint tests fail, use debugging strategies to identify and resolve issues:**
+
+### Common Failure Patterns
+
+**Import Errors:**
+- **Problem**: Module not found errors indicate missing exports
+- **Solution**: Ensure modules are properly exported and environment is configured
+
+**Functionality Errors:**
+- **Problem**: Implementation doesn't work as expected (shape mismatches, incorrect outputs)
+- **Debug approach**: Use verbose testing to get detailed error information
+
+**Integration Errors:**
+- **Problem**: Modules don't work together due to missing dependencies
+- **Solution**: Verify prerequisite capabilities before testing advanced features
+
+**📖 See [Essential Commands](tito-essentials.html)** for complete debugging command reference.
+
+### Checkpoint Test Structure
+
+**Each checkpoint test follows this pattern:**
+```python
+# Example: checkpoint_01_foundation.py
+import sys
+sys.path.append('/path/to/tinytorch')
+
+try:
+    from tinytorch.core.tensor import Tensor
+    print("✅ Tensor import successful")
+except ImportError as e:
+    print(f"❌ Tensor import failed: {e}")
+    sys.exit(1)
+
+# Test basic functionality
+tensor = Tensor([[1, 2], [3, 4]])
+assert tensor.shape == (2, 2), f"Expected shape (2, 2), got {tensor.shape}"
+print("✅ Basic tensor operations working")
+
+# Test integration capabilities
+result = tensor + tensor
+assert result.data.tolist() == [[2, 4], [6, 8]], "Addition failed"
+print("✅ Tensor arithmetic working")
+
+print("🏆 Foundation checkpoint PASSED")
+```
+
+---
+
+## 🚀 Advanced Usage Features
+
+**The checkpoint system supports advanced development workflows:**
+
+### Batch Testing
+- Test multiple checkpoints simultaneously
+- Test ranges of checkpoints for comprehensive validation
+- Validate all completed checkpoints for regression testing
+
+### Custom Checkpoint Development
+- Create custom checkpoint tests for extensions
+- Run custom validation with verbose output
+- Extend the checkpoint system for specialized needs
+
+### Performance Profiling
+- Profile checkpoint execution performance
+- Analyze memory usage during testing
+- Identify bottlenecks in capability validation
+
+**📖 See [Essential Commands](tito-essentials.html)** for complete command reference and advanced usage examples.

site/community.md

@@ -0,0 +1,58 @@
+# 🌍 Community Ecosystem
+
+**Building Together**
+
+---
+
+## 🎯 Overview
+
+TinyTorch is more than just a course—it's a growing community of students, educators, and ML engineers learning systems engineering from first principles.
+
+---
+
+## 📊 Community Platform (Coming Soon)
+
+<div style="background: #e3f2fd; border: 2px solid #2196f3; padding: 1.5rem; border-radius: 0.5rem; margin: 2rem 0;">
+<h3 style="margin: 0 0 1rem 0; color: #1565c0;">🚧 Building Community Features</h3>
+<p style="margin: 0; color: #1565c0;">We're creating live community features including activity dashboards, study partner matching, and real-time progress tracking. Stay tuned!</p>
+</div>
+
+### Planned Features
+
+**📊 Live Dashboard**
+- Real-time community activity
+- Global learning progress
+- Module completion stats
+
+**🤝 Connection Hub**
+- Find study partners
+- Join study groups
+- Connect with peers
+
+**🌍 Global Reach**
+- See who's learning worldwide
+- Geographic distribution
+- Community milestones
+
+---
+
+## 🚀 Get Involved Now
+
+**Learn Together**
+- Ask questions in [GitHub Discussions](https://github.com/harvard-edge/TinyTorch/discussions)
+- Share your progress and projects
+- Help others debug their implementations
+
+**Contribute**
+- Report bugs and issues on GitHub
+- Improve documentation
+- Submit fixes and optimizations
+
+**Stay Connected**
+- Star the project on [GitHub](https://github.com/harvard-edge/TinyTorch)
+- Follow development updates
+- Share TinyTorch with others
+
+---
+
+**Build ML systems. Learn together. Grow the community.** 🌍

site/conf.py

@@ -0,0 +1,39 @@
+###############################################################################
+# Auto-generated by `jupyter-book config`
+# If you wish to continue using _config.yml, make edits to that file and
+# re-generate this one.
+###############################################################################
+author = 'Prof. Vijay Janapa Reddi (Harvard University)'
+bibtex_bibfiles = ['references.bib']
+comments_config = {'hypothesis': False, 'utterances': False}
+copyright = '2025'
+exclude_patterns = ['**.ipynb_checkpoints', '**/.DS_Store', '**/.venv/**', '**/__pycache__/**', '.DS_Store', '.venv', 'Thumbs.db', '_build', 'appendices']
+extensions = ['sphinx_togglebutton', 'sphinx_copybutton', 'myst_nb', 'jupyter_book', 'sphinx_thebe', 'sphinx_comments', 'sphinx_external_toc', 'sphinx.ext.intersphinx', 'sphinx_design', 'sphinx_book_theme', 'sphinxcontrib.mermaid', 'sphinxcontrib.bibtex', 'sphinx_jupyterbook_latex', 'sphinx_multitoc_numbering']
+external_toc_exclude_missing = True
+external_toc_path = '_toc.yml'
+html_baseurl = ''
+html_css_files = ['custom.css']
+html_favicon = '_static/favicon.svg'
+html_js_files = ['wip-banner.js']
+html_logo = 'logo-tinytorch-white.png'
+html_sourcelink_suffix = ''
+html_static_path = ['_static']
+html_theme = 'sphinx_book_theme'
+html_theme_options = {'search_bar_text': 'Search this book...', 'launch_buttons': {'notebook_interface': 'classic', 'binderhub_url': '', 'jupyterhub_url': '', 'thebe': False, 'colab_url': '', 'deepnote_url': ''}, 'path_to_docs': 'book', 'repository_url': 'https://github.com/mlsysbook/TinyTorch', 'repository_branch': 'main', 'extra_footer': '', 'home_page_in_toc': True, 'announcement': '', 'analytics': {'google_analytics_id': '', 'plausible_analytics_domain': '', 'plausible_analytics_url': 'https://plausible.io/js/script.js'}, 'use_repository_button': True, 'use_edit_page_button': True, 'use_issues_button': True}
+html_title = 'TinyTorch'
+latex_engine = 'pdflatex'
+mermaid_version = '10.6.1'
+myst_enable_extensions = ['colon_fence', 'deflist', 'html_admonition', 'html_image', 'linkify', 'replacements', 'smartquotes', 'substitution', 'tasklist']
+myst_url_schemes = ['mailto', 'http', 'https']
+nb_execution_allow_errors = True
+nb_execution_cache_path = ''
+nb_execution_excludepatterns = []
+nb_execution_in_temp = False
+nb_execution_mode = 'cache'
+nb_execution_timeout = 300
+nb_output_stderr = 'show'
+numfig = True
+pygments_style = 'sphinx'
+suppress_warnings = ['myst.domains']
+use_jupyterbook_latex = True
+use_multitoc_numbering = True

site/intro.md

@@ -0,0 +1,217 @@
+<div id="wip-banner" class="wip-banner">
+  <div class="wip-banner-content">
+    <div class="wip-banner-title">
+      <span class="icon">🚧</span>
+      <span class="icon">⚠️</span>
+      <span>Under Construction - Active Development</span>
+      <span class="icon">🔨</span>
+      <span class="icon">🚧</span>
+    </div>
+    <div class="wip-banner-description">
+      TinyTorch is under active construction! We're building in public and sharing our progress for early feedback. Expect frequent updates, changes, and improvements as we develop the framework together with the community.
+    </div>
+    <button id="wip-banner-toggle" class="wip-banner-toggle" title="Collapse banner">
+      <i class="fas fa-chevron-up"></i>
+    </button>
+    <button id="wip-banner-close" class="wip-banner-close" title="Dismiss banner">
+      ×
+    </button>
+  </div>
+</div>
+
+# TinyTorch: Build ML Systems from Scratch
+
+<h2 style="background: linear-gradient(135deg, #E74C3C 0%, #E67E22 50%, #F39C12 100%); -webkit-background-clip: text; -webkit-text-fill-color: transparent; background-clip: text; text-align: center; font-size: 2.5rem; margin: 3rem 0;">
+Don't just import it. Build it.
+</h2>
+
+## What is TinyTorch?
+
+TinyTorch is an educational ML systems course where you **build complete neural networks from scratch**. Instead of blindly using PyTorch or TensorFlow as black boxes, you implement every component yourself—from tensors and gradients to optimizers and attention mechanisms—gaining deep understanding of how modern ML frameworks actually work.
+
+**Core Learning Approach**: Build → Profile → Optimize. You'll implement each system component, measure its performance characteristics, and understand the engineering trade-offs that shape production ML systems.
+
+## Three-Tier Learning Pathway
+
+TinyTorch organizes learning through **three pedagogically-motivated tiers** that follow ML history:
+
+**🏗️ Foundation Tier (Modules 01-07)**: Build mathematical infrastructure - tensors, autograd, optimizers
+**🏛️ Architecture Tier (Modules 08-13)**: Implement modern AI - CNNs for vision, transformers for language
+**⚡ Optimization Tier (Modules 14-20)**: Deploy production systems - profiling, quantization, acceleration
+
+Each tier builds complete, working systems with clear career connections and practical skills.
+
+**📖 See [Complete Three-Tier Structure](chapters/00-introduction.html#three-tier-learning-pathway-build-complete-ml-systems)** for detailed tier breakdown, time estimates, and learning outcomes.
+
+## 🏆 Prove Your Mastery Through History
+
+As you complete modules, unlock **historical milestone demonstrations** that prove what you've built works! From Rosenblatt's 1957 perceptron to modern CNNs achieving 75%+ accuracy on CIFAR-10, each milestone recreates a breakthrough using YOUR implementations:
+
+- **🧠 1957: Perceptron** - First trainable network with YOUR Linear layer
+- **⚡ 1969: XOR Solution** - Multi-layer networks with YOUR autograd
+- **🔢 1986: MNIST MLP** - Backpropagation achieving 95%+ with YOUR optimizers
+- **🖼️ 1998: CIFAR-10 CNN** - Spatial intelligence with YOUR Conv2d (75%+ accuracy!)
+- **🤖 2017: Transformers** - Language generation with YOUR attention
+- **⚡ 2024: Systems Age** - Production optimization with YOUR profiling
+
+**📖 See [Journey Through ML History](chapters/milestones.html)** for complete milestone details and requirements.
+
+## Why Build Instead of Use?
+
+The difference between using a library and understanding a system is the difference between being limited by tools and being empowered to create them. When you build from scratch, you transform from a framework user into a systems engineer:
+
+<div style="display: grid; grid-template-columns: 1fr 1fr; gap: 1.5rem; margin: 2rem 0;">
+
+<!-- Top Row: Using Libraries Examples -->
+<div style="background: #fff5f5; border: 1px solid #feb2b2; padding: 1.5rem; border-radius: 0.5rem; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+<h3 style="margin: 0 0 1rem 0; color: #c53030; font-size: 1.1rem;">❌ Using PyTorch</h3>
+
+```python
+import torch.nn as nn
+import torch.optim as optim
+
+model = nn.Linear(784, 10)
+optimizer = optim.Adam(model.parameters(), lr=0.001)
+
+# Your model trains but then...
+# 🔥 OOM error! Why?
+# 🔥 Loss is NaN! How to debug?
+# 🔥 Training is slow! What's the bottleneck?
+```
+
+<p style="color: #c53030; font-weight: 500; margin-top: 1rem; font-size: 0.9rem;">
+You're stuck when things break
+</p>
+</div>
+
+<div style="background: #fff5f5; border: 1px solid #feb2b2; padding: 1.5rem; border-radius: 0.5rem; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+<h3 style="margin: 0 0 1rem 0; color: #c53030; font-size: 1.1rem;">❌ Using TensorFlow</h3>
+
+```python
+import tensorflow as tf
+
+model = tf.keras.Sequential([
+    tf.keras.layers.Dense(128, activation='relu'),
+    tf.keras.layers.Dense(10)
+])
+
+# Magic happens somewhere...
+# 🤷 How are gradients computed?
+# 🤷 Why this initialization?
+# 🤷 What's happening in backward pass?
+```
+
+<p style="color: #c53030; font-weight: 500; margin-top: 1rem; font-size: 0.9rem;">
+Magic boxes you can't understand
+</p>
+</div>
+
+<!-- Bottom Row: Building Your Own Examples -->
+<div style="background: #f0fff4; border: 1px solid #9ae6b4; padding: 1.5rem; border-radius: 0.5rem; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+<h3 style="margin: 0 0 1rem 0; color: #2f855a; font-size: 1.1rem;">✅ Building TinyTorch</h3>
+
+```python
+class Linear:
+    def __init__(self, in_features, out_features):
+        self.weight = randn(in_features, out_features) * 0.01
+        self.bias = zeros(out_features)
+
+    def forward(self, x):
+        self.input = x  # Save for backward
+        return x @ self.weight + self.bias
+
+    def backward(self, grad):
+        # You wrote this! You know exactly why:
+        self.weight.grad = self.input.T @ grad
+        self.bias.grad = grad.sum(axis=0)
+        return grad @ self.weight.T
+```
+
+<p style="color: #2f855a; font-weight: 500; margin-top: 1rem; font-size: 0.9rem;">
+You can debug anything
+</p>
+</div>
+
+<div style="background: #f0fff4; border: 1px solid #9ae6b4; padding: 1.5rem; border-radius: 0.5rem; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+<h3 style="margin: 0 0 1rem 0; color: #2f855a; font-size: 1.1rem;">✅ Building KV Cache</h3>
+
+```python
+class KVCache:
+    def __init__(self, max_seq_len, n_heads, head_dim):
+        # You understand EXACTLY the memory layout:
+        self.k_cache = zeros(max_seq_len, n_heads, head_dim)
+        self.v_cache = zeros(max_seq_len, n_heads, head_dim)
+        # That's why GPT needs GBs of RAM!
+
+    def update(self, k, v, pos):
+        # You know why position matters:
+        self.k_cache[pos:pos+len(k)] = k  # Reuse past computations
+        self.v_cache[pos:pos+len(v)] = v  # O(n²) → O(n) speedup!
+        # Now you understand why context windows are limited
+```
+
+<p style="color: #2f855a; font-weight: 500; margin-top: 1rem; font-size: 0.9rem;">
+You master modern LLM optimizations
+</p>
+</div>
+
+</div>
+
+## Who Is This For?
+
+**Perfect if you're asking these questions:**
+
+**ML Systems Engineers**: "Why does my model training OOM at batch size 32? How do attention mechanisms scale quadratically with sequence length? When does data loading become the bottleneck?" You'll build and profile every component, understanding memory hierarchies, computational complexity, and system bottlenecks that production ML systems face daily.
+
+**Students & Researchers**: "How does that `nn.Linear()` call actually compute gradients? Why does Adam optimizer need 3× the memory of SGD? What's actually happening during a forward pass?" You'll implement the mathematics you learned in class and discover how theoretical concepts become practical systems with real performance implications.
+
+**Performance Engineers**: "Where are the actual bottlenecks in transformer inference? How does KV-cache reduce computation by 10-100×? Why does my CNN use 4GB of memory?" By building these systems from scratch, you'll understand memory access patterns, cache efficiency, and optimization opportunities that profilers alone can't teach.
+
+**Academics & Educators**: "How can I teach ML systems—not just ML algorithms?" TinyTorch provides a complete pedagogical framework emphasizing systems thinking: memory profiling, performance analysis, and scaling behavior are built into every module, not added as an afterthought.
+
+**ML Practitioners**: "Why does training slow down after epoch 10? How do I debug gradient explosions? When should I use mixed precision?" Even experienced engineers often treat frameworks as black boxes. By understanding the systems underneath, you'll debug faster, optimize better, and make informed architectural decisions.
+
+## How to Choose Your Learning Path
+
+**Three Learning Approaches**: You can **build complete tiers** (implement all 20 modules), **focus on specific tiers** (target your skill gaps), or **explore selectively** (study key concepts). Each tier builds complete, working systems.
+
+<div style="display: grid; grid-template-columns: repeat(2, 1fr); gap: 1.5rem; margin: 3rem 0;">
+
+<!-- Top Row -->
+<div style="background: #f8f9fa; border: 1px solid #dee2e6; padding: 2rem; border-radius: 0.5rem; text-align: center;">
+<h3 style="margin: 0 0 1rem 0; font-size: 1.2rem; color: #495057;">🔬 Quick Start</h3>
+<p style="margin: 0 0 1.5rem 0; font-size: 0.95rem; color: #6c757d;">15 minutes setup • Try foundational modules • Hands-on experience</p>
+<a href="quickstart-guide.html" style="display: inline-block; background: #007bff; color: white; padding: 0.75rem 1.5rem; border-radius: 0.25rem; text-decoration: none; font-weight: 500; font-size: 1rem;">Start Building →</a>
+</div>
+
+<div style="background: #f0fff4; border: 1px solid #9ae6b4; padding: 2rem; border-radius: 0.5rem; text-align: center;">
+<h3 style="margin: 0 0 1rem 0; font-size: 1.2rem; color: #495057;">📚 Full Course</h3>
+<p style="margin: 0 0 1.5rem 0; font-size: 0.95rem; color: #6c757d;">8+ weeks study • Complete ML framework • Systems understanding</p>
+<a href="chapters/00-introduction.html" style="display: inline-block; background: #28a745; color: white; padding: 0.75rem 1.5rem; border-radius: 0.25rem; text-decoration: none; font-weight: 500; font-size: 1rem;">Course Overview →</a>
+</div>
+
+<!-- Bottom Row -->
+<div style="background: #faf5ff; border: 1px solid #b794f6; padding: 2rem; border-radius: 0.5rem; text-align: center;">
+<h3 style="margin: 0 0 1rem 0; font-size: 1.2rem; color: #495057;">🎓 Instructors</h3>
+<p style="margin: 0 0 1.5rem 0; font-size: 0.95rem; color: #6c757d;">Classroom-ready • NBGrader integration • Automated grading</p>
+<a href="usage-paths/classroom-use.html" style="display: inline-block; background: #6f42c1; color: white; padding: 0.75rem 1.5rem; border-radius: 0.25rem; text-decoration: none; font-weight: 500; font-size: 1rem;">Teaching Guide →</a>
+</div>
+
+<div style="background: #fff8dc; border: 1px solid #daa520; padding: 2rem; border-radius: 0.5rem; text-align: center;">
+<h3 style="margin: 0 0 1rem 0; font-size: 1.2rem; color: #495057;">📊 Learning Community</h3>
+<p style="margin: 0 0 1.5rem 0; font-size: 0.95rem; color: #6c757d;">Track progress • Join competitions • Student leaderboard</p>
+<a href="leaderboard.html" style="display: inline-block; background: #b8860b; color: white; padding: 0.75rem 1.5rem; border-radius: 0.25rem; text-decoration: none; font-weight: 500; font-size: 1rem;">View Progress →</a>
+</div>
+
+</div>
+
+## Getting Started
+
+Whether you're just exploring or ready to dive in, here are helpful resources: **📖 See [Essential Commands](tito-essentials.html)** for complete setup and command reference, or **📖 See [Three-Tier Learning Structure](chapters/00-introduction.html#three-tier-learning-pathway-build-complete-ml-systems)** for detailed tier breakdown and learning outcomes.
+
+**Additional Resources**:
+- **[Progress Tracking](learning-progress.html)** - Monitor your learning journey with 21 capability checkpoints
+- **[Testing Framework](testing-framework.html)** - Understand our comprehensive validation system
+- **[Documentation & Guides](resources.html)** - Complete technical documentation and tutorials
+
+TinyTorch is more than a course—it's a community of learners building together. Join thousands exploring ML systems from the ground up.

site/learning-progress.md

@@ -0,0 +1,130 @@
+# Track Your Progress
+
+<div style="background: #f8f9fa; padding: 2rem; border-radius: 0.5rem; margin: 2rem 0; text-align: center;">
+<h2 style="margin: 0 0 1rem 0; color: #495057;">Monitor Your Learning Journey</h2>
+<p style="margin: 0; font-size: 1.1rem; color: #6c757d;">Track your capability development through 16 essential ML systems skills</p>
+</div>
+
+**Purpose**: Monitor your capability development through the 21-checkpoint system. Track progress from foundation skills to production ML systems mastery.
+
+Track your progression through 21 essential ML systems capabilities. Each checkpoint represents fundamental competencies you'll master through hands-on implementation—from tensor operations to production-ready systems.
+
+## How to Track Your Progress
+
+<div style="background: #e3f2fd; padding: 1.5rem; border-radius: 0.5rem; border-left: 4px solid #2196f3; margin: 1.5rem 0;">
+<h4 style="margin: 0 0 1rem 0; color: #1976d2;">🎯 Capability-Based Learning</h4>
+
+Use TinyTorch's 21-checkpoint system to monitor your capability development. Track progress from foundation skills to production ML systems mastery.
+
+**📖 See [Essential Commands](tito-essentials.html)** for complete progress tracking commands and workflow.
+
+</div>
+
+## Your Learning Path Overview
+
+TinyTorch organizes learning through **three pedagogically-motivated tiers**, each building essential ML systems capabilities:
+
+**📖 See [Three-Tier Learning Structure](chapters/00-introduction.html#three-tier-learning-pathway-build-complete-ml-systems)** for detailed tier breakdown, time estimates, and learning outcomes.
+
+## Student Learning Journey
+
+### Typical Student Progression by Tier
+- **🏗️ Foundation Tier (6-8 weeks)**: Build mathematical infrastructure - tensors, autograd, optimizers, training loops
+- **🏛️ Architecture Tier (4-6 weeks)**: Implement modern AI architectures - CNNs for vision, transformers for language
+- **⚡ Optimization Tier (4-6 weeks)**: Deploy production systems - profiling, quantization, acceleration
+
+### Study Approaches
+- **Complete Builder** (14-18 weeks): Implement all three tiers from scratch
+- **Focused Explorer** (4-8 weeks): Pick specific tiers based on your goals
+- **Guided Learner** (8-12 weeks): Study implementations with hands-on exercises
+
+**📖 See [Quick Start Guide](quickstart-guide.html)** for immediate hands-on experience with your first module.
+
+## 21 Core Capabilities
+
+Track progress through essential ML systems competencies:
+
+```{admonition} Capability Tracking
+:class: note
+Each checkpoint validates mastery of fundamental ML systems skills.
+```
+
+| Checkpoint | Capability Question | Modules Required | Status |
+|------------|-------------------|------------------|--------|
+| 00 | Can I set up my environment? | 01 | ⬜ Setup |
+| 01 | Can I manipulate tensors? | 02 | ⬜ Foundation |
+| 02 | Can I add nonlinearity? | 03 | ⬜ Intelligence |
+| 03 | Can I build network layers? | 04 | ⬜ Components |
+| 04 | Can I measure loss? | 05 | ⬜ Networks |
+| 05 | Can I compute gradients? | 06 | ⬜ Learning |
+| 06 | Can I optimize parameters? | 07 | ⬜ Optimization |
+| 07 | Can I train models? | 08 | ⬜ Training |
+| 08 | Can I process images? | 09 | ⬜ Vision |
+| 09 | Can I load data efficiently? | 10 | ⬜ Data |
+| 10 | Can I process text? | 11 | ⬜ Language |
+| 11 | Can I create embeddings? | 12 | ⬜ Representation |
+| 12 | Can I implement attention? | 13 | ⬜ Attention |
+| 13 | Can I build transformers? | 14 | ⬜ Architecture |
+| 14 | Can I profile performance? | 14 | ⬜ Deployment |
+| 15 | Can I quantize models? | 15 | ⬜ Quantization |
+| 16 | Can I compress networks? | 16 | ⬜ Compression |
+| 17 | Can I cache computations? | 17 | ⬜ Memoization |
+| 18 | Can I accelerate algorithms? | 18 | ⬜ Acceleration |
+| 19 | Can I benchmark competitively? | 19 | ⬜ Competition |
+| 20 | Can I build complete language models? | 20 | ⬜ TinyGPT Capstone |
+
+**📖 See [Essential Commands](tito-essentials.html)** for progress monitoring commands.
+
+---
+
+## Capability Development Approach
+
+### Foundation Building (Checkpoints 0-3)
+**Capability Focus**: Core computational infrastructure
+- Environment configuration and dependency management
+- Mathematical foundations with tensor operations
+- Neural intelligence through nonlinear activation functions
+- Network component abstractions and forward propagation
+
+### Learning Systems (Checkpoints 4-7)
+**Capability Focus**: Training and optimization
+- Loss measurement and error quantification
+- Automatic differentiation for gradient computation
+- Parameter optimization with advanced algorithms
+- Complete training loop implementation
+
+### Advanced Architectures (Checkpoints 8-13)
+**Capability Focus**: Specialized neural networks
+- Spatial processing for computer vision systems
+- Efficient data loading and preprocessing pipelines
+- Natural language processing and tokenization
+- Representation learning with embeddings
+- Attention mechanisms for sequence understanding
+- Complete transformer architecture mastery
+
+### Production Systems (Checkpoints 14-15)
+**Capability Focus**: Performance and deployment
+- Profiling, optimization, and bottleneck analysis
+- End-to-end ML systems engineering
+- Production-ready deployment and monitoring
+
+---
+
+## Start Building Capabilities
+
+Begin developing ML systems competencies immediately:
+
+<div style="background: #f8f9fa; padding: 2rem; border-radius: 0.5rem; margin: 2rem 0; text-align: center;">
+<h3 style="margin: 0 0 1rem 0; color: #495057;">Begin Capability Development</h3>
+<p style="margin: 0 0 1.5rem 0; color: #6c757d;">Start with foundational capabilities and progress systematically</p>
+<a href="quickstart-guide.html" style="display: inline-block; background: #007bff; color: white; padding: 0.75rem 1.5rem; border-radius: 0.25rem; text-decoration: none; font-weight: 500; margin-right: 1rem;">15-Minute Start →</a>
+<a href="chapters/01-setup.html" style="display: inline-block; background: #28a745; color: white; padding: 0.75rem 1.5rem; border-radius: 0.25rem; text-decoration: none; font-weight: 500;">Begin Setup →</a>
+</div>
+
+## Track Your Progress
+
+To monitor your capability development and learning progression, use the TITO checkpoint commands.
+
+**📖 See [Essential Commands](tito-essentials.html)** for complete command reference and usage examples.
+
+**Approach**: You're building ML systems engineering capabilities through hands-on implementation. Each capability checkpoint validates practical competency, not just theoretical understanding.

site/quickstart-guide.md

@@ -0,0 +1,207 @@
+# Quick Start Guide
+
+<div style="background: #f8f9fa; padding: 2rem; border-radius: 0.5rem; margin: 2rem 0; text-align: center;">
+<h2 style="margin: 0 0 1rem 0; color: #495057;">From Zero to Building Neural Networks</h2>
+<p style="margin: 0; font-size: 1.1rem; color: #6c757d;">Complete setup + first module in 15 minutes</p>
+</div>
+
+**Purpose**: Get hands-on experience building ML systems in 15 minutes. Complete setup verification and build your first neural network component from scratch.
+
+## ⚡ 2-Minute Setup
+
+Let's get you ready to build ML systems:
+
+<div style="background: #e3f2fd; padding: 1.5rem; border-radius: 0.5rem; border-left: 4px solid #2196f3; margin: 1.5rem 0;">
+<h4 style="margin: 0 0 1rem 0; color: #1976d2;">Step 1: One-Command Setup</h4>
+
+```bash
+# Clone repository
+git clone https://github.com/mlsysbook/TinyTorch.git
+cd TinyTorch
+
+# Automated setup (handles everything!)
+./setup-environment.sh
+
+# Activate environment
+source activate.sh
+```
+
+**What this does:**
+- ✅ Creates optimized virtual environment (arm64 on Apple Silicon)
+- ✅ Installs all dependencies (NumPy, Jupyter, Rich, PyTorch for validation)
+- ✅ Configures TinyTorch in development mode
+- ✅ Verifies installation
+
+**📖 See [Essential Commands](tito-essentials.html)** for detailed workflow and troubleshooting.
+
+</div>
+
+<div style="background: #f0fdf4; padding: 1.5rem; border-radius: 0.5rem; border-left: 4px solid #22c55e; margin: 1.5rem 0;">
+<h4 style="margin: 0 0 1rem 0; color: #15803d;">Step 2: Verify Setup</h4>
+
+```bash
+# Run system diagnostics
+tito system doctor
+```
+
+You should see all green checkmarks! This confirms your environment is ready for hands-on ML systems building.
+
+**📖 See [Essential Commands](tito-essentials.html)** for verification commands and troubleshooting.
+
+</div>
+
+## 🏗️ 15-Minute First Module Walkthrough
+
+Let's build your first neural network component and unlock your first capability:
+
+### Module 01: Tensor Foundations
+
+<div style="background: #fffbeb; padding: 1.5rem; border-radius: 0.5rem; border-left: 4px solid #f59e0b; margin: 1.5rem 0;">
+
+**🎯 Learning Goal:** Build N-dimensional arrays - the foundation of all neural networks
+
+**⏱️ Time:** 15 minutes
+
+**💻 Action:** Start with Module 01 to build tensor operations from scratch.
+
+```bash
+# Navigate to the tensor module
+cd modules/01_tensor
+jupyter lab tensor_dev.py
+```
+
+You'll implement core tensor operations:
+- N-dimensional array creation
+- Basic mathematical operations (add, multiply, matmul)
+- Shape manipulation (reshape, transpose)
+- Memory layout understanding
+
+**Key Implementation:** Build the `Tensor` class that forms the foundation of all neural networks
+
+**📖 See [Essential Commands](tito-essentials.html)** for module workflow commands.
+
+**✅ Achievement Unlocked:** Foundation capability - "Can I create and manipulate the building blocks of ML?"
+
+</div>
+
+### Next Step: Module 02 - Activations
+
+<div style="background: #fdf2f8; padding: 1.5rem; border-radius: 0.5rem; border-left: 4px solid #ec4899; margin: 1.5rem 0;">
+
+**🎯 Learning Goal:** Add nonlinearity - the key to neural network intelligence
+
+**⏱️ Time:** 10 minutes
+
+**💻 Action:** Continue with Module 02 to add activation functions.
+
+You'll implement essential activation functions:
+- ReLU (Rectified Linear Unit) - the workhorse of deep learning
+- Softmax - for probability distributions
+- Understand gradient flow and numerical stability
+- Learn why nonlinearity enables learning
+
+**Key Implementation:** Build activation functions that allow neural networks to learn complex patterns
+
+**📖 See [Essential Commands](tito-essentials.html)** for module development workflow.
+
+**✅ Achievement Unlocked:** Intelligence capability - "Can I add nonlinearity to enable learning?"
+
+</div>
+
+## 📊 Track Your Progress
+
+After completing your first modules:
+
+<div style="background: #f8f9fa; padding: 1.5rem; border: 1px solid #dee2e6; border-radius: 0.5rem; margin: 1.5rem 0;">
+
+**Check your new capabilities:** Track your progress through the 21-checkpoint system to see your growing ML systems expertise.
+
+**📖 See [Track Your Progress](learning-progress.html)** for detailed capability tracking and [Essential Commands](tito-essentials.html)** for progress monitoring commands.
+
+</div>
+
+## 🏆 Unlock Historical Milestones
+
+As you progress, **prove what you've built** by recreating history's greatest ML breakthroughs:
+
+<div style="background: linear-gradient(135deg, #667eea 0%, #764ba2 100%); padding: 2rem; border-radius: 0.5rem; margin: 1.5rem 0; color: white;">
+
+**After Module 04**: Build **Rosenblatt's 1957 Perceptron** - the first trainable neural network  
+**After Module 06**: Solve the **1969 XOR Crisis** with multi-layer networks  
+**After Module 08**: Achieve **95%+ accuracy on MNIST** with 1986 backpropagation  
+**After Module 09**: Hit **75%+ on CIFAR-10** with 1998 CNNs - your North Star goal! 🎯
+
+**📖 See [Journey Through ML History](chapters/milestones.html)** for complete milestone demonstrations.
+
+</div>
+
+**Why Milestones Matter**: These aren't toy demos - they're historically significant achievements proving YOUR implementations work at production scale!
+
+## 🎯 What You Just Accomplished
+
+In 15 minutes, you've:
+
+<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(250px, 1fr)); gap: 1rem; margin: 2rem 0;">
+
+<div style="background: #e6fffa; padding: 1rem; border-radius: 0.5rem; border-left: 3px solid #26d0ce;">
+<h4 style="margin: 0 0 0.5rem 0; color: #0d9488;">🔧 Setup Complete</h4>
+<p style="margin: 0; font-size: 0.9rem;">Installed TinyTorch and verified your environment</p>
+</div>
+
+<div style="background: #f0f9ff; padding: 1rem; border-radius: 0.5rem; border-left: 3px solid #3b82f6;">
+<h4 style="margin: 0 0 0.5rem 0; color: #1d4ed8;">🧱 Created Foundation</h4>
+<p style="margin: 0; font-size: 0.9rem;">Implemented core tensor operations from scratch</p>
+</div>
+
+<div style="background: #fefce8; padding: 1rem; border-radius: 0.5rem; border-left: 3px solid #eab308;">
+<h4 style="margin: 0 0 0.5rem 0; color: #a16207;">🏆 First Capability</h4>
+<p style="margin: 0; font-size: 0.9rem;">Earned your first ML systems capability checkpoint</p>
+</div>
+
+</div>
+
+## 🚀 Your Next Steps
+
+<div style="background: #f8f9fa; padding: 2rem; border-radius: 0.5rem; margin: 2rem 0;">
+
+### Immediate Next Actions (Choose One):
+
+**🔥 Continue Building (Recommended):** Begin Module 03 to add intelligence to your network with nonlinear activation functions.
+
+**📚 Learn the Workflow:**
+- **📖 See [Essential Commands](tito-essentials.html)** for complete TITO command guide
+- **📖 See [Track Your Progress](learning-progress.html)** for the full learning path
+
+**🎓 For Instructors:**
+- **📖 See [Classroom Setup Guide](usage-paths/classroom-use.html)** for NBGrader integration and grading workflow
+
+</div>
+
+## 💡 Pro Tips for Continued Success
+
+<div style="background: #fff5f5; padding: 1.5rem; border: 1px solid #fed7d7; border-radius: 0.5rem; margin: 1rem 0;">
+
+**Essential Development Practices:**
+- Always verify your environment before starting
+- Track your progress through capability checkpoints
+- Follow the standard module development workflow
+- Use diagnostic commands when debugging issues
+
+**📖 See [Essential Commands](tito-essentials.html)** for complete workflow commands and troubleshooting guide.
+
+</div>
+
+## 🌟 You're Now a TinyTorch Builder!
+
+<div style="background: #f8f9fa; padding: 2rem; border-radius: 0.5rem; margin: 2rem 0; text-align: center;">
+<h3 style="margin: 0 0 1rem 0; color: #495057;">Ready to Build Production ML Systems</h3>
+<p style="margin: 0 0 1.5rem 0; color: #6c757d;">You've proven you can build ML components from scratch. Time to keep going!</p>
+<a href="chapters/03-activations.html" style="display: inline-block; background: #007bff; color: white; padding: 0.75rem 1.5rem; border-radius: 0.25rem; text-decoration: none; font-weight: 500; margin-right: 1rem;">Continue Building →</a>
+<a href="tito-essentials.html" style="display: inline-block; background: #28a745; color: white; padding: 0.75rem 1.5rem; border-radius: 0.25rem; text-decoration: none; font-weight: 500;">Master Commands →</a>
+</div>
+
+---
+
+**What makes TinyTorch different:** You're not just learning *about* neural networks—you're building them from fundamental mathematical operations. Every line of code you write builds toward complete ML systems mastery.
+
+**Next milestone:** After Module 08, you'll train real neural networks on actual datasets using 100% your own code!

site/requirements.txt

@@ -0,0 +1,32 @@
+# TinyTorch Course Dependencies for Binder/Colab
+# Keep synchronized with main requirements.txt
+
+# Core numerical computing
+numpy>=1.21.0,<2.0.0
+matplotlib>=3.5.0
+
+# Data handling
+PyYAML>=6.0
+
+# Rich terminal formatting (for development feedback)
+rich>=13.0.0
+
+# Jupyter environment
+jupyter>=1.0.0
+jupyterlab>=4.0.0
+ipykernel>=6.0.0
+ipywidgets>=8.0.0
+
+# Sphinx extensions
+sphinxcontrib-mermaid>=0.9.2
+
+# Type checking support  
+typing-extensions>=4.0.0
+
+# For executing TinyTorch code
+setuptools>=70.0.0
+wheel>=0.42.0
+
+# Optional: for advanced visualizations
+# plotly>=5.0.0
+# seaborn>=0.11.0

site/resources.md

@@ -0,0 +1,106 @@
+# 📚 Additional Learning Resources
+
+<div style="background: #f8f9fa; border: 1px solid #dee2e6; padding: 2rem; border-radius: 0.5rem; text-align: center; margin: 2rem 0;">
+<h2 style="margin: 0 0 1rem 0; color: #495057;">Complement Your TinyTorch Journey</h2>
+<p style="margin: 0; color: #6c757d;">Carefully selected resources for broader context, alternative perspectives, and production tools</p>
+</div>
+
+While TinyTorch teaches you to build complete ML systems from scratch, these resources provide broader context, alternative perspectives, and production tools.
+
+**TinyTorch Learning Resources:**
+- **📖 See [Track Your Progress](learning-progress.html)** for monitoring capability development and learning progression
+- **📖 See [Progress Tracking](checkpoint-system.html)** for technical details on capability testing
+- **📖 See [Testing Guide](testing-framework.html)** for comprehensive testing methodology
+- **📖 See [Achievement Showcase](leaderboard.html)** for portfolio development and career readiness
+
+---
+
+## 🎓 Academic Courses
+
+### Machine Learning Systems
+- **[CS 329S: Machine Learning Systems Design](https://stanford-cs329s.github.io/)** (Stanford)  
+  *Production ML systems, infrastructure, and deployment at scale*
+
+- **[CS 6.S965: TinyML and Efficient Deep Learning](https://hanlab.mit.edu/courses/2024-fall-65940)** (MIT)  
+  *Edge computing, model compression, and efficient ML algorithms*
+
+- **[CS 249r: Tiny Machine Learning](https://sites.google.com/g.harvard.edu/tinyml/home)** (Harvard)  
+  *TinyML systems, edge AI, and resource-constrained machine learning*
+
+### Deep Learning Foundations
+- **[CS 231n: Convolutional Neural Networks](http://cs231n.stanford.edu/)** (Stanford)  
+  *Computer vision and CNN architectures - complements TinyTorch spatial modules*
+
+- **[CS 224n: Natural Language Processing](http://web.stanford.edu/class/cs224n/)** (Stanford)  
+  *NLP and transformers - perfect follow-up to TinyTorch attention module*
+
+---
+
+## 📖 Recommended Books
+
+### Systems & Engineering
+- **[Machine Learning Systems](https://mlsysbook.ai)** by Prof. Vijay Janapa Reddi (Harvard)  
+  *Comprehensive systems perspective on ML engineering and optimization - the perfect companion to TinyTorch*
+
+- **[Designing Machine Learning Systems](https://www.oreilly.com/library/view/designing-machine-learning/9781098107956/)** by Chip Huyen  
+  *Production ML engineering, data pipelines, and system design*
+
+- **[Machine Learning Engineering](https://www.mlebook.com/wiki/doku.php)** by Andriy Burkov  
+  *End-to-end ML project lifecycle and best practices*
+
+### Implementation & Theory
+- **[Deep Learning](https://www.deeplearningbook.org/)** by Ian Goodfellow, Yoshua Bengio, Aaron Courville  
+  *Mathematical foundations - the theory behind what you implement in TinyTorch*
+
+- **[Hands-On Machine Learning](https://www.oreilly.com/library/view/hands-on-machine-learning/9781098125967/)** by Aurélien Géron  
+  *Practical implementations using established frameworks*
+
+---
+
+## 🛠️ Alternative Implementations
+
+**Different approaches to building ML systems from scratch - see how others tackle the same challenge:**
+
+### Minimal Frameworks
+- **[Micrograd](https://github.com/karpathy/micrograd)** by Andrej Karpathy  
+  *Minimal autograd engine in 100 lines. **Micrograd shows you the math, TinyTorch shows you the systems.***
+
+- **[Microtorch](https://github.com/Kipre/microtorch)** by Kipre  
+  *PyTorch-like API in pure Python. **Microtorch focuses on clean API design, TinyTorch emphasizes systems engineering and scalability.***
+
+- **[Tinygrad](https://github.com/geohot/tinygrad)** by George Hotz  
+  *Performance-focused educational framework. **Tinygrad optimizes for speed, TinyTorch optimizes for learning.***
+
+- **[Neural Networks from Scratch](https://nnfs.io/)** by Harrison Kinsley  
+  *Math-heavy implementation approach. **NNFS focuses on algorithms, TinyTorch focuses on systems engineering.***
+
+---
+
+## 🏭 Production Internals
+
+### Framework Deep Dives
+- **[PyTorch Internals](http://blog.ezyang.com/2019/05/pytorch-internals/)** by Edward Yang  
+  *How PyTorch actually works under the hood - a great read as see what you built in TinyTorch corresponds to the real PyTorch*
+
+- **[PyTorch Documentation: Extending PyTorch](https://pytorch.org/docs/stable/notes/extending.html)**  
+  *Custom operators and autograd functions - apply your TinyTorch knowledge*
+
+---
+
+*Building ML systems from scratch gives you the implementation foundation most ML engineers lack. These resources help you apply that knowledge to broader systems and production environments.*
+
+## 🚀 Ready to Begin Your Journey?
+
+**Start with the fundamentals and build your way up.**
+
+**📖 See [Essential Commands](tito-essentials.html)** for complete TITO command reference.
+
+**Your Next Steps:**
+1. **📖 See [Quick Start Guide](quickstart-guide.html)** for 15-minute hands-on experience
+2. **📖 See [Track Your Progress](learning-progress.html)** for understanding capability development
+3. **📖 See [Course Introduction](chapters/00-introduction.html)** for deep dive into course philosophy
+
+<div style="background: #f8f9fa; border: 1px solid #dee2e6; padding: 1.5rem; border-radius: 0.5rem; margin: 2rem 0; text-align: center;">
+<h4 style="margin: 0 0 1rem 0; color: #495057;">🎯 Transform from Framework User to Systems Engineer</h4>
+<p style="margin: 0; color: #6c757d;">These external resources complement the hands-on systems building you'll do in TinyTorch</p>
+</div> 

site/testing-framework.md

@@ -0,0 +1,384 @@
+# 🧪 Testing Framework
+
+```{admonition} Test-Driven ML Engineering
+:class: tip
+TinyTorch's testing framework ensures your implementations are not just educational, but production-ready and reliable.
+```
+
+## 🎯 Testing Philosophy: Verify Understanding Through Implementation
+
+TinyTorch testing goes beyond checking syntax - it validates that you understand ML systems engineering through working implementations.
+
+## ⚡ Quick Start: Validate Your Implementation
+
+### 🚀 Run Everything (Recommended)
+```bash
+# Complete validation suite
+tito test --comprehensive
+
+# Expected output:
+# 🧪 Running 16 module tests...
+# 🔗 Running integration tests...
+# 📊 Running performance benchmarks...
+# ✅ Overall TinyTorch Health: 100.0%
+```
+
+### 🎯 Target-Specific Testing
+```bash
+# Test what you just built
+tito module complete 02_tensor && tito checkpoint test 01
+
+# Quick module check
+tito test --module attention --verbose
+
+# Performance validation
+tito test --performance --module training
+```
+
+## 🔬 Testing Levels: From Components to Systems
+
+### 1. 🧩 Module-Level Testing
+**Goal**: Verify individual components work correctly in isolation
+
+```bash
+# Test what you just implemented
+tito test --module tensor --verbose
+tito test --module attention --detailed
+
+# Quick health check for specific module
+tito module validate spatial
+
+# Debug failing module
+tito test --module autograd --debug
+```
+
+**What Gets Tested:**
+- ✅ Core functionality (forward pass, backward pass)
+- ✅ Memory usage patterns and leaks
+- ✅ Mathematical correctness vs reference implementations
+- ✅ Edge cases and error handling
+
+### 2. 🔗 Integration Testing  
+**Goal**: Ensure modules work together seamlessly
+
+```bash
+# Test module dependencies
+tito test --integration --focus training
+
+# Validate export/import chain
+tito test --exports --all-modules
+
+# Full pipeline validation
+tito test --pipeline --from tensor --to training
+```
+
+**Integration Scenarios:**
+- **Tensor → Autograd**: Gradient flow works correctly
+- **Spatial → Training**: CNN training pipeline functions end-to-end
+- **Attention → TinyGPT**: Transformer components integrate properly
+- **All Modules**: Complete framework functionality
+
+### 3. 🏆 Checkpoint Testing
+**Goal**: Validate you've achieved specific learning capabilities
+
+```bash
+# Test your current capabilities
+tito checkpoint test 01  # "Can I create and manipulate tensors?"
+tito checkpoint test 08  # "Can I train neural networks end-to-end?"
+tito checkpoint test 13  # "Can I build attention mechanisms?"
+
+# Progressive capability validation
+tito checkpoint validate --from 00 --to 15
+```
+
+**📖 See [Complete Checkpoint System Documentation](checkpoint-system.html)** for technical implementation details.
+
+**Key Capability Categories:**
+- **Foundation (00-03)**: Building blocks of neural networks
+- **Training (04-08)**: End-to-end learning systems  
+- **Architecture (09-14)**: Advanced model architectures
+- **Optimization (15+)**: Production-ready systems
+
+### 4. 📊 Performance & Systems Testing
+**Goal**: Verify your implementation meets performance expectations
+
+```bash
+# Memory usage analysis
+tito test --memory --module training --profile
+
+# Speed benchmarking
+tito test --speed --compare-baseline
+
+# Scaling behavior validation
+tito test --scaling --model-sizes 1M,5M,10M
+```
+
+**Performance Metrics:**
+- **Memory efficiency**: Peak usage, gradient memory, batch scaling
+- **Training speed**: Convergence time, throughput (samples/sec)
+- **Inference latency**: Forward pass time, batch processing efficiency
+- **Scaling behavior**: Performance vs model size, memory vs accuracy trade-offs
+
+### 5. 🌍 Real-World Example Validation
+**Goal**: Demonstrate production-ready functionality
+
+```bash
+# Train actual models
+tito example train-mnist-mlp     # 95%+ accuracy target
+tito example train-cifar-cnn     # 75%+ accuracy target  
+tito example generate-text       # TinyGPT coherent generation
+
+# Production scenarios
+tito example benchmark-inference  # Speed/memory competitive analysis
+tito example deploy-edge         # Resource-constrained deployment
+```
+
+## 🏗️ Test Architecture: Systems Engineering Approach
+
+### 📋 Progressive Testing Pattern
+Every TinyTorch module follows consistent testing standards:
+
+```python
+# Module testing template (every module follows this pattern)
+class ModuleTest:
+    def test_core_functionality(self):     # Basic operations work
+    def test_mathematical_correctness(self): # Matches reference implementations  
+    def test_memory_usage(self):          # No memory leaks, efficient usage
+    def test_integration_ready(self):     # Exports correctly for other modules
+    def test_real_world_usage(self):      # Works in actual ML pipelines
+```
+
+### 📁 Test Organization Structure
+```bash
+tests/
+├── checkpoints/                    # 16 capability validation tests
+│   ├── checkpoint_00_environment.py   # Development setup working
+│   ├── checkpoint_01_foundation.py    # Tensor operations mastered
+│   └── checkpoint_15_capstone.py      # Complete ML systems expertise
+├── integration/                    # Cross-module compatibility
+│   ├── test_training_pipeline.py      # End-to-end training works
+│   └── test_module_exports.py         # All modules export correctly  
+├── performance/                    # Systems performance validation
+│   ├── memory_profiling.py           # Memory usage analysis
+│   └── speed_benchmarks.py           # Computational performance
+└── examples/                      # Real-world usage validation
+    ├── test_mnist_training.py         # Actual MNIST training works
+    └── test_cifar_cnn.py             # CNN achieves 75%+ on CIFAR-10
+```
+
+## 📊 Understanding Test Results
+
+### 🎯 Health Status Interpretation
+| Score | Status | Action Required |
+|-------|--------|----------------|
+| **100%** | 🟢 Excellent | All systems operational, ready for production |
+| **95-99%** | 🟡 Good | Minor issues, investigate warnings |
+| **90-94%** | 🟠 Caution | Some failing tests, address specific modules |
+| **<90%** | 🔴 Issues | Significant problems, requires immediate attention |
+
+### 🚦 Module Status Indicators
+- ✅ **Passing**: Module implemented correctly, all tests green
+- ⚠️ **Warning**: Minor issues detected, functionality mostly intact  
+- ❌ **Failing**: Critical errors, module needs debugging
+- 🚧 **In Progress**: Module under development, tests expected to fail
+- 🎯 **Checkpoint Ready**: Module ready for capability testing
+
+## 💡 Best Practices: Test-Driven ML Engineering
+
+### 🔄 During Active Development
+```bash
+# Continuous validation workflow
+tito test --module tensor         # After implementing core functionality
+tito test --integration tensor    # After module completion  
+tito checkpoint test 01          # After achieving milestone
+```
+
+**Development Testing Pattern:**
+1. **Write minimal test first**: Define expected behavior before implementation
+2. **Test each component**: Validate individual functions as you build them
+3. **Integration early**: Test module interactions frequently, not just at the end
+4. **Performance check**: Monitor memory and speed throughout development
+
+### ✅ Before Code Commits
+```bash
+# Pre-commit validation checklist
+tito test --comprehensive        # Full test suite passes
+tito system doctor              # Environment is healthy
+tito checkpoint status          # All achieved capabilities still work
+```
+
+**Commit Readiness Criteria:**
+- ✅ All tests pass (100% health status)
+- ✅ No memory leaks detected in performance tests
+- ✅ Integration tests confirm module exports work
+- ✅ Checkpoint tests validate learning objectives met
+
+### 🎯 Before Module Completion
+```bash
+# Module completion validation
+tito test --module mymodule --comprehensive
+tito test --integration --focus mymodule  
+tito module validate mymodule
+tito module complete mymodule    # Only after all tests pass
+```
+
+## 🔧 Troubleshooting Guide
+
+### 🚨 Common Test Failures & Solutions
+
+#### Module Import Errors
+```bash
+# Problem: Module won't import
+❌ ModuleNotFoundError: No module named 'tinytorch.core.tensor'
+
+# Solution: Check module export
+tito module complete tensor      # Ensure module is properly exported
+tito system doctor             # Verify Python path and virtual environment
+```
+
+#### Mathematical Correctness Failures  
+```bash
+# Problem: Your implementation doesn't match reference
+❌ AssertionError: Expected 0.5, got 0.48 (tolerance: 0.01)
+
+# Debug process:
+tito test --module tensor --debug          # Get detailed failure info
+python -c "import tinytorch; help(tinytorch.tensor)"  # Check implementation
+```
+
+#### Memory Usage Issues
+```bash
+# Problem: Memory tests failing
+❌ Memory usage: 150MB (expected: <100MB)
+
+# Investigation:
+tito test --memory --profile tensor       # Get memory profile
+tito test --scaling --module tensor       # Check scaling behavior
+```
+
+#### Integration Test Failures
+```bash
+# Problem: Modules don't work together
+❌ Integration test: tensor→autograd failed
+
+# Debugging approach:
+tito test --integration --focus autograd --verbose
+tito test --exports tensor                # Check tensor exports correctly
+tito test --imports autograd             # Check autograd imports correctly
+```
+
+### 🔍 Advanced Debugging Techniques
+
+#### Verbose Test Output
+```bash
+# Get detailed test information
+tito test --module attention --verbose --debug
+
+# See exact error locations
+tito test --traceback --module training
+```
+
+#### Performance Profiling
+```bash
+# Memory usage analysis
+tito test --memory --profile --module spatial
+
+# Speed profiling  
+tito test --speed --profile --module training --iterations 100
+```
+
+#### Environment Validation
+```bash
+# Complete environment check
+tito system doctor --comprehensive
+
+# Specific dependency verification
+tito system check-dependencies --module autograd
+```
+
+### 📋 Test Failure Decision Tree
+
+```
+Test Failed?
+├── Import Error?
+│   ├── Run `tito system doctor`
+│   └── Check virtual environment activation
+├── Mathematical Error?
+│   ├── Compare with reference implementation
+│   └── Check tensor shapes and dtypes
+├── Memory Error? 
+│   ├── Profile memory usage patterns
+│   └── Check for memory leaks in loops
+├── Integration Error?
+│   ├── Test modules individually first
+│   └── Verify export/import chain
+└── Performance Error?
+    ├── Profile bottlenecks
+    └── Check algorithmic complexity
+```
+
+## 🎯 Testing Philosophy: Building Reliable ML Systems
+
+The TinyTorch testing framework embodies professional ML engineering principles:
+
+### 🧩 KISS Principle in Testing
+- **Consistent patterns**: Every module follows identical testing structure - learn once, apply everywhere
+- **Actionable feedback**: Tests provide specific error messages with exact fix suggestions
+- **Essential focus**: Tests validate critical functionality without unnecessary complexity
+
+### 🔗 Systems Engineering Mindset
+- **Integration-first**: Tests verify components work together, not just in isolation
+- **Real-world validation**: Examples prove your code works on actual datasets (CIFAR-10, MNIST)
+- **Performance consciousness**: All tests include memory and speed awareness
+
+### 📚 Educational Excellence
+- **Understanding verification**: Tests confirm you grasp concepts, not just syntax
+- **Progressive mastery**: Capabilities build systematically through checkpoint validation
+- **Immediate feedback**: Know instantly if your implementation meets professional standards
+
+### 🚀 Production Readiness
+- **Professional standards**: Tests match industry-level validation practices
+- **Scalability validation**: Ensure your code works at realistic data sizes
+- **Reliability assurance**: Comprehensive testing prevents production failures
+
+---
+
+## 🏆 Success Metrics
+
+```{admonition} Testing Success
+:class: tip
+A well-tested TinyTorch implementation should achieve:
+- **100% test suite passing** - All functionality works correctly
+- **>95% memory efficiency** - Comparable to reference implementations  
+- **Real dataset success** - MNIST 95%+, CIFAR-10 75%+ accuracy targets
+- **Clean integration** - All modules work together seamlessly
+```
+
+**Remember**: TinyTorch testing doesn't just verify your code works - it confirms you understand ML systems engineering well enough to build production-ready implementations. 
+
+Your testing discipline here translates directly to building reliable ML systems in industry settings!
+
+## 🚀 Next Steps
+
+**Ready to start testing your implementations?**
+
+```bash
+# Begin with comprehensive health check
+tito test --comprehensive
+
+# Start building and testing your first module
+tito module complete 01_setup
+
+# Track your testing progress
+tito checkpoint status
+```
+
+**Testing Integration with Your Learning Path:**
+- **📖 See [Track Your Progress](learning-progress.html)** for how testing fits into capability development
+- **📖 See [Track Capabilities](checkpoint-system.html)** for automated testing and progress validation
+- **📖 See [Showcase Achievements](leaderboard.html)** for how testing validates the skills you can claim
+
+<div style="background: #e3f2fd; border: 2px solid #1976d2; padding: 1.5rem; border-radius: 0.5rem; margin: 2rem 0; text-align: center;">
+<h4 style="margin: 0 0 1rem 0; color: #1565c0;">🎯 Testing Excellence = ML Systems Mastery</h4>
+<p style="margin: 0; color: #1976d2;">Every test you write and run builds the discipline needed for production ML engineering</p>
+</div>

site/tito-essentials.md

@@ -0,0 +1,272 @@
+# Essential TITO Commands
+
+<div style="background: #f8f9fa; padding: 2rem; border-radius: 0.5rem; margin: 2rem 0; text-align: center;">
+<h2 style="margin: 0 0 1rem 0; color: #495057;">Master the TinyTorch CLI in Minutes</h2>
+<p style="margin: 0; font-size: 1.1rem; color: #6c757d;">Everything you need to build ML systems efficiently</p>
+</div>
+
+**Purpose**: Complete command reference for the TITO CLI. Master the essential commands for development workflow, progress tracking, and system management.
+
+## 🚀 First 4 Commands (Start Here)
+
+Every TinyTorch journey begins with these essential commands:
+
+<div style="display: grid; grid-template-columns: 1fr; gap: 1rem; margin: 2rem 0;">
+
+<div style="background: #e3f2fd; padding: 1.5rem; border-radius: 0.5rem; border-left: 4px solid #2196f3;">
+<h4 style="margin: 0 0 0.5rem 0; color: #1976d2;">📋 Check Your Environment</h4>
+<code style="background: #263238; color: #ffffff; padding: 0.5rem; border-radius: 0.25rem; display: block; margin: 0.5rem 0;">tito system doctor</code>
+<p style="margin: 0.5rem 0 0 0; font-size: 0.9rem; color: #64748b;">Verify your setup is ready for development</p>
+</div>
+
+<div style="background: #f0fdf4; padding: 1.5rem; border-radius: 0.5rem; border-left: 4px solid #22c55e;">
+<h4 style="margin: 0 0 0.5rem 0; color: #15803d;">🎯 Track Your Progress</h4>
+<code style="background: #263238; color: #ffffff; padding: 0.5rem; border-radius: 0.25rem; display: block; margin: 0.5rem 0;">tito checkpoint status</code>
+<p style="margin: 0.5rem 0 0 0; font-size: 0.9rem; color: #64748b;">See which capabilities you've mastered</p>
+</div>
+
+<div style="background: #fffbeb; padding: 1.5rem; border-radius: 0.5rem; border-left: 4px solid #f59e0b;">
+<h4 style="margin: 0 0 0.5rem 0; color: #d97706;">🔨 Work on a Module</h4>
+<code style="background: #263238; color: #ffffff; padding: 0.5rem; border-radius: 0.25rem; display: block; margin: 0.5rem 0;">tito module work 02_tensor</code>
+<p style="margin: 0.5rem 0 0 0; font-size: 0.9rem; color: #64748b;">Open and start building tensor operations</p>
+</div>
+
+<div style="background: #fdf2f8; padding: 1.5rem; border-radius: 0.5rem; border-left: 4px solid #ec4899;">
+<h4 style="margin: 0 0 0.5rem 0; color: #be185d;">✅ Complete Your Work</h4>
+<code style="background: #263238; color: #ffffff; padding: 0.5rem; border-radius: 0.25rem; display: block; margin: 0.5rem 0;">tito module complete 02_tensor</code>
+<p style="margin: 0.5rem 0 0 0; font-size: 0.9rem; color: #64748b;">Export your code and test your capabilities</p>
+</div>
+
+</div>
+
+## 🔄 Your Daily Learning Workflow
+
+Follow this proven pattern for effective learning:
+
+<div style="background: #f8f9fa; padding: 1.5rem; border: 1px solid #dee2e6; border-radius: 0.5rem; margin: 1.5rem 0;">
+
+**Morning Start:**
+```bash
+# 1. Check environment
+tito system doctor
+
+# 2. See your progress  
+tito checkpoint status
+
+# 3. Start working on next module
+tito module work 03_activations
+```
+
+**During Development:**
+```bash
+# Test your understanding anytime
+tito checkpoint test 02
+
+# View your learning timeline
+tito checkpoint timeline
+```
+
+**End of Session:**
+```bash
+# Complete and export your work
+tito module complete 03_activations
+
+# Celebrate your progress!
+tito checkpoint status
+```
+
+</div>
+
+## 💪 Most Important Commands (Top 10)
+
+Master these commands for maximum efficiency:
+
+### 🏥 System & Health
+<div style="background: #f8f9fa; padding: 1rem; border-radius: 0.25rem; margin: 1rem 0;">
+
+**System Check**
+```bash
+tito system doctor
+```
+*Diagnose environment issues before they block you*
+
+**Module Status**  
+```bash
+tito module status
+```
+*See all available modules and your completion status*
+
+</div>
+
+### 📊 Progress Tracking  
+<div style="background: #f8f9fa; padding: 1rem; border-radius: 0.25rem; margin: 1rem 0;">
+
+**Capability Overview**
+```bash
+tito checkpoint status
+```
+*Quick view of your 16 core capabilities*
+
+**Detailed Progress**
+```bash
+tito checkpoint status --detailed
+```
+*Module-by-module breakdown with test status*
+
+**Visual Timeline**
+```bash
+tito checkpoint timeline
+```
+*See your learning journey in beautiful visual format*
+
+</div>
+
+### 🔨 Module Development
+<div style="background: #f8f9fa; padding: 1rem; border-radius: 0.25rem; margin: 1rem 0;">
+
+**Start Working**
+```bash
+tito module work 05_dense
+```
+*Open module and start building*
+
+**Export to Package**
+```bash
+tito module complete 05_dense
+```
+*Export your code to the TinyTorch package + run capability test*
+
+**Quick Export (No Test)**
+```bash
+tito module export 05_dense
+```
+*Export without running capability tests*
+
+</div>
+
+### 🧪 Testing & Validation
+<div style="background: #f8f9fa; padding: 1rem; border-radius: 0.25rem; margin: 1rem 0;">
+
+**Test Specific Capability**
+```bash
+tito checkpoint test 03
+```
+*Verify you've mastered a specific capability*
+
+**Run Checkpoint with Details**
+```bash
+tito checkpoint run 03 --verbose
+```
+*See detailed output of capability validation*
+
+</div>
+
+## 🎓 Learning Stages & Commands
+
+### Stage 1: Foundation (Modules 1-4)
+**Key Commands:**
+- `tito module work 01_setup` → `tito module complete 01_setup`
+- `tito checkpoint test 00` (Environment)
+- `tito checkpoint test 01` (Foundation)
+
+### Stage 2: Core Learning (Modules 5-8)  
+**Key Commands:**
+- `tito checkpoint status` (Track your capabilities)
+- `tito checkpoint timeline` (Visual progress)
+- Complete modules 5-8 systematically
+
+### Stage 3: Advanced Systems (Modules 9+)
+**Key Commands:**
+- `tito checkpoint timeline --horizontal` (Linear view)
+- Focus on systems optimization modules
+- Use `tito checkpoint test XX` for validation
+
+## 👩‍🏫 Instructor Commands (NBGrader)
+
+For instructors managing the course:
+
+<div style="background: #f3e5f5; padding: 1rem; border-radius: 0.25rem; margin: 1rem 0;">
+
+**Setup Course:**
+```bash
+tito nbgrader init              # Initialize NBGrader environment
+tito nbgrader status            # Check assignment status
+```
+
+**Manage Assignments:**
+```bash
+tito nbgrader generate 01_setup  # Create assignment from module
+tito nbgrader release 01_setup   # Release to students
+tito nbgrader collect 01_setup   # Collect submissions
+tito nbgrader autograde 01_setup # Automatic grading
+```
+
+**Reports & Export:**
+```bash
+tito nbgrader report            # Generate grade report
+tito nbgrader export            # Export grades to CSV
+```
+
+*For detailed instructor workflow, see [Instructor Guide](usage-paths/classroom-use.html)*
+
+</div>
+
+## 🚨 Troubleshooting Commands
+
+When things go wrong, these commands help:
+
+<div style="background: #fff5f5; padding: 1.5rem; border: 1px solid #fed7d7; border-radius: 0.5rem; margin: 1rem 0;">
+
+**Environment Issues:**
+```bash
+tito system doctor          # Diagnose problems
+tito system info           # Show configuration details
+```
+
+**Module Problems:**
+```bash
+tito module status         # Check what's available
+tito module info 02_tensor # Get specific module details
+```
+
+**Progress Confusion:**
+```bash
+tito checkpoint status --detailed    # See exactly where you are
+tito checkpoint timeline            # Visualize your progress
+```
+
+</div>
+
+## 🎯 Pro Tips for Efficiency
+
+<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(250px, 1fr)); gap: 1rem; margin: 2rem 0;">
+
+<div style="background: #e6fffa; padding: 1rem; border-radius: 0.5rem; border-left: 3px solid #26d0ce;">
+<h4 style="margin: 0 0 0.5rem 0; color: #0d9488;">🔥 Hot Tip</h4>
+<p style="margin: 0; font-size: 0.9rem;">Use tab completion! Type `tito mod` + TAB to auto-complete commands</p>
+</div>
+
+<div style="background: #f0f9ff; padding: 1rem; border-radius: 0.5rem; border-left: 3px solid #3b82f6;">
+<h4 style="margin: 0 0 0.5rem 0; color: #1d4ed8;">⚡ Speed Boost</h4>
+<p style="margin: 0; font-size: 0.9rem;">Alias common commands: `alias ts='tito checkpoint status'`</p>
+</div>
+
+<div style="background: #fefce8; padding: 1rem; border-radius: 0.5rem; border-left: 3px solid #eab308;">
+<h4 style="margin: 0 0 0.5rem 0; color: #a16207;">🎯 Focus</h4>
+<p style="margin: 0; font-size: 0.9rem;">Always run `tito system doctor` first when starting a new session</p>
+</div>
+
+</div>
+
+## 🚀 Ready to Build?
+
+<div style="background: #f8f9fa; padding: 2rem; border-radius: 0.5rem; margin: 2rem 0; text-align: center;">
+<h3 style="margin: 0 0 1rem 0; color: #495057;">Start Your TinyTorch Journey</h3>
+<p style="margin: 0 0 1.5rem 0; color: #6c757d;">Follow the 2-minute setup and begin building ML systems from scratch</p>
+<a href="quickstart-guide.html" style="display: inline-block; background: #007bff; color: white; padding: 0.75rem 1.5rem; border-radius: 0.25rem; text-decoration: none; font-weight: 500; margin-right: 1rem;">2-Minute Setup →</a>
+<a href="learning-progress.html" style="display: inline-block; background: #28a745; color: white; padding: 0.75rem 1.5rem; border-radius: 0.25rem; text-decoration: none; font-weight: 500;">Track Progress →</a>
+</div>
+
+---
+
+*Master these commands and you'll build ML systems with confidence. Every command is designed to accelerate your learning and keep you focused on what matters: building production-quality ML frameworks from scratch.*

site/usage-paths/classroom-use.md

@@ -0,0 +1,228 @@
+# TinyTorch for Instructors: Complete ML Systems Course
+
+<div style="background: #e3f2fd; border: 1px solid #2196f3; padding: 1rem; border-radius: 0.5rem; margin: 1rem 0;">
+<strong>📖 Course Overview & Benefits:</strong> This page explains WHAT TinyTorch offers for ML education and WHY it's effective.<br>
+<strong>📖 For Setup & Daily Workflow:</strong> See <a href="../instructor-guide.html">Technical Instructor Guide</a> for step-by-step NBGrader setup and semester management.
+</div>
+
+<div style="background: #f8f9fa; border: 1px solid #dee2e6; padding: 2rem; border-radius: 0.5rem; text-align: center; margin: 2rem 0;">
+<h2 style="margin: 0 0 1rem 0; color: #495057;">🏫 Turn-Key ML Systems Education</h2>
+<p style="font-size: 1.1rem; margin: 0; color: #6c757d;">Transform students from framework users to systems engineers</p>
+</div>
+
+**Transform Your ML Teaching:** Replace black-box API courses with deep systems understanding. Your students will build neural networks from scratch, understand every operation, and graduate job-ready for ML engineering roles.
+
+---
+
+## 🎯 Complete Course Infrastructure
+
+<div style="background: #f8f9fa; border-left: 4px solid #007bff; padding: 1.5rem; margin: 1.5rem 0;">
+<h4 style="margin: 0 0 1rem 0; color: #0056b3;">What You Get: Production-Ready Course Materials</h4>
+<div style="display: grid; grid-template-columns: 1fr 1fr; gap: 1rem;">
+<div>
+<ul style="margin: 0; padding-left: 1rem;">
+<li><strong>Three-tier progression</strong> (20 modules) with NBGrader integration</li>
+<li><strong>200+ automated tests</strong> for immediate feedback</li>
+<li><strong>Professional CLI tools</strong> for development workflow</li>
+<li><strong>Real datasets</strong> (CIFAR-10, text generation)</li>
+</ul>
+</div>
+<div>
+<ul style="margin: 0; padding-left: 1rem;">
+<li><strong>Complete instructor guide</strong> with setup & grading</li>
+<li><strong>Flexible pacing</strong> (8-20 weeks depending on depth)</li>
+<li><strong>Industry practices</strong> (Git, testing, documentation)</li>
+<li><strong>Academic foundation</strong> from university research</li>
+</ul>
+</div>
+</div>
+</div>
+
+**Course Duration:** 14-16 weeks (flexible pacing)  
+**Student Outcome:** Complete ML framework supporting vision AND language models
+
+```{admonition} Complete Instructor Documentation
+:class: tip
+**See our comprehensive [Instructor Guide](../instructor-guide.md)** for:
+- Complete setup walkthrough (30 minutes)
+- Weekly assignment workflow with NBGrader
+- Grading automation and feedback generation
+- Student support and troubleshooting
+- End-to-end course management
+- Quick reference commands
+```
+
+---
+
+## 🌟 Why TinyTorch for Your Classroom
+
+<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(300px, 1fr)); gap: 1.5rem; margin: 2rem 0;">
+
+<div style="background: #e8f5e8; padding: 1.5rem; border-radius: 0.5rem; border-left: 4px solid #4caf50;">
+<h4 style="margin: 0 0 1rem 0; color: #2e7d32;">🎯 Deep Learning Outcomes</h4>
+<p style="margin: 0 0 0.5rem 0; font-weight: 600;">Students build neural networks from scratch</p>
+<ul style="margin: 0; font-size: 0.9rem; color: #64748b;">
+<li>Graduates understand deep systems architecture</li>
+<li>Can debug ML issues from first principles</li>
+<li>Prepared for ML engineering roles</li>
+<li>Confident implementing novel architectures</li>
+</ul>
+</div>
+
+<div style="background: #fff3e0; padding: 1.5rem; border-radius: 0.5rem; border-left: 4px solid #ff9800;">
+<h4 style="margin: 0 0 1rem 0; color: #f57c00;">⚡ Zero-Setup Teaching</h4>
+<p style="margin: 0 0 0.5rem 0; font-weight: 600;">30-minute instructor setup, then focus on teaching</p>
+<ul style="margin: 0; font-size: 0.9rem; color: #64748b;">
+<li><strong>NBGrader integration</strong>: Automated grading & feedback</li>
+<li><strong>One-command workflows</strong>: Generate, release, collect assignments</li>
+<li><strong>Progress dashboards</strong>: Track all students at a glance</li>
+<li><strong>Flexible pacing</strong>: Adapt to your semester schedule</li>
+</ul>
+</div>
+
+<div style="background: #f3e5f5; padding: 1.5rem; border-radius: 0.5rem; border-left: 4px solid #9c27b0;">
+<h4 style="margin: 0 0 1rem 0; color: #7b1fa2;">🏆 Industry-Standard Workflow</h4>
+<p style="margin: 0 0 0.5rem 0; font-weight: 600;">Students learn professional ML engineering practices</p>
+<ul style="margin: 0; font-size: 0.9rem; color: #64748b;">
+<li><strong>Git workflow</strong>: Feature branches, commits, merges</li>
+<li><strong>CLI tools</strong>: Professional development environment</li>
+<li><strong>Testing culture</strong>: Every implementation immediately validated</li>
+<li><strong>Documentation</strong>: Clear code, explanations, insights</li>
+</ul>
+</div>
+
+<div style="background: #e1f5fe; padding: 1.5rem; border-radius: 0.5rem; border-left: 4px solid #03a9f4;">
+<h4 style="margin: 0 0 1rem 0; color: #0277bd;">🔬 Deep Systems Understanding</h4>
+<p style="margin: 0 0 0.5rem 0; font-weight: 600;">Beyond APIs: Students understand how ML really works</p>
+<ul style="margin: 0; font-size: 0.9rem; color: #64748b;">
+<li><strong>Memory analysis</strong>: Profile and optimize resource usage</li>
+<li><strong>Performance insights</strong>: Understand computational complexity</li>
+<li><strong>Production context</strong>: How PyTorch/TensorFlow actually work</li>
+<li><strong>Systems thinking</strong>: Architecture, scaling, optimization</li>
+</ul>
+</div>
+
+</div>
+
+---
+
+## Course Module Overview
+
+The TinyTorch course consists of 20 progressive modules organized into learning stages.
+
+**📖 See [Complete Course Structure](../chapters/00-introduction.html#course-structure)** for detailed module descriptions, learning objectives, and prerequisites for each module.
+
+---
+
+## Academic Learning Goals
+
+**What Students Will Achieve:**
+- Build deep systems understanding through implementation
+- Bridge gap between ML theory and engineering practice
+- Prepare for real-world ML systems challenges
+- Enable research into novel architectures and optimizations
+
+**Core Capabilities Developed:**
+- Implement neural networks from scratch
+- Understand autograd and backpropagation deeply
+- Optimize models for production deployment
+- Build complete frameworks supporting vision and language
+
+---
+
+## 🚀 Quick Start for Instructors
+
+<div style="background: #f8f9fa; border: 1px solid #dee2e6; border-radius: 0.5rem; padding: 2rem; margin: 2rem 0;">
+<h3 style="margin: 0 0 1rem 0; text-align: center; color: #495057;">⏱️ 30 Minutes to Teaching-Ready Course</h3>
+<p style="text-align: center; margin: 0 0 1.5rem 0; color: #6c757d;">Three simple steps to transform your ML teaching</p>
+
+<div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(250px, 1fr)); gap: 1.5rem;">
+
+<div style="background: white; padding: 1.5rem; border-radius: 0.5rem; border: 1px solid #dee2e6;">
+<h4 style="color: #495057; margin: 0 0 0.5rem 0;">1️⃣ Clone & Setup (10 min)</h4>
+<div style="background: #f8f9fa; padding: 1rem; border-radius: 0.25rem; font-family: monospace; font-size: 0.85rem; margin: 0.5rem 0;">
+git clone TinyTorch<br>
+cd TinyTorch<br>
+source .venv/bin/activate<br>
+pip install -r requirements.txt
+</div>
+<p style="font-size: 0.9rem; margin: 0; color: #6c757d;">One-time environment setup</p>
+</div>
+
+<div style="background: white; padding: 1.5rem; border-radius: 0.5rem; border: 1px solid #dee2e6;">
+<h4 style="color: #495057; margin: 0 0 0.5rem 0;">2️⃣ Initialize Course (10 min)</h4>
+<div style="background: #f8f9fa; padding: 1rem; border-radius: 0.25rem; font-family: monospace; font-size: 0.85rem; margin: 0.5rem 0;">
+tito nbgrader init<br>
+tito module status --comprehensive
+</div>
+<p style="font-size: 0.9rem; margin: 0; color: #6c757d;">NBGrader integration & health check</p>
+</div>
+
+<div style="background: white; padding: 1.5rem; border-radius: 0.5rem; border: 1px solid #dee2e6;">
+<h4 style="color: #495057; margin: 0 0 0.5rem 0;">3️⃣ First Assignment (10 min)</h4>
+<div style="background: #f8f9fa; padding: 1rem; border-radius: 0.25rem; font-family: monospace; font-size: 0.85rem; margin: 0.5rem 0;">
+tito nbgrader generate 01_setup<br>
+tito nbgrader release 01_setup
+</div>
+<p style="font-size: 0.9rem; margin: 0; color: #6c757d;">Ready to distribute to students!</p>
+</div>
+
+</div>
+
+<div style="text-align: center; margin-top: 1.5rem;">
+<a href="../instructor-guide.html" style="display: inline-block; background: #007bff; color: white; padding: 0.5rem 1rem; border-radius: 0.25rem; text-decoration: none; font-weight: 500; margin-right: 1rem;">📖 Complete Instructor Guide</a>
+<a href="../testing-framework.html" style="display: inline-block; background: #28a745; color: white; padding: 0.5rem 1rem; border-radius: 0.25rem; text-decoration: none; font-weight: 500;">🧪 Testing Framework Guide</a>
+</div>
+
+</div>
+
+---
+
+## 📋 Assessment Options
+
+### Automated Grading
+- NBGrader integration for all modules
+- Automatic test execution and scoring
+- Detailed feedback generation
+
+### Flexible Point Distribution
+- Customize weights per module
+- Add bonus challenges
+- Include participation components
+
+### Project-Based Assessment
+- Combine modules into larger projects
+- Capstone project for final evaluation
+- Portfolio development opportunities
+
+---
+
+## Instructor Resources
+
+### Documentation
+- [Complete Instructor Guide](../instructor-guide.md) - Detailed setup and workflow
+- [Quick Reference Card](../../NBGrader_Quick_Reference.md) - Essential commands
+- Module-specific teaching notes in each chapter
+
+### Support Tools
+- `tito module status --comprehensive` - System health dashboard
+- `tito nbgrader status` - Assignment tracking
+- `tito nbgrader report` - Grade export
+
+### Community
+- GitHub Issues for technical support
+- Instructor discussion forum (coming soon)
+- Regular updates and improvements
+
+---
+
+## 📞 Next Steps
+
+1. **📖 Read the [Instructor Guide](../instructor-guide.md)** for complete details
+2. **🚀 Start with Module 0: [Introduction](../chapters/00-introduction.md)** to see the system overview
+3. **💻 Set up your environment** following the guide
+4. **📧 Contact us** for instructor support
+
+---
+
+*Ready to teach the most comprehensive ML systems course? Let's build something amazing together!* 🎓