mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-05-08 08:33:03 -05:00
f9309e8b9d
📦 Module File Organization: - Renamed networks_dev.py → dense_dev.py in 05_dense module - Renamed cnn_dev.py → spatial_dev.py in 06_spatial module - Added new 07_attention module with attention_dev.py - Updated module.yaml files to reference correct filenames - Updated #| default_exp directives for proper package exports 🔄 Core Package Updates: - Added tinytorch.core.dense (Sequential, MLP architectures) - Added tinytorch.core.spatial (Conv2D, pooling operations) - Added tinytorch.core.attention (self-attention mechanisms) - Updated all core modules with latest implementations - Fixed tensor assignment issues in compression module 🧪 Test Integration Fixes: - Updated integration tests to use correct module imports - Fixed tensor activation tests for new module structure - Ensured compatibility with renamed components - Maintained 100% individual module test success rate Result: Complete 14-module TinyTorch framework with proper organization, working integrations, and comprehensive test coverage ready for production use.
1107 lines
42 KiB
Plaintext
1107 lines
42 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0eb7442f",
|
||
"metadata": {
|
||
"cell_marker": "\"\"\""
|
||
},
|
||
"source": [
|
||
"# CNN - Convolutional Neural Networks\n",
|
||
"\n",
|
||
"Welcome to the CNN module! Here you'll implement the core building block of modern computer vision: the convolutional layer.\n",
|
||
"\n",
|
||
"## Learning Goals\n",
|
||
"- Understand the convolution operation and its importance in computer vision\n",
|
||
"- Implement Conv2D with explicit for-loops to understand the sliding window mechanism\n",
|
||
"- Build convolutional layers that can detect spatial patterns in images\n",
|
||
"- Compose Conv2D with other layers to build complete convolutional networks\n",
|
||
"- See how convolution enables parameter sharing and translation invariance\n",
|
||
"\n",
|
||
"## Build → Use → Reflect\n",
|
||
"1. **Build**: Conv2D layer using sliding window convolution from scratch\n",
|
||
"2. **Use**: Transform images and see feature maps emerge\n",
|
||
"3. **Reflect**: How CNNs learn hierarchical spatial patterns\n",
|
||
"\n",
|
||
"## What You'll Learn\n",
|
||
"By the end of this module, you'll understand:\n",
|
||
"- How convolution works as a sliding window operation\n",
|
||
"- Why convolution is perfect for spatial data like images\n",
|
||
"- How to build learnable convolutional layers\n",
|
||
"- The CNN pipeline: Conv2D → Activation → Flatten → Dense\n",
|
||
"- How parameter sharing makes CNNs efficient"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "dcbef292",
|
||
"metadata": {
|
||
"lines_to_next_cell": 1,
|
||
"nbgrader": {
|
||
"grade": false,
|
||
"grade_id": "cnn-imports",
|
||
"locked": false,
|
||
"schema_version": 3,
|
||
"solution": false,
|
||
"task": false
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"#| default_exp core.spatial\n",
|
||
"\n",
|
||
"#| export\n",
|
||
"import numpy as np\n",
|
||
"import os\n",
|
||
"import sys\n",
|
||
"from typing import List, Tuple, Optional\n",
|
||
"import matplotlib.pyplot as plt\n",
|
||
"\n",
|
||
"# Import from the main package - try package first, then local modules\n",
|
||
"try:\n",
|
||
" from tinytorch.core.tensor import Tensor\n",
|
||
" from tinytorch.core.layers import Dense\n",
|
||
" from tinytorch.core.activations import ReLU\n",
|
||
"except ImportError:\n",
|
||
" # For development, import from local modules\n",
|
||
" sys.path.append(os.path.join(os.path.dirname(__file__), '..', '01_tensor'))\n",
|
||
" sys.path.append(os.path.join(os.path.dirname(__file__), '..', '02_activations'))\n",
|
||
" sys.path.append(os.path.join(os.path.dirname(__file__), '..', '03_layers'))\n",
|
||
" from tensor_dev import Tensor\n",
|
||
" from activations_dev import ReLU\n",
|
||
" from layers_dev import Dense"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "708b859a",
|
||
"metadata": {
|
||
"lines_to_next_cell": 1,
|
||
"nbgrader": {
|
||
"grade": false,
|
||
"grade_id": "cnn-setup",
|
||
"locked": false,
|
||
"schema_version": 3,
|
||
"solution": false,
|
||
"task": false
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"#| hide\n",
|
||
"#| export\n",
|
||
"def _should_show_plots():\n",
|
||
" \"\"\"Check if we should show plots (disable during testing)\"\"\"\n",
|
||
" # Check multiple conditions that indicate we're in test mode\n",
|
||
" is_pytest = (\n",
|
||
" 'pytest' in sys.modules or\n",
|
||
" 'test' in sys.argv or\n",
|
||
" os.environ.get('PYTEST_CURRENT_TEST') is not None or\n",
|
||
" any('test' in arg for arg in sys.argv) or\n",
|
||
" any('pytest' in arg for arg in sys.argv)\n",
|
||
" )\n",
|
||
" \n",
|
||
" # Show plots in development mode (when not in test mode)\n",
|
||
" return not is_pytest"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "afb3e12a",
|
||
"metadata": {
|
||
"nbgrader": {
|
||
"grade": false,
|
||
"grade_id": "cnn-welcome",
|
||
"locked": false,
|
||
"schema_version": 3,
|
||
"solution": false,
|
||
"task": false
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"print(\"🔥 TinyTorch CNN Module\")\n",
|
||
"print(f\"NumPy version: {np.__version__}\")\n",
|
||
"print(f\"Python version: {sys.version_info.major}.{sys.version_info.minor}\")\n",
|
||
"print(\"Ready to build convolutional neural networks!\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0c0fae33",
|
||
"metadata": {
|
||
"cell_marker": "\"\"\""
|
||
},
|
||
"source": [
|
||
"## 📦 Where This Code Lives in the Final Package\n",
|
||
"\n",
|
||
"**Learning Side:** You work in `modules/source/05_cnn/cnn_dev.py` \n",
|
||
"**Building Side:** Code exports to `tinytorch.core.cnn`\n",
|
||
"\n",
|
||
"```python\n",
|
||
"# Final package structure:\n",
|
||
"from tinytorch.core.cnn import Conv2D, conv2d_naive, flatten # CNN operations!\n",
|
||
"from tinytorch.core.layers import Dense # Fully connected layers\n",
|
||
"from tinytorch.core.activations import ReLU # Nonlinearity\n",
|
||
"from tinytorch.core.tensor import Tensor # Foundation\n",
|
||
"```\n",
|
||
"\n",
|
||
"**Why this matters:**\n",
|
||
"- **Learning:** Focused modules for deep understanding of convolution\n",
|
||
"- **Production:** Proper organization like PyTorch's `torch.nn.Conv2d`\n",
|
||
"- **Consistency:** All CNN operations live together in `core.cnn`\n",
|
||
"- **Integration:** Works seamlessly with other TinyTorch components"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "3f3d5bdd",
|
||
"metadata": {
|
||
"cell_marker": "\"\"\"",
|
||
"lines_to_next_cell": 1
|
||
},
|
||
"source": [
|
||
"## Step 1: Understanding Convolution\n",
|
||
"\n",
|
||
"### What is Convolution?\n",
|
||
"**Convolution** is a mathematical operation that slides a small filter (kernel) across an input, computing dot products at each position.\n",
|
||
"\n",
|
||
"### Why Convolution is Perfect for Images\n",
|
||
"- **Local patterns**: Images have local structure (edges, textures)\n",
|
||
"- **Translation invariance**: Same pattern can appear anywhere\n",
|
||
"- **Parameter sharing**: One filter detects the pattern everywhere\n",
|
||
"- **Spatial hierarchy**: Multiple layers build increasingly complex features\n",
|
||
"\n",
|
||
"### The Fundamental Insight\n",
|
||
"**Convolution is pattern matching!** The kernel learns to detect specific patterns:\n",
|
||
"- **Edge detectors**: Find boundaries between objects\n",
|
||
"- **Texture detectors**: Recognize surface patterns\n",
|
||
"- **Shape detectors**: Identify geometric forms\n",
|
||
"- **Feature detectors**: Combine simple patterns into complex features\n",
|
||
"\n",
|
||
"### Real-World Applications\n",
|
||
"- **Image processing**: Detect edges, blur, sharpen\n",
|
||
"- **Computer vision**: Recognize objects, faces, text\n",
|
||
"- **Medical imaging**: Detect tumors, analyze scans\n",
|
||
"- **Autonomous driving**: Identify traffic signs, pedestrians\n",
|
||
"\n",
|
||
"### Visual Intuition\n",
|
||
"```\n",
|
||
"Input Image: Kernel: Output Feature Map:\n",
|
||
"[1, 2, 3] [1, 0] [1*1+2*0+4*0+5*(-1), 2*1+3*0+5*0+6*(-1)]\n",
|
||
"[4, 5, 6] [0, -1] [4*1+5*0+7*0+8*(-1), 5*1+6*0+8*0+9*(-1)]\n",
|
||
"[7, 8, 9]\n",
|
||
"```\n",
|
||
"\n",
|
||
"The kernel slides across the input, computing dot products at each position.\n",
|
||
"\n",
|
||
"Let's implement this step by step!"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "2e7367d3",
|
||
"metadata": {
|
||
"lines_to_next_cell": 1,
|
||
"nbgrader": {
|
||
"grade": false,
|
||
"grade_id": "conv2d-naive",
|
||
"locked": false,
|
||
"schema_version": 3,
|
||
"solution": true,
|
||
"task": false
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"#| export\n",
|
||
"def conv2d_naive(input: np.ndarray, kernel: np.ndarray) -> np.ndarray:\n",
|
||
" \"\"\"\n",
|
||
" Naive 2D convolution (single channel, no stride, no padding).\n",
|
||
" \n",
|
||
" Args:\n",
|
||
" input: 2D input array (H, W)\n",
|
||
" kernel: 2D filter (kH, kW)\n",
|
||
" Returns:\n",
|
||
" 2D output array (H-kH+1, W-kW+1)\n",
|
||
" \n",
|
||
" TODO: Implement the sliding window convolution using for-loops.\n",
|
||
" \n",
|
||
" APPROACH:\n",
|
||
" 1. Get input dimensions: H, W = input.shape\n",
|
||
" 2. Get kernel dimensions: kH, kW = kernel.shape\n",
|
||
" 3. Calculate output dimensions: out_H = H - kH + 1, out_W = W - kW + 1\n",
|
||
" 4. Create output array: np.zeros((out_H, out_W))\n",
|
||
" 5. Use nested loops to slide the kernel:\n",
|
||
" - i loop: output rows (0 to out_H-1)\n",
|
||
" - j loop: output columns (0 to out_W-1)\n",
|
||
" - di loop: kernel rows (0 to kH-1)\n",
|
||
" - dj loop: kernel columns (0 to kW-1)\n",
|
||
" 6. For each (i,j), compute: output[i,j] += input[i+di, j+dj] * kernel[di, dj]\n",
|
||
" \n",
|
||
" EXAMPLE:\n",
|
||
" Input: [[1, 2, 3], Kernel: [[1, 0],\n",
|
||
" [4, 5, 6], [0, -1]]\n",
|
||
" [7, 8, 9]]\n",
|
||
" \n",
|
||
" Output[0,0] = 1*1 + 2*0 + 4*0 + 5*(-1) = 1 - 5 = -4\n",
|
||
" Output[0,1] = 2*1 + 3*0 + 5*0 + 6*(-1) = 2 - 6 = -4\n",
|
||
" Output[1,0] = 4*1 + 5*0 + 7*0 + 8*(-1) = 4 - 8 = -4\n",
|
||
" Output[1,1] = 5*1 + 6*0 + 8*0 + 9*(-1) = 5 - 9 = -4\n",
|
||
" \n",
|
||
" HINTS:\n",
|
||
" - Start with output = np.zeros((out_H, out_W))\n",
|
||
" - Use four nested loops: for i in range(out_H): for j in range(out_W): for di in range(kH): for dj in range(kW):\n",
|
||
" - Accumulate the sum: output[i,j] += input[i+di, j+dj] * kernel[di, dj]\n",
|
||
" \"\"\"\n",
|
||
" ### BEGIN SOLUTION\n",
|
||
" # Get input and kernel dimensions\n",
|
||
" H, W = input.shape\n",
|
||
" kH, kW = kernel.shape\n",
|
||
" \n",
|
||
" # Calculate output dimensions\n",
|
||
" out_H, out_W = H - kH + 1, W - kW + 1\n",
|
||
" \n",
|
||
" # Initialize output array\n",
|
||
" output = np.zeros((out_H, out_W), dtype=input.dtype)\n",
|
||
" \n",
|
||
" # Sliding window convolution with four nested loops\n",
|
||
" for i in range(out_H):\n",
|
||
" for j in range(out_W):\n",
|
||
" for di in range(kH):\n",
|
||
" for dj in range(kW):\n",
|
||
" output[i, j] += input[i + di, j + dj] * kernel[di, dj]\n",
|
||
" \n",
|
||
" return output\n",
|
||
" ### END SOLUTION"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f864dd60",
|
||
"metadata": {
|
||
"cell_marker": "\"\"\""
|
||
},
|
||
"source": [
|
||
"### 🧪 Unit Test: Convolution Operation\n",
|
||
"\n",
|
||
"Let's test your convolution implementation right away! This is the core operation that powers computer vision.\n",
|
||
"\n",
|
||
"**This is a unit test** - it tests one specific function (conv2d_naive) in isolation."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "0a397c9b",
|
||
"metadata": {
|
||
"nbgrader": {
|
||
"grade": true,
|
||
"grade_id": "test-conv2d-naive-immediate",
|
||
"locked": true,
|
||
"points": 10,
|
||
"schema_version": 3,
|
||
"solution": false,
|
||
"task": false
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Test conv2d_naive function immediately after implementation\n",
|
||
"print(\"🔬 Unit Test: Convolution Operation...\")\n",
|
||
"\n",
|
||
"# Test simple 3x3 input with 2x2 kernel\n",
|
||
"try:\n",
|
||
" input_array = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]], dtype=np.float32)\n",
|
||
" kernel_array = np.array([[1, 0], [0, 1]], dtype=np.float32) # Identity-like kernel\n",
|
||
" \n",
|
||
" result = conv2d_naive(input_array, kernel_array)\n",
|
||
" expected = np.array([[6, 8], [12, 14]], dtype=np.float32) # 1+5, 2+6, 4+8, 5+9\n",
|
||
" \n",
|
||
" print(f\"Input:\\n{input_array}\")\n",
|
||
" print(f\"Kernel:\\n{kernel_array}\")\n",
|
||
" print(f\"Result:\\n{result}\")\n",
|
||
" print(f\"Expected:\\n{expected}\")\n",
|
||
" \n",
|
||
" assert np.allclose(result, expected), f\"Convolution failed: expected {expected}, got {result}\"\n",
|
||
" print(\"✅ Simple convolution test passed\")\n",
|
||
" \n",
|
||
"except Exception as e:\n",
|
||
" print(f\"❌ Simple convolution test failed: {e}\")\n",
|
||
" raise\n",
|
||
"\n",
|
||
"# Test edge detection kernel\n",
|
||
"try:\n",
|
||
" input_array = np.array([[1, 1, 1], [1, 1, 1], [1, 1, 1]], dtype=np.float32)\n",
|
||
" edge_kernel = np.array([[-1, -1], [-1, 3]], dtype=np.float32) # Edge detection\n",
|
||
" \n",
|
||
" result = conv2d_naive(input_array, edge_kernel)\n",
|
||
" expected = np.array([[0, 0], [0, 0]], dtype=np.float32) # Uniform region = no edges\n",
|
||
" \n",
|
||
" assert np.allclose(result, expected), f\"Edge detection failed: expected {expected}, got {result}\"\n",
|
||
" print(\"✅ Edge detection test passed\")\n",
|
||
" \n",
|
||
"except Exception as e:\n",
|
||
" print(f\"❌ Edge detection test failed: {e}\")\n",
|
||
" raise\n",
|
||
"\n",
|
||
"# Test output shape\n",
|
||
"try:\n",
|
||
" input_5x5 = np.random.randn(5, 5).astype(np.float32)\n",
|
||
" kernel_3x3 = np.random.randn(3, 3).astype(np.float32)\n",
|
||
" \n",
|
||
" result = conv2d_naive(input_5x5, kernel_3x3)\n",
|
||
" expected_shape = (3, 3) # 5-3+1 = 3\n",
|
||
" \n",
|
||
" assert result.shape == expected_shape, f\"Output shape wrong: expected {expected_shape}, got {result.shape}\"\n",
|
||
" print(\"✅ Output shape test passed\")\n",
|
||
" \n",
|
||
"except Exception as e:\n",
|
||
" print(f\"❌ Output shape test failed: {e}\")\n",
|
||
" raise\n",
|
||
"\n",
|
||
"# Show the convolution process\n",
|
||
"print(\"🎯 Convolution behavior:\")\n",
|
||
"print(\" Slides kernel across input\")\n",
|
||
"print(\" Computes dot product at each position\")\n",
|
||
"print(\" Output size = Input size - Kernel size + 1\")\n",
|
||
"print(\"📈 Progress: Convolution operation ✓\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "21785fd9",
|
||
"metadata": {
|
||
"cell_marker": "\"\"\"",
|
||
"lines_to_next_cell": 1
|
||
},
|
||
"source": [
|
||
"## Step 2: Building the Conv2D Layer\n",
|
||
"\n",
|
||
"### What is a Conv2D Layer?\n",
|
||
"A **Conv2D layer** is a learnable convolutional layer that:\n",
|
||
"- Has learnable kernel weights (initialized randomly)\n",
|
||
"- Applies convolution to input tensors\n",
|
||
"- Integrates with the rest of the neural network\n",
|
||
"\n",
|
||
"### Why Conv2D Layers Matter\n",
|
||
"- **Feature learning**: Kernels learn to detect useful patterns\n",
|
||
"- **Composability**: Can be stacked with other layers\n",
|
||
"- **Efficiency**: Shared weights reduce parameters dramatically\n",
|
||
"- **Translation invariance**: Same patterns detected anywhere in the image\n",
|
||
"\n",
|
||
"### Real-World Applications\n",
|
||
"- **Image classification**: Recognize objects in photos\n",
|
||
"- **Object detection**: Find and locate objects\n",
|
||
"- **Medical imaging**: Detect anomalies in scans\n",
|
||
"- **Autonomous driving**: Identify road features\n",
|
||
"\n",
|
||
"### Design Decisions\n",
|
||
"- **Kernel size**: Typically 3×3 or 5×5 for balance of locality and capacity\n",
|
||
"- **Initialization**: Small random values to break symmetry\n",
|
||
"- **Integration**: Works with Tensor class and other layers"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "1419fa91",
|
||
"metadata": {
|
||
"lines_to_next_cell": 1,
|
||
"nbgrader": {
|
||
"grade": false,
|
||
"grade_id": "conv2d-class",
|
||
"locked": false,
|
||
"schema_version": 3,
|
||
"solution": true,
|
||
"task": false
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"#| export\n",
|
||
"class Conv2D:\n",
|
||
" \"\"\"\n",
|
||
" 2D Convolutional Layer (single channel, single filter, no stride/pad).\n",
|
||
" \n",
|
||
" A learnable convolutional layer that applies a kernel to detect spatial patterns.\n",
|
||
" Perfect for building the foundation of convolutional neural networks.\n",
|
||
" \"\"\"\n",
|
||
" \n",
|
||
" def __init__(self, kernel_size: Tuple[int, int]):\n",
|
||
" \"\"\"\n",
|
||
" Initialize Conv2D layer with random kernel.\n",
|
||
" \n",
|
||
" Args:\n",
|
||
" kernel_size: (kH, kW) - size of the convolution kernel\n",
|
||
" \n",
|
||
" TODO: Initialize a random kernel with small values.\n",
|
||
" \n",
|
||
" APPROACH:\n",
|
||
" 1. Store kernel_size as instance variable\n",
|
||
" 2. Initialize random kernel with small values\n",
|
||
" 3. Use proper initialization for stable training\n",
|
||
" \n",
|
||
" EXAMPLE:\n",
|
||
" Conv2D((2, 2)) creates:\n",
|
||
" - kernel: shape (2, 2) with small random values\n",
|
||
" \n",
|
||
" HINTS:\n",
|
||
" - Store kernel_size as self.kernel_size\n",
|
||
" - Initialize kernel: np.random.randn(kH, kW) * 0.1 (small values)\n",
|
||
" - Convert to float32 for consistency\n",
|
||
" \"\"\"\n",
|
||
" ### BEGIN SOLUTION\n",
|
||
" # Store kernel size\n",
|
||
" self.kernel_size = kernel_size\n",
|
||
" kH, kW = kernel_size\n",
|
||
" \n",
|
||
" # Initialize random kernel with small values\n",
|
||
" self.kernel = np.random.randn(kH, kW).astype(np.float32) * 0.1\n",
|
||
" ### END SOLUTION\n",
|
||
" \n",
|
||
" def forward(self, x):\n",
|
||
" \"\"\"\n",
|
||
" Forward pass: apply convolution to input tensor.\n",
|
||
" \n",
|
||
" Args:\n",
|
||
" x: Input tensor (2D for simplicity)\n",
|
||
" \n",
|
||
" Returns:\n",
|
||
" Output tensor after convolution\n",
|
||
" \n",
|
||
" TODO: Implement forward pass using conv2d_naive function.\n",
|
||
" \n",
|
||
" APPROACH:\n",
|
||
" 1. Extract numpy array from input tensor\n",
|
||
" 2. Apply conv2d_naive with stored kernel\n",
|
||
" 3. Return result wrapped in Tensor\n",
|
||
" \n",
|
||
" EXAMPLE:\n",
|
||
" x = Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) # shape (3, 3)\n",
|
||
" layer = Conv2D((2, 2))\n",
|
||
" y = layer(x) # shape (2, 2)\n",
|
||
" \n",
|
||
" HINTS:\n",
|
||
" - Use x.data to get numpy array\n",
|
||
" - Use conv2d_naive(x.data, self.kernel)\n",
|
||
" - Return Tensor(result) to wrap the result\n",
|
||
" \"\"\"\n",
|
||
" ### BEGIN SOLUTION\n",
|
||
" # Apply convolution using naive implementation\n",
|
||
" result = conv2d_naive(x.data, self.kernel)\n",
|
||
" return type(x)(result)\n",
|
||
" ### END SOLUTION\n",
|
||
" \n",
|
||
" def __call__(self, x):\n",
|
||
" \"\"\"Make layer callable: layer(x) same as layer.forward(x)\"\"\"\n",
|
||
" return self.forward(x)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "bcbe5521",
|
||
"metadata": {
|
||
"cell_marker": "\"\"\""
|
||
},
|
||
"source": [
|
||
"### 🧪 Unit Test: Conv2D Layer\n",
|
||
"\n",
|
||
"Let's test your Conv2D layer implementation! This is a learnable convolutional layer that can be trained.\n",
|
||
"\n",
|
||
"**This is a unit test** - it tests one specific class (Conv2D) in isolation."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "dc785267",
|
||
"metadata": {
|
||
"nbgrader": {
|
||
"grade": true,
|
||
"grade_id": "test-conv2d-layer-immediate",
|
||
"locked": true,
|
||
"points": 10,
|
||
"schema_version": 3,
|
||
"solution": false,
|
||
"task": false
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Test Conv2D layer immediately after implementation\n",
|
||
"print(\"🔬 Unit Test: Conv2D Layer...\")\n",
|
||
"\n",
|
||
"# Create a Conv2D layer\n",
|
||
"try:\n",
|
||
" layer = Conv2D(kernel_size=(2, 2))\n",
|
||
" print(f\"Conv2D layer created with kernel size: {layer.kernel_size}\")\n",
|
||
" print(f\"Kernel shape: {layer.kernel.shape}\")\n",
|
||
" \n",
|
||
" # Test that kernel is initialized properly\n",
|
||
" assert layer.kernel.shape == (2, 2), f\"Kernel shape should be (2, 2), got {layer.kernel.shape}\"\n",
|
||
" assert not np.allclose(layer.kernel, 0), \"Kernel should not be all zeros\"\n",
|
||
" print(\"✅ Conv2D layer initialization successful\")\n",
|
||
" \n",
|
||
" # Test with sample input\n",
|
||
" x = Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
|
||
" print(f\"Input shape: {x.shape}\")\n",
|
||
" \n",
|
||
" y = layer(x)\n",
|
||
" print(f\"Output shape: {y.shape}\")\n",
|
||
" print(f\"Output: {y}\")\n",
|
||
" \n",
|
||
" # Verify shapes\n",
|
||
" assert y.shape == (2, 2), f\"Output shape should be (2, 2), got {y.shape}\"\n",
|
||
" assert isinstance(y, Tensor), \"Output should be a Tensor\"\n",
|
||
" print(\"✅ Conv2D layer forward pass successful\")\n",
|
||
" \n",
|
||
"except Exception as e:\n",
|
||
" print(f\"❌ Conv2D layer test failed: {e}\")\n",
|
||
" raise\n",
|
||
"\n",
|
||
"# Test different kernel sizes\n",
|
||
"try:\n",
|
||
" layer_3x3 = Conv2D(kernel_size=(3, 3))\n",
|
||
" x_5x5 = Tensor([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15], [16, 17, 18, 19, 20], [21, 22, 23, 24, 25]])\n",
|
||
" y_3x3 = layer_3x3(x_5x5)\n",
|
||
" \n",
|
||
" assert y_3x3.shape == (3, 3), f\"3x3 kernel output should be (3, 3), got {y_3x3.shape}\"\n",
|
||
" print(\"✅ Different kernel sizes work correctly\")\n",
|
||
" \n",
|
||
"except Exception as e:\n",
|
||
" print(f\"❌ Different kernel sizes test failed: {e}\")\n",
|
||
" raise\n",
|
||
"\n",
|
||
"# Show the layer behavior\n",
|
||
"print(\"🎯 Conv2D layer behavior:\")\n",
|
||
"print(\" Learnable kernel weights\")\n",
|
||
"print(\" Applies convolution to detect patterns\")\n",
|
||
"print(\" Can be trained end-to-end\")\n",
|
||
"print(\"📈 Progress: Convolution operation ✓, Conv2D layer ✓\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "04dd554d",
|
||
"metadata": {
|
||
"cell_marker": "\"\"\"",
|
||
"lines_to_next_cell": 1
|
||
},
|
||
"source": [
|
||
"## Step 3: Flattening for Dense Layers\n",
|
||
"\n",
|
||
"### What is Flattening?\n",
|
||
"**Flattening** converts multi-dimensional tensors to 1D vectors, enabling connection between convolutional and dense layers.\n",
|
||
"\n",
|
||
"### Why Flattening is Needed\n",
|
||
"- **Interface compatibility**: Conv2D outputs 2D, Dense expects 1D\n",
|
||
"- **Network composition**: Connect spatial features to classification\n",
|
||
"- **Standard practice**: Almost all CNNs use this pattern\n",
|
||
"- **Dimension management**: Preserve information while changing shape\n",
|
||
"\n",
|
||
"### The Pattern\n",
|
||
"```\n",
|
||
"Conv2D → ReLU → Conv2D → ReLU → Flatten → Dense → Output\n",
|
||
"```\n",
|
||
"\n",
|
||
"### Real-World Usage\n",
|
||
"- **Classification**: Final layers need 1D input for class probabilities\n",
|
||
"- **Feature extraction**: Convert spatial features to vector representations\n",
|
||
"- **Transfer learning**: Extract features from pre-trained CNNs"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "fc87c45d",
|
||
"metadata": {
|
||
"lines_to_next_cell": 1,
|
||
"nbgrader": {
|
||
"grade": false,
|
||
"grade_id": "flatten-function",
|
||
"locked": false,
|
||
"schema_version": 3,
|
||
"solution": true,
|
||
"task": false
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"#| export\n",
|
||
"def flatten(x):\n",
|
||
" \"\"\"\n",
|
||
" Flatten a 2D tensor to 1D (for connecting to Dense layers).\n",
|
||
" \n",
|
||
" Args:\n",
|
||
" x: Input tensor to flatten\n",
|
||
" \n",
|
||
" Returns:\n",
|
||
" Flattened tensor with batch dimension preserved\n",
|
||
" \n",
|
||
" TODO: Implement flattening operation.\n",
|
||
" \n",
|
||
" APPROACH:\n",
|
||
" 1. Get the numpy array from the tensor\n",
|
||
" 2. Use .flatten() to convert to 1D\n",
|
||
" 3. Add batch dimension with [None, :]\n",
|
||
" 4. Return Tensor wrapped around the result\n",
|
||
" \n",
|
||
" EXAMPLE:\n",
|
||
" Input: Tensor([[1, 2], [3, 4]]) # shape (2, 2)\n",
|
||
" Output: Tensor([[1, 2, 3, 4]]) # shape (1, 4)\n",
|
||
" \n",
|
||
" HINTS:\n",
|
||
" - Use x.data.flatten() to get 1D array\n",
|
||
" - Add batch dimension: result[None, :]\n",
|
||
" - Return Tensor(result)\n",
|
||
" \"\"\"\n",
|
||
" ### BEGIN SOLUTION\n",
|
||
" # Flatten the tensor and add batch dimension\n",
|
||
" flattened = x.data.flatten()\n",
|
||
" result = flattened[None, :] # Add batch dimension\n",
|
||
" return type(x)(result)\n",
|
||
" ### END SOLUTION"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "26b6d81b",
|
||
"metadata": {
|
||
"cell_marker": "\"\"\""
|
||
},
|
||
"source": [
|
||
"### 🧪 Unit Test: Flatten Function\n",
|
||
"\n",
|
||
"Let's test your flatten function! This connects convolutional layers to dense layers.\n",
|
||
"\n",
|
||
"**This is a unit test** - it tests one specific function (flatten) in isolation."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "c49e29e6",
|
||
"metadata": {
|
||
"nbgrader": {
|
||
"grade": true,
|
||
"grade_id": "test-flatten-immediate",
|
||
"locked": true,
|
||
"points": 10,
|
||
"schema_version": 3,
|
||
"solution": false,
|
||
"task": false
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Test flatten function immediately after implementation\n",
|
||
"print(\"🔬 Unit Test: Flatten Function...\")\n",
|
||
"\n",
|
||
"# Test case 1: 2x2 tensor\n",
|
||
"try:\n",
|
||
" x = Tensor([[1, 2], [3, 4]])\n",
|
||
" flattened = flatten(x)\n",
|
||
" \n",
|
||
" print(f\"Input: {x}\")\n",
|
||
" print(f\"Flattened: {flattened}\")\n",
|
||
" print(f\"Flattened shape: {flattened.shape}\")\n",
|
||
" \n",
|
||
" # Verify shape and content\n",
|
||
" assert flattened.shape == (1, 4), f\"Flattened shape should be (1, 4), got {flattened.shape}\"\n",
|
||
" expected_data = np.array([[1, 2, 3, 4]])\n",
|
||
" assert np.array_equal(flattened.data, expected_data), f\"Flattened data should be {expected_data}, got {flattened.data}\"\n",
|
||
" print(\"✅ 2x2 flatten test passed\")\n",
|
||
" \n",
|
||
"except Exception as e:\n",
|
||
" print(f\"❌ 2x2 flatten test failed: {e}\")\n",
|
||
" raise\n",
|
||
"\n",
|
||
"# Test case 2: 3x3 tensor\n",
|
||
"try:\n",
|
||
" x2 = Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
|
||
" flattened2 = flatten(x2)\n",
|
||
" \n",
|
||
" assert flattened2.shape == (1, 9), f\"Flattened shape should be (1, 9), got {flattened2.shape}\"\n",
|
||
" expected_data2 = np.array([[1, 2, 3, 4, 5, 6, 7, 8, 9]])\n",
|
||
" assert np.array_equal(flattened2.data, expected_data2), f\"Flattened data should be {expected_data2}, got {flattened2.data}\"\n",
|
||
" print(\"✅ 3x3 flatten test passed\")\n",
|
||
" \n",
|
||
"except Exception as e:\n",
|
||
" print(f\"❌ 3x3 flatten test failed: {e}\")\n",
|
||
" raise\n",
|
||
"\n",
|
||
"# Test case 3: Different shapes\n",
|
||
"try:\n",
|
||
" x3 = Tensor([[1, 2, 3, 4], [5, 6, 7, 8]]) # 2x4\n",
|
||
" flattened3 = flatten(x3)\n",
|
||
" \n",
|
||
" assert flattened3.shape == (1, 8), f\"Flattened shape should be (1, 8), got {flattened3.shape}\"\n",
|
||
" expected_data3 = np.array([[1, 2, 3, 4, 5, 6, 7, 8]])\n",
|
||
" assert np.array_equal(flattened3.data, expected_data3), f\"Flattened data should be {expected_data3}, got {flattened3.data}\"\n",
|
||
" print(\"✅ Different shapes flatten test passed\")\n",
|
||
" \n",
|
||
"except Exception as e:\n",
|
||
" print(f\"❌ Different shapes flatten test failed: {e}\")\n",
|
||
" raise\n",
|
||
"\n",
|
||
"# Show the flattening behavior\n",
|
||
"print(\"🎯 Flatten behavior:\")\n",
|
||
"print(\" Converts 2D tensor to 1D\")\n",
|
||
"print(\" Preserves batch dimension\")\n",
|
||
"print(\" Enables connection to Dense layers\")\n",
|
||
"print(\"📈 Progress: Convolution operation ✓, Conv2D layer ✓, Flatten ✓\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "5fa61ae7",
|
||
"metadata": {
|
||
"cell_marker": "\"\"\""
|
||
},
|
||
"source": [
|
||
"## Step 4: Comprehensive Test - Complete CNN Pipeline\n",
|
||
"\n",
|
||
"### Real-World CNN Applications\n",
|
||
"Let's test our CNN components in realistic scenarios:\n",
|
||
"\n",
|
||
"#### **Image Classification Pipeline**\n",
|
||
"```python\n",
|
||
"# The standard CNN pattern\n",
|
||
"Conv2D → ReLU → Flatten → Dense → Output\n",
|
||
"```\n",
|
||
"\n",
|
||
"#### **Multi-layer CNN**\n",
|
||
"```python\n",
|
||
"# Deeper pattern for complex features\n",
|
||
"Conv2D → ReLU → Conv2D → ReLU → Flatten → Dense → Output\n",
|
||
"```\n",
|
||
"\n",
|
||
"#### **Feature Extraction**\n",
|
||
"```python\n",
|
||
"# Extract spatial features then classify\n",
|
||
"image → CNN features → dense classifier → predictions\n",
|
||
"```\n",
|
||
"\n",
|
||
"This comprehensive test ensures our CNN components work together for real computer vision applications!"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "5b92503c",
|
||
"metadata": {
|
||
"lines_to_next_cell": 1,
|
||
"nbgrader": {
|
||
"grade": true,
|
||
"grade_id": "test-comprehensive",
|
||
"locked": true,
|
||
"points": 15,
|
||
"schema_version": 3,
|
||
"solution": false,
|
||
"task": false
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Comprehensive test - complete CNN applications\n",
|
||
"print(\"🔬 Comprehensive Test: Complete CNN Applications...\")\n",
|
||
"\n",
|
||
"try:\n",
|
||
" # Test 1: Simple CNN Pipeline\n",
|
||
" print(\"\\n1. Simple CNN Pipeline Test:\")\n",
|
||
" \n",
|
||
" # Create pipeline: Conv2D → ReLU → Flatten → Dense\n",
|
||
" conv = Conv2D(kernel_size=(2, 2))\n",
|
||
" relu = ReLU()\n",
|
||
" dense = Dense(input_size=4, output_size=3)\n",
|
||
" \n",
|
||
" # Input image\n",
|
||
" image = Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
|
||
" \n",
|
||
" # Forward pass\n",
|
||
" features = conv(image) # (3,3) → (2,2)\n",
|
||
" activated = relu(features) # (2,2) → (2,2)\n",
|
||
" flattened = flatten(activated) # (2,2) → (1,4)\n",
|
||
" output = dense(flattened) # (1,4) → (1,3)\n",
|
||
" \n",
|
||
" assert features.shape == (2, 2), f\"Conv output shape wrong: {features.shape}\"\n",
|
||
" assert activated.shape == (2, 2), f\"ReLU output shape wrong: {activated.shape}\"\n",
|
||
" assert flattened.shape == (1, 4), f\"Flatten output shape wrong: {flattened.shape}\"\n",
|
||
" assert output.shape == (1, 3), f\"Dense output shape wrong: {output.shape}\"\n",
|
||
" \n",
|
||
" print(\"✅ Simple CNN pipeline works correctly\")\n",
|
||
" \n",
|
||
" # Test 2: Multi-layer CNN\n",
|
||
" print(\"\\n2. Multi-layer CNN Test:\")\n",
|
||
" \n",
|
||
" # Create deeper pipeline: Conv2D → ReLU → Conv2D → ReLU → Flatten → Dense\n",
|
||
" conv1 = Conv2D(kernel_size=(2, 2))\n",
|
||
" relu1 = ReLU()\n",
|
||
" conv2 = Conv2D(kernel_size=(2, 2))\n",
|
||
" relu2 = ReLU()\n",
|
||
" dense_multi = Dense(input_size=9, output_size=2)\n",
|
||
" \n",
|
||
" # Larger input for multi-layer processing\n",
|
||
" large_image = Tensor([[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15], [16, 17, 18, 19, 20], [21, 22, 23, 24, 25]])\n",
|
||
" \n",
|
||
" # Forward pass\n",
|
||
" h1 = conv1(large_image) # (5,5) → (4,4)\n",
|
||
" h2 = relu1(h1) # (4,4) → (4,4)\n",
|
||
" h3 = conv2(h2) # (4,4) → (3,3)\n",
|
||
" h4 = relu2(h3) # (3,3) → (3,3)\n",
|
||
" h5 = flatten(h4) # (3,3) → (1,9)\n",
|
||
" output_multi = dense_multi(h5) # (1,9) → (1,2)\n",
|
||
" \n",
|
||
" assert h1.shape == (4, 4), f\"Conv1 output wrong: {h1.shape}\"\n",
|
||
" assert h3.shape == (3, 3), f\"Conv2 output wrong: {h3.shape}\"\n",
|
||
" assert h5.shape == (1, 9), f\"Flatten output wrong: {h5.shape}\"\n",
|
||
" assert output_multi.shape == (1, 2), f\"Final output wrong: {output_multi.shape}\"\n",
|
||
" \n",
|
||
" print(\"✅ Multi-layer CNN works correctly\")\n",
|
||
" \n",
|
||
" # Test 3: Image Classification Scenario\n",
|
||
" print(\"\\n3. Image Classification Test:\")\n",
|
||
" \n",
|
||
" # Simulate digit classification with 8x8 image\n",
|
||
" digit_image = Tensor([[1, 0, 0, 1, 1, 0, 0, 1],\n",
|
||
" [0, 1, 0, 1, 1, 0, 1, 0],\n",
|
||
" [0, 0, 1, 1, 1, 1, 0, 0],\n",
|
||
" [1, 1, 1, 0, 0, 1, 1, 1],\n",
|
||
" [1, 0, 0, 1, 1, 0, 0, 1],\n",
|
||
" [0, 1, 1, 0, 0, 1, 1, 0],\n",
|
||
" [0, 0, 1, 1, 1, 1, 0, 0],\n",
|
||
" [1, 1, 0, 0, 0, 0, 1, 1]])\n",
|
||
" \n",
|
||
" # CNN for digit classification\n",
|
||
" feature_extractor = Conv2D(kernel_size=(3, 3)) # (8,8) → (6,6)\n",
|
||
" activation = ReLU()\n",
|
||
" classifier = Dense(input_size=36, output_size=10) # 10 digit classes\n",
|
||
" \n",
|
||
" # Forward pass\n",
|
||
" features = feature_extractor(digit_image)\n",
|
||
" activated_features = activation(features)\n",
|
||
" feature_vector = flatten(activated_features)\n",
|
||
" digit_scores = classifier(feature_vector)\n",
|
||
" \n",
|
||
" assert features.shape == (6, 6), f\"Feature extraction shape wrong: {features.shape}\"\n",
|
||
" assert feature_vector.shape == (1, 36), f\"Feature vector shape wrong: {feature_vector.shape}\"\n",
|
||
" assert digit_scores.shape == (1, 10), f\"Digit scores shape wrong: {digit_scores.shape}\"\n",
|
||
" \n",
|
||
" print(\"✅ Image classification scenario works correctly\")\n",
|
||
" \n",
|
||
" # Test 4: Feature Extraction and Composition\n",
|
||
" print(\"\\n4. Feature Extraction Test:\")\n",
|
||
" \n",
|
||
" # Create modular feature extractor\n",
|
||
" feature_conv = Conv2D(kernel_size=(2, 2))\n",
|
||
" feature_activation = ReLU()\n",
|
||
" \n",
|
||
" # Create classifier head\n",
|
||
" classifier_head = Dense(input_size=4, output_size=3)\n",
|
||
" \n",
|
||
" # Test composition\n",
|
||
" test_image = Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
|
||
" \n",
|
||
" # Extract features\n",
|
||
" extracted_features = feature_conv(test_image)\n",
|
||
" activated_features = feature_activation(extracted_features)\n",
|
||
" feature_representation = flatten(activated_features)\n",
|
||
" \n",
|
||
" # Classify\n",
|
||
" predictions = classifier_head(feature_representation)\n",
|
||
" \n",
|
||
" assert extracted_features.shape == (2, 2), f\"Feature extraction wrong: {extracted_features.shape}\"\n",
|
||
" assert feature_representation.shape == (1, 4), f\"Feature representation wrong: {feature_representation.shape}\"\n",
|
||
" assert predictions.shape == (1, 3), f\"Predictions wrong: {predictions.shape}\"\n",
|
||
" \n",
|
||
" print(\"✅ Feature extraction and composition works correctly\")\n",
|
||
" \n",
|
||
" print(\"\\n🎉 Comprehensive test passed! Your CNN components work correctly for:\")\n",
|
||
" print(\" • Image classification pipelines\")\n",
|
||
" print(\" • Multi-layer feature extraction\")\n",
|
||
" print(\" • Spatial pattern recognition\")\n",
|
||
" print(\" • End-to-end CNN workflows\")\n",
|
||
" print(\"📈 Progress: Complete CNN architecture ready for computer vision!\")\n",
|
||
" \n",
|
||
"except Exception as e:\n",
|
||
" print(f\"❌ Comprehensive test failed: {e}\")\n",
|
||
" raise\n",
|
||
"\n",
|
||
"print(\"📈 Final Progress: Complete CNN system ready for computer vision!\")\n",
|
||
"\n",
|
||
"def test_convolution_operation():\n",
|
||
" \"\"\"Test convolution operation implementation comprehensively.\"\"\"\n",
|
||
" print(\"🔬 Unit Test: Convolution Operation...\")\n",
|
||
" \n",
|
||
" # Test basic convolution\n",
|
||
" input_data = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
|
||
" kernel = np.array([[1, 0], [0, 1]])\n",
|
||
" result = conv2d_naive(input_data, kernel)\n",
|
||
" \n",
|
||
" assert result.shape == (2, 2), \"Convolution should produce correct output shape\"\n",
|
||
" expected = np.array([[6, 8], [12, 14]])\n",
|
||
" assert np.array_equal(result, expected), \"Convolution should produce correct values\"\n",
|
||
" \n",
|
||
" print(\"✅ Convolution operation works correctly\")\n",
|
||
"\n",
|
||
"def test_conv2d_layer():\n",
|
||
" \"\"\"Test Conv2D layer implementation comprehensively.\"\"\"\n",
|
||
" print(\"🔬 Unit Test: Conv2D Layer...\")\n",
|
||
" \n",
|
||
" # Test Conv2D layer\n",
|
||
" conv = Conv2D(kernel_size=(3, 3))\n",
|
||
" input_tensor = Tensor(np.random.randn(6, 6))\n",
|
||
" output = conv(input_tensor)\n",
|
||
" \n",
|
||
" assert output.shape == (4, 4), \"Conv2D should produce correct output shape\"\n",
|
||
" assert hasattr(conv, 'kernel'), \"Conv2D should have kernel attribute\"\n",
|
||
" assert conv.kernel.shape == (3, 3), \"Kernel should have correct shape\"\n",
|
||
" \n",
|
||
" print(\"✅ Conv2D layer works correctly\")\n",
|
||
"\n",
|
||
"def test_flatten_function():\n",
|
||
" \"\"\"Test flatten function implementation comprehensively.\"\"\"\n",
|
||
" print(\"🔬 Unit Test: Flatten Function...\")\n",
|
||
" \n",
|
||
" # Test flatten function\n",
|
||
" input_2d = Tensor([[1, 2], [3, 4]])\n",
|
||
" flattened = flatten(input_2d)\n",
|
||
" \n",
|
||
" assert flattened.shape == (1, 4), \"Flatten should produce output with batch dimension\"\n",
|
||
" expected = np.array([[1, 2, 3, 4]])\n",
|
||
" assert np.array_equal(flattened.data, expected), \"Flatten should preserve values\"\n",
|
||
" \n",
|
||
" print(\"✅ Flatten function works correctly\")\n",
|
||
"\n",
|
||
"# CNN pipeline integration test moved to tests/integration/test_cnn_pipeline.py"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0d45209f",
|
||
"metadata": {
|
||
"cell_marker": "\"\"\""
|
||
},
|
||
"source": [
|
||
"## 🧪 Module Testing\n",
|
||
"\n",
|
||
"Time to test your implementation! This section uses TinyTorch's standardized testing framework to ensure your implementation works correctly.\n",
|
||
"\n",
|
||
"**This testing section is locked** - it provides consistent feedback across all modules and cannot be modified."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"id": "30104333",
|
||
"metadata": {
|
||
"nbgrader": {
|
||
"grade": false,
|
||
"grade_id": "standardized-testing",
|
||
"locked": true,
|
||
"schema_version": 3,
|
||
"solution": false,
|
||
"task": false
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# =============================================================================\n",
|
||
"# STANDARDIZED MODULE TESTING - DO NOT MODIFY\n",
|
||
"# This cell is locked to ensure consistent testing across all TinyTorch modules\n",
|
||
"# =============================================================================\n",
|
||
"\n",
|
||
"if __name__ == \"__main__\":\n",
|
||
" from tito.tools.testing import run_module_tests_auto\n",
|
||
" \n",
|
||
" # Automatically discover and run all tests in this module\n",
|
||
" success = run_module_tests_auto(\"CNN\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e346137e",
|
||
"metadata": {
|
||
"cell_marker": "\"\"\""
|
||
},
|
||
"source": [
|
||
"## 🎯 Module Summary\n",
|
||
"\n",
|
||
"Congratulations! You've successfully implemented the core components of convolutional neural networks:\n",
|
||
"\n",
|
||
"### What You've Accomplished\n",
|
||
"✅ **Convolution Operation**: Implemented the sliding window mechanism from scratch \n",
|
||
"✅ **Conv2D Layer**: Built learnable convolutional layers with random initialization \n",
|
||
"✅ **Flatten Function**: Created the bridge between convolutional and dense layers \n",
|
||
"✅ **CNN Pipelines**: Composed complete systems for image processing \n",
|
||
"✅ **Real Applications**: Tested on image classification and feature extraction\n",
|
||
"\n",
|
||
"### Key Concepts You've Learned\n",
|
||
"- **Convolution as pattern matching**: Kernels detect specific features\n",
|
||
"- **Sliding window mechanism**: How convolution processes spatial data\n",
|
||
"- **Parameter sharing**: Same kernel applied across the entire image\n",
|
||
"- **Spatial hierarchy**: Multiple layers build complex features\n",
|
||
"- **CNN architecture**: Conv2D → Activation → Flatten → Dense pattern\n",
|
||
"\n",
|
||
"### Mathematical Foundations\n",
|
||
"- **Convolution operation**: dot product of kernel and image patches\n",
|
||
"- **Output size calculation**: (input_size - kernel_size + 1)\n",
|
||
"- **Translation invariance**: Same pattern detected anywhere in input\n",
|
||
"- **Feature maps**: Spatial representations of detected patterns\n",
|
||
"\n",
|
||
"### Real-World Applications\n",
|
||
"- **Image classification**: Object recognition, medical imaging\n",
|
||
"- **Computer vision**: Face detection, autonomous driving\n",
|
||
"- **Pattern recognition**: Texture analysis, edge detection\n",
|
||
"- **Feature extraction**: Transfer learning, representation learning\n",
|
||
"\n",
|
||
"### CNN Architecture Insights\n",
|
||
"- **Kernel size**: 3×3 most common, balances locality and capacity\n",
|
||
"- **Stacking layers**: Builds hierarchical feature representations\n",
|
||
"- **Spatial reduction**: Each layer reduces spatial dimensions\n",
|
||
"- **Channel progression**: Typically increase channels while reducing spatial size\n",
|
||
"\n",
|
||
"### Performance Characteristics\n",
|
||
"- **Parameter efficiency**: Dramatic reduction vs. fully connected\n",
|
||
"- **Translation invariance**: Robust to object location changes\n",
|
||
"- **Computational efficiency**: Parallel processing of spatial regions\n",
|
||
"- **Memory considerations**: Feature maps require storage during forward pass\n",
|
||
"\n",
|
||
"### Next Steps\n",
|
||
"1. **Export your code**: Use NBDev to export to the `tinytorch` package\n",
|
||
"2. **Test your implementation**: Run the complete test suite\n",
|
||
"3. **Build CNN architectures**: \n",
|
||
" ```python\n",
|
||
" from tinytorch.core.cnn import Conv2D, flatten\n",
|
||
" from tinytorch.core.layers import Dense\n",
|
||
" from tinytorch.core.activations import ReLU\n",
|
||
" \n",
|
||
" # Create CNN\n",
|
||
" conv = Conv2D(kernel_size=(3, 3))\n",
|
||
" relu = ReLU()\n",
|
||
" dense = Dense(input_size=36, output_size=10)\n",
|
||
" \n",
|
||
" # Process image\n",
|
||
" features = relu(conv(image))\n",
|
||
" predictions = dense(flatten(features))\n",
|
||
" ```\n",
|
||
"4. **Explore advanced CNNs**: Pooling, multiple channels, modern architectures!\n",
|
||
"\n",
|
||
"**Ready for the next challenge?** Let's build data loaders to handle real datasets efficiently!"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"jupytext": {
|
||
"main_language": "python"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|