github-starred/TinyTorch
2
0
Fork 0
mirror of https://github.com/MLSysBook/TinyTorch.git synced 2026-05-02 00:52:45 -05:00
Code Issues 7 Packages Projects Releases Wiki Activity

Standardize all modules to follow NBGrader style guide

Browse Source 
- Updated 7 non-compliant modules for consistency
- Module 01_setup: Added EXAMPLE USAGE sections with code examples
- Module 02_tensor: Added STEP-BY-STEP IMPLEMENTATION and LEARNING CONNECTIONS
- Module 05_dense: Added LEARNING CONNECTIONS to all functions
- Module 06_spatial: Added STEP-BY-STEP and LEARNING CONNECTIONS
- Module 08_dataloader: Added LEARNING CONNECTIONS sections
- Module 11_training: Added STEP-BY-STEP and LEARNING CONNECTIONS
- Module 14_benchmarking: Added STEP-BY-STEP and LEARNING CONNECTIONS
- All modules now follow consistent format per NBGRADER_STYLE_GUIDE.md
- Preserved all existing solution blocks and functionality
This commit is contained in:
Vijay Janapa Reddi
2025-09-16 16:48:14 -04:00
parent 0a0197b72c
commit 6349c218d2
7 changed files with 402 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files

28
modules/source/06_spatial/spatial_dev.py
View File

@@ -145,7 +145,7 @@ def conv2d_naive(input: np.ndarray, kernel: np.ndarray) -> np.ndarray:
TODO: Implement the sliding window convolution using for-loops.
APPROACH:
STEP-BY-STEP IMPLEMENTATION:
1. Get input dimensions: H, W = input.shape
2. Get kernel dimensions: kH, kW = kernel.shape
3. Calculate output dimensions: out_H = H - kH + 1, out_W = W - kW + 1
@@ -157,6 +157,12 @@ def conv2d_naive(input: np.ndarray, kernel: np.ndarray) -> np.ndarray:
- dj loop: kernel columns (0 to kW-1)
6. For each (i,j), compute: output[i,j] += input[i+di, j+dj] * kernel[di, dj]
LEARNING CONNECTIONS:
- **Computer Vision Foundation**: Convolution is the core operation in CNNs and image processing
- **Feature Detection**: Different kernels detect edges, textures, and patterns in images
- **Spatial Hierarchies**: Convolution preserves spatial relationships while extracting features
- **Production CNNs**: Understanding the basic operation helps optimize GPU implementations
EXAMPLE:
Input: [[1, 2, 3], Kernel: [[1, 0],
[4, 5, 6], [0, -1]]
@@ -467,10 +473,16 @@ def flatten(x):
TODO: Implement flattening operation.
APPROACH:
STEP-BY-STEP IMPLEMENTATION:
1. Get the numpy array from the tensor
2. Use .flatten() to convert to 1D
3. Add batch dimension with [None, :]
LEARNING CONNECTIONS:
- **CNN to MLP Transition**: Flattening connects convolutional and dense layers
- **Spatial to Vector**: Converts 2D feature maps to vectors for classification
- **Memory Layout**: Understanding how tensors are stored and reshaped in memory
- **Framework Design**: All major frameworks (PyTorch, TensorFlow) use similar patterns
4. Return Tensor wrapped around the result
EXAMPLE:
@@ -955,6 +967,18 @@ class ConvolutionProfiler:
TODO: Implement convolution operation profiling.
STEP-BY-STEP IMPLEMENTATION:
1. Profile different kernel sizes and their computational costs
2. Measure memory usage patterns for spatial operations
3. Analyze cache efficiency and memory access patterns
4. Identify optimization opportunities for production systems
LEARNING CONNECTIONS:
- **Performance Optimization**: Understanding computational costs of different kernel sizes
- **Memory Efficiency**: Cache-friendly access patterns improve performance significantly
- **Production Scaling**: Profiling guides hardware selection and deployment strategies
- **GPU Optimization**: Spatial operations are ideal for parallel processing
APPROACH:
1. Time convolution operations with different kernel sizes
2. Analyze memory usage patterns for spatial operations