mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-04-27 21:07:37 -05:00
9199199845
- Add 17 intermediate test points across 6 modules for immediate student feedback - Tensor module: Tests after creation, properties, arithmetic, and operators - Activations module: Tests after each activation function (ReLU, Sigmoid, Tanh, Softmax) - Layers module: Tests after matrix multiplication and Dense layer implementation - Networks module: Tests after Sequential class and MLP creation - CNN module: Tests after convolution, Conv2D layer, and flatten operations - DataLoader module: Tests after Dataset interface and DataLoader class - All tests include visual progress indicators and behavioral explanations - Maintains NBGrader compliance with proper metadata and point allocation - Enables steady forward progress and better debugging for students - 100% test success rate across all modules and integration testing
🧠 Module X: CNN - Convolutional Neural Networks
📊 Module Info
- Difficulty: ⭐⭐⭐ Advanced
- Time Estimate: 6-8 hours
- Prerequisites: Tensor, Activations, Layers, Networks modules
- Next Steps: Training, Computer Vision modules
Implement the core building block of modern computer vision: the convolutional layer.
🎯 Learning Objectives
- Understand the convolution operation (sliding window, local connectivity, weight sharing)
- Implement Conv2D with explicit for-loops (single channel, single filter, no stride/pad)
- Visualize how convolution builds feature maps
- Compose Conv2D with other layers to build a simple ConvNet
- (Stretch) Explore stride, padding, pooling, and multi-channel input
🧠 Build → Use → Understand
- Build: Implement Conv2D from scratch (for-loop)
- Use: Compose Conv2D with ReLU, Flatten, Dense to build a ConvNet
- Understand: Visualize and analyze how convolution works
📚 What You'll Build
- Conv2D (for-loop): The core operation, implemented by you
- Conv2D Layer: Wrap your function in a layer class
- Simple ConvNet: Compose Conv2D → ReLU → Flatten → Dense
- Visualization: See how the filter slides and builds the output
🛠️ Provided Functionality
- Stride and Padding: Provided as utilities or stretch goals
- Multi-channel/Filter Support: Provided or as stretch
- Pooling (Max/Avg): Optional, provided or as stretch
- Flatten Layer: Provided
- Visualization: Provided for learning
- Tests: Provided for feedback
🤔 Why Focus on the For-Loop?
Implementing the convolution for-loop is the best way to understand what makes CNNs powerful. You’ll see exactly how the filter slides, how local patterns are captured, and why this operation is so efficient for images. Other features (stride, padding, pooling) are important, but the core insight comes from building the basic operation yourself.
🚀 Getting Started
cd modules/cnn
jupyter notebook cnn_dev.ipynb # or edit cnn_dev.py
📖 Module Structure
modules/cnn/
├── cnn_dev.py # Main development file (work here!)
├── cnn_dev.ipynb # Jupyter notebook version
├── tests/
│ └── test_cnn.py # Tests for your implementation
├── README.md # This file
🧪 Testing Your Implementation
# Run tests
python -m pytest tests/test_cnn.py -v
🌟 Stretch Goals
- Add stride and padding support
- Support multi-channel input/output
- Implement pooling layers
- Visualize learned filters and feature maps
💡 Key Insight
Convolution is a new, fundamental building block. By implementing it yourself, you’ll understand the magic behind modern vision models!