mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-05-06 17:57:31 -05:00
d14f92a9b2
MAJOR IMPROVEMENT: Simplified test discovery logic - Removed restrictive valid_patterns requirement from testing framework - Any function starting with 'test_' is now automatically discovered - Follows standard pytest conventions - no maintenance overhead - Eliminates need to manually add patterns for new test functions CLEANED UP: Test function names across all 10 modules - Removed redundant '_comprehensive' suffix from all test functions - Updated 40+ test function names to be more concise and readable: * 00_setup: 6 functions (test_personal_info, test_system_info, etc.) * 01_tensor: 4 functions (test_tensor_creation, test_tensor_properties, etc.) * 02_activations: 1 function (test_activations) * 03_layers: 3 functions (test_matrix_multiplication, test_dense_layer, etc.) * 04_networks: 4 functions (test_sequential_networks, test_mlp_creation, etc.) * 05_cnn: 3 functions (test_convolution_operation, test_conv2d_layer, etc.) * 06_dataloader: 4 functions (test_dataset_interface, test_dataloader, etc.) * 07_autograd: 6 functions (test_variable_class, test_add_operation, etc.) * 08_optimizers: 5 functions (test_gradient_descent_step, test_sgd_optimizer, etc.) * 09_training: 6 functions (test_mse_loss, test_crossentropy_loss, etc.) * 10_compression: 6 functions (already cleaned up) VERIFICATION: All tests still pass - All 10 modules tested successfully with new discovery logic - Total test count maintained: 47 inline tests across all modules - No functionality lost, only improved maintainability RESULT: Much cleaner, more maintainable testing framework following standard conventions
🧠 Module X: CNN - Convolutional Neural Networks
📊 Module Info
- Difficulty: ⭐⭐⭐ Advanced
- Time Estimate: 6-8 hours
- Prerequisites: Tensor, Activations, Layers, Networks modules
- Next Steps: Training, Computer Vision modules
Implement the core building block of modern computer vision: the convolutional layer.
🎯 Learning Objectives
- Understand the convolution operation (sliding window, local connectivity, weight sharing)
- Implement Conv2D with explicit for-loops (single channel, single filter, no stride/pad)
- Visualize how convolution builds feature maps
- Compose Conv2D with other layers to build a simple ConvNet
- (Stretch) Explore stride, padding, pooling, and multi-channel input
🧠 Build → Use → Understand
- Build: Implement Conv2D from scratch (for-loop)
- Use: Compose Conv2D with ReLU, Flatten, Dense to build a ConvNet
- Understand: Visualize and analyze how convolution works
📚 What You'll Build
- Conv2D (for-loop): The core operation, implemented by you
- Conv2D Layer: Wrap your function in a layer class
- Simple ConvNet: Compose Conv2D → ReLU → Flatten → Dense
- Visualization: See how the filter slides and builds the output
🛠️ Provided Functionality
- Stride and Padding: Provided as utilities or stretch goals
- Multi-channel/Filter Support: Provided or as stretch
- Pooling (Max/Avg): Optional, provided or as stretch
- Flatten Layer: Provided
- Visualization: Provided for learning
- Tests: Provided for feedback
🤔 Why Focus on the For-Loop?
Implementing the convolution for-loop is the best way to understand what makes CNNs powerful. You’ll see exactly how the filter slides, how local patterns are captured, and why this operation is so efficient for images. Other features (stride, padding, pooling) are important, but the core insight comes from building the basic operation yourself.
🚀 Getting Started
cd modules/cnn
jupyter notebook cnn_dev.ipynb # or edit cnn_dev.py
📖 Module Structure
modules/cnn/
├── cnn_dev.py # Main development file (work here!)
├── cnn_dev.ipynb # Jupyter notebook version
├── tests/
│ └── test_cnn.py # Tests for your implementation
├── README.md # This file
🧪 Testing Your Implementation
# Run tests
python -m pytest tests/test_cnn.py -v
🌟 Stretch Goals
- Add stride and padding support
- Support multi-channel input/output
- Implement pooling layers
- Visualize learned filters and feature maps
💡 Key Insight
Convolution is a new, fundamental building block. By implementing it yourself, you’ll understand the magic behind modern vision models!