github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-05-08 18:01:20 -05:00
Code Issues 81 Packages Projects Releases 24 Wiki Activity

[GH-ISSUE #1278] tinytorch module 09 / milestone 04 potential defect? #5709

New Issue
Closed
opened 2026-04-21 21:44:17 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-04-21 21:44:17 -05:00
Owner
Copy Link

Originally created by @asgalon on GitHub (Mar 23, 2026).
Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1278

While doing routine tests for an unrelated issue, I encountered the following error running milestone04 / 02_lecun_cifar10.py:

((.venv) ) La-Bestia:PycharmProjects/cs249r_book/tinytorch[1058]> python milestones/04_1998_cnn/02_lecun_cifar10.py   
🎯 CIFAR-10 CNN - Natural Image Recognition with YOUR Convolution Modules!
   Historical significance: CNNs revolutionized computer vision
   YOUR achievement: Spatial feature extraction on real photos
   Components used: YOUR Conv2D + MaxPool2D + complete system

======================================================================
🖼️  VISUALIZING CNN FEATURE EXTRACTION:
======================================================================

    How YOUR CNN Sees Images:           Feature Maps at Each Layer:

    Original Image (32×32×3):           After Conv1 (30×30×32):
    ┌────────────────┐                 ┌─┬─┬─┬─┬─┬─┬─┬─┬─┐
    │ [Cat in grass] │                 │Edge detectors...│ 32 filters
    │ Complex scene  │ → Conv+ReLU →   │Texture maps...  │ detect
    │ Many patterns  │                 │Color gradients. │ features
    └────────────────┘                 └─┴─┴─┴─┴─┴─┴─┴─┴─┘

    After Pool1 (15×15×32):            After Conv2 (13×13×64):
    ┌─────────┐                        ┌─┬─┬─┬─┬─┬─┬─┬─┬─┐
    │Reduced  │                        │Cat ears...      │ 64 filters
    │spatial  │ → Conv+ReLU →          │Cat eyes...      │ combine
    │dimension│                        │Grass texture... │ features
    └─────────┘                        └─┴─┴─┴─┴─┴─┴─┴─┴─┘

    After Pool2 + Flatten:             Classification:
    [6×6×64 = 2304 features] → Dense → [plane|car|bird|CAT|...]
                                              Highest probability

    Key CNN Advantages YOUR Implementation Provides:
    ✓ SPATIAL HIERARCHY: Low → High level features
    ✓ PARAMETER SHARING: 3×3 kernel used everywhere
    ✓ TRANSLATION INVARIANCE: Detects patterns anywhere
    ✓ AUTOMATIC FEATURE LEARNING: No manual engineering!
    
======================================================================

📥 Loading CIFAR-10 dataset...

============================================================
  This milestone requires the CIFAR-10 dataset
============================================================

  Dataset Info:
  +------------------+--------------------------------------+
  | Name             | CIFAR-10                             |
  | Description      | 60,000 natural images (32x32 RGB) in 10 classes |
  | Download Size    | ~170 MB                              |
  | Location         | /Users/peter/PycharmProjects/cs249r_ |
  +------------------+--------------------------------------+

  System Check:
    * Available space: 1883147.6 MB
    * Required space:  ~180 MB
    * Status: Ready to download

  Download CIFAR-10? [Y/n]: 
============================================================
📥 Downloading cifar-10-python.tar.gz...
   [████████████���█████████████████] 100.0% (162.6/162.6 MB)
✅ Download complete!
📦 Extracting CIFAR-10...
✅ Extraction complete!
/Users/peter/PycharmProjects/cs249r_book/tinytorch/milestones/data_manager.py:223: VisibleDeprecationWarning: dtype(): align should be passed as Python or NumPy boolean but got `align=0`. Did you mean to pass a tuple to create a subarray type? (Deprecated NumPy 2.4)
  batch = pickle.load(f, encoding='bytes')
/Users/peter/PycharmProjects/cs249r_book/tinytorch/milestones/data_manager.py:230: VisibleDeprecationWarning: dtype(): align should be passed as Python or NumPy boolean but got `align=0`. Did you mean to pass a tuple to create a subarray type? (Deprecated NumPy 2.4)
  test_batch = pickle.load(f, encoding='bytes')
📊 CIFAR-10 loaded: 50000 training, 10000 test images
✅ Loaded 50000 training, 10000 test images

📦 Creating YOUR Dataset and DataLoader (Module 05)...

======================================================================
📦 YOUR DataLoader in Action (Module 05)
======================================================================

  Dataset Configuration:
  +--------------------+--------------------------------------------+
  | Training Images    | 50000                                      |
  | Test Images        | 10000                                      |
  | Batch Size         | 32                                         |
  | Batches/Epoch      | 1563                                       |
  | Shuffle            | Enabled (training only)                    |
  +--------------------+--------------------------------------------+

  How YOUR DataLoader Processes Data:

  Raw Dataset (50,000 images):
  +-----------------------------------------------+
  | img_0 | img_1 | img_2 | ... | img_49999       |
  +-----------------------------------------------+
              |
              v YOUR DataLoader shuffles indices
              |
  Shuffled Order: [23451, 8923, 45021, 102, ...]
              |
              v YOUR DataLoader creates batches
              |
  +--------------+  +--------------+       +--------------+
  | Batch 1      |  | Batch 2      |  ...  | Batch 1563  |
  | 32 images   |  | 32 images   |       | 16 images   |
  | shuffled!    |  | shuffled!    |       | (remainder)  |
  +--------------+  +--------------+       +--------------+
              |
              v Fed to YOUR CNN one batch at a time
              |
  Memory: Only 32 images loaded at once (not 50,000!)

  Why This Matters:
    * Without batching: 50,000 x 32x32x3 = 146.5 MB in memory
    * With YOUR DataLoader: 32 x 32x32x3 = 0.09 MB per batch
    * Shuffling: Prevents model from memorizing order
======================================================================

   ✅ Data Augmentation: RandomFlip + RandomCrop (training only)
🧠 Building CIFAR-10 CNN with YOUR Tiny🔥Torch modules...
Traceback (most recent call last):
  File "/Users/peter/PycharmProjects/cs249r_book/tinytorch/milestones/04_1998_cnn/02_lecun_cifar10.py", line 722, in <module>
    main()
  File "/Users/peter/PycharmProjects/cs249r_book/tinytorch/milestones/04_1998_cnn/02_lecun_cifar10.py", line 673, in main
    model = CIFARCNN()
            ^^^^^^^^^^
  File "/Users/peter/PycharmProjects/cs249r_book/tinytorch/milestones/04_1998_cnn/02_lecun_cifar10.py", line 235, in __init__
    self.pool = MaxPool2d(pool_size=(2, 2))  # Module 09: YOUR pooling!
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: MaxPool2d.__init__() got an unexpected keyword argument 'pool_size'

The init method of MaxPool2d in 09_convolutions.py looks like this:

    """
    2D Max Pooling layer for spatial dimension reduction.

    Applies maximum operation over spatial windows, preserving
    the strongest activations while reducing computational load.

    Args:
        kernel_size: Size of pooling window (int or tuple)
        stride: Stride of pooling operation (default: same as kernel_size)
        padding: Zero-padding added to input (default: 0)
    """

    def __init__(self, kernel_size, stride=None, padding=0):

I guess the milestone code is missing an update? Probably should be (kernel_size=2, stride=2) instead of pool_size=(2,2) ?

Originally created by @asgalon on GitHub (Mar 23, 2026). Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1278 While doing routine tests for an unrelated issue, I encountered the following error running milestone04 / 02_lecun_cifar10.py: ``` ((.venv) ) La-Bestia:PycharmProjects/cs249r_book/tinytorch[1058]> python milestones/04_1998_cnn/02_lecun_cifar10.py 🎯 CIFAR-10 CNN - Natural Image Recognition with YOUR Convolution Modules! Historical significance: CNNs revolutionized computer vision YOUR achievement: Spatial feature extraction on real photos Components used: YOUR Conv2D + MaxPool2D + complete system ====================================================================== 🖼️ VISUALIZING CNN FEATURE EXTRACTION: ====================================================================== How YOUR CNN Sees Images: Feature Maps at Each Layer: Original Image (32×32×3): After Conv1 (30×30×32): ┌────────────────┐ ┌─┬─┬─┬─┬─┬─┬─┬─┬─┐ │ [Cat in grass] │ │Edge detectors...│ 32 filters │ Complex scene │ → Conv+ReLU → │Texture maps... │ detect │ Many patterns │ │Color gradients. │ features └────────────────┘ └─┴─┴─┴─┴─┴─┴─┴─┴─┘ After Pool1 (15×15×32): After Conv2 (13×13×64): ┌─────────┐ ┌─┬─┬─┬─┬─┬─┬─┬─┬─┐ │Reduced │ │Cat ears... │ 64 filters │spatial │ → Conv+ReLU → │Cat eyes... │ combine │dimension│ │Grass texture... │ features └─────────┘ └─┴─┴─┴─┴─┴─┴─┴─┴─┘ After Pool2 + Flatten: Classification: [6×6×64 = 2304 features] → Dense → [plane|car|bird|CAT|...] Highest probability Key CNN Advantages YOUR Implementation Provides: ✓ SPATIAL HIERARCHY: Low → High level features ✓ PARAMETER SHARING: 3×3 kernel used everywhere ✓ TRANSLATION INVARIANCE: Detects patterns anywhere ✓ AUTOMATIC FEATURE LEARNING: No manual engineering! ====================================================================== 📥 Loading CIFAR-10 dataset... ============================================================ This milestone requires the CIFAR-10 dataset ============================================================ Dataset Info: +------------------+--------------------------------------+ | Name | CIFAR-10 | | Description | 60,000 natural images (32x32 RGB) in 10 classes | | Download Size | ~170 MB | | Location | /Users/peter/PycharmProjects/cs249r_ | +------------------+--------------------------------------+ System Check: * Available space: 1883147.6 MB * Required space: ~180 MB * Status: Ready to download Download CIFAR-10? [Y/n]: ============================================================ 📥 Downloading cifar-10-python.tar.gz... [████████████���█████████████████] 100.0% (162.6/162.6 MB) ✅ Download complete! 📦 Extracting CIFAR-10... ✅ Extraction complete! /Users/peter/PycharmProjects/cs249r_book/tinytorch/milestones/data_manager.py:223: VisibleDeprecationWarning: dtype(): align should be passed as Python or NumPy boolean but got `align=0`. Did you mean to pass a tuple to create a subarray type? (Deprecated NumPy 2.4) batch = pickle.load(f, encoding='bytes') /Users/peter/PycharmProjects/cs249r_book/tinytorch/milestones/data_manager.py:230: VisibleDeprecationWarning: dtype(): align should be passed as Python or NumPy boolean but got `align=0`. Did you mean to pass a tuple to create a subarray type? (Deprecated NumPy 2.4) test_batch = pickle.load(f, encoding='bytes') 📊 CIFAR-10 loaded: 50000 training, 10000 test images ✅ Loaded 50000 training, 10000 test images 📦 Creating YOUR Dataset and DataLoader (Module 05)... ====================================================================== 📦 YOUR DataLoader in Action (Module 05) ====================================================================== Dataset Configuration: +--------------------+--------------------------------------------+ | Training Images | 50000 | | Test Images | 10000 | | Batch Size | 32 | | Batches/Epoch | 1563 | | Shuffle | Enabled (training only) | +--------------------+--------------------------------------------+ How YOUR DataLoader Processes Data: Raw Dataset (50,000 images): +-----------------------------------------------+ | img_0 | img_1 | img_2 | ... | img_49999 | +-----------------------------------------------+ | v YOUR DataLoader shuffles indices | Shuffled Order: [23451, 8923, 45021, 102, ...] | v YOUR DataLoader creates batches | +--------------+ +--------------+ +--------------+ | Batch 1 | | Batch 2 | ... | Batch 1563 | | 32 images | | 32 images | | 16 images | | shuffled! | | shuffled! | | (remainder) | +--------------+ +--------------+ +--------------+ | v Fed to YOUR CNN one batch at a time | Memory: Only 32 images loaded at once (not 50,000!) Why This Matters: * Without batching: 50,000 x 32x32x3 = 146.5 MB in memory * With YOUR DataLoader: 32 x 32x32x3 = 0.09 MB per batch * Shuffling: Prevents model from memorizing order ====================================================================== ✅ Data Augmentation: RandomFlip + RandomCrop (training only) 🧠 Building CIFAR-10 CNN with YOUR Tiny🔥Torch modules... Traceback (most recent call last): File "/Users/peter/PycharmProjects/cs249r_book/tinytorch/milestones/04_1998_cnn/02_lecun_cifar10.py", line 722, in <module> main() File "/Users/peter/PycharmProjects/cs249r_book/tinytorch/milestones/04_1998_cnn/02_lecun_cifar10.py", line 673, in main model = CIFARCNN() ^^^^^^^^^^ File "/Users/peter/PycharmProjects/cs249r_book/tinytorch/milestones/04_1998_cnn/02_lecun_cifar10.py", line 235, in __init__ self.pool = MaxPool2d(pool_size=(2, 2)) # Module 09: YOUR pooling! ^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: MaxPool2d.__init__() got an unexpected keyword argument 'pool_size' ``` The __init__ method of MaxPool2d in 09_convolutions.py looks like this: ``` """ 2D Max Pooling layer for spatial dimension reduction. Applies maximum operation over spatial windows, preserving the strongest activations while reducing computational load. Args: kernel_size: Size of pooling window (int or tuple) stride: Stride of pooling operation (default: same as kernel_size) padding: Zero-padding added to input (default: 0) """ def __init__(self, kernel_size, stride=None, padding=0): ``` I guess the milestone code is missing an update? Probably should be (kernel_size=2, stride=2) instead of pool_size=(2,2) ?
GiteaMirror added the type: bugarea: book labels 2026-04-21 21:44:17 -05:00
GiteaMirror commented 2026-04-21 21:44:19 -05:00
Author
Owner
Copy Link

@profvjreddi commented on GitHub (Mar 28, 2026):

Thanks for the detailed report, @asgalon! This has been fixed. I made it such that the milestone script now uses kernel_size to match the MaxPool2d API in 09_convolutions.py in a66f2c66a6.

Appreciate you taking the time to trace through the traceback and identify the exact mismatch. Closing this out, and thank you again so much!

@all-contributors please add @asgalon as a contributor for 🪲 Bug in TinyTorch

<!-- gh-comment-id:4148561920 --> @profvjreddi commented on GitHub (Mar 28, 2026): Thanks for the detailed report, @asgalon! This has been fixed. I made it such that the milestone script now uses `kernel_size` to match the MaxPool2d API in `09_convolutions.py` in https://github.com/harvard-edge/cs249r_book/commit/a66f2c66a64b3d391947c1bc3b0022bede87c18a. Appreciate you taking the time to trace through the traceback and identify the exact mismatch. Closing this out, and thank you again so much! @all-contributors please add @asgalon as a contributor for 🪲 Bug in TinyTorch
GiteaMirror commented 2026-04-21 21:44:19 -05:00
Author
Owner
Copy Link

@github-actions[bot] commented on GitHub (Mar 28, 2026):

I couldn't find a GitHub username in that comment. 🤔

Your comment: @all-contributors please add @ as a contributor for 🪲 Bug in TinyTorch

Example formats that work:

@all-contributors @jane-doe fixed typos in the documentation
@all-contributors please add @john_smith for Doc in TinyTorch
@all-contributors @user123 for code, doc in tinytorch, book
@all-contributors @dev42 implemented the new caching feature in tinytorch

Contribution types: bug, code, doc, design, ideas, review, test, tool

Projects (one or more): book, tinytorch, kits, labs — specify in comment (e.g. "in TinyTorch, Book") or auto-detected from PR file paths.

<!-- gh-comment-id:4148562224 --> @github-actions[bot] commented on GitHub (Mar 28, 2026): I couldn't find a GitHub username in that comment. :thinking: **Your comment:** @all-contributors please add @ as a contributor for 🪲 Bug in TinyTorch **Example formats that work:** ``` @all-contributors @jane-doe fixed typos in the documentation @all-contributors please add @john_smith for Doc in TinyTorch @all-contributors @user123 for code, doc in tinytorch, book @all-contributors @dev42 implemented the new caching feature in tinytorch ``` **Contribution types:** bug, code, doc, design, ideas, review, test, tool **Projects (one or more):** book, tinytorch, kits, labs — specify in comment (e.g. "in TinyTorch, Book") or auto-detected from PR file paths.
GiteaMirror commented 2026-04-21 21:44:20 -05:00
Author
Owner
Copy Link

@github-actions[bot] commented on GitHub (Mar 28, 2026):

I've added @asgalon as a contributor to tinytorch! 🎉

Recognized for: code
Project(s): tinytorch (explicitly mentioned in comment)
Based on: @all-contributors please add @asgalon as a contributor for 🪲 Bug in TinyTorch

The contributor list has been updated in:

  • tinytorch/.all-contributorsrc, tinytorch/README.md
  • Main README.md

We love recognizing our contributors! ❤️

<!-- gh-comment-id:4148566446 --> @github-actions[bot] commented on GitHub (Mar 28, 2026): I've added @asgalon as a contributor to **tinytorch**! :tada: **Recognized for:** code **Project(s):** tinytorch (explicitly mentioned in comment) **Based on:** @all-contributors please add @asgalon as a contributor for 🪲 Bug in TinyTorch The contributor list has been updated in: - `tinytorch/.all-contributorsrc`, `tinytorch/README.md` - Main `README.md` We love recognizing our contributors! :heart:
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area: book
area: collabs
area: kits
area: socratiq
area: tinytorch
area: tools
area: website
bug
dependencies
format: epub
format: pdf
link-health
link-rot
priority-high
pull-request
Mirrored from GitHub Pull Request
 staffml
type: bug
type: citation
type: code
type: errata
type: improvement
type: new
type: question
vault-sli
No Label area: book type: bug
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#5709
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?