github-starred/TinyTorch
2
0
Fork 0
mirror of https://github.com/MLSysBook/TinyTorch.git synced 2026-03-11 22:25:29 -05:00
Code Issues 7 Packages Projects Releases Wiki Activity

🔄 Restore NBGrader workflow and clean up remaining artifacts

Browse Source 
✅ NBGRADER WORKFLOW RESTORED:
- Restored assignments/ directory with 6 source assignments
- Restored nbgrader_config.py and gradebook.db
- Restored tito/commands/nbgrader.py for full NBGrader integration
- Restored bin/generate_student_notebooks.py

🧹 CLEANUP COMPLETED:
- Removed outdated tests/ directory (less comprehensive than module tests)
- Cleaned up Python cache files (__pycache__)
- Removed .pytest_cache directory
- Preserved all essential functionality

📚 DOCUMENTATION UPDATED:
- Added NBGrader workflow to INSTRUCTOR_GUIDE.md
- Updated README.md with NBGrader integration info
- Clear instructor workflow: Create solutions → Generate student versions → Release → Grade

✅ VERIFIED WORKING:
- tito nbgrader generate 00_setup ✅
- tito nbgrader status ✅
- tito system doctor ✅
- Module tests still pass ✅

🎯 INSTRUCTOR WORKFLOW NOW COMPLETE:
1. Create instructor solutions in modules/XX/XX_dev.py
2. Generate student versions: tito nbgrader generate XX
3. Release assignments: tito nbgrader release XX
4. Collect & grade: tito nbgrader collect XX && tito nbgrader autograde XX

Repository now properly supports full instructor → student workflow with NBGrader
?
This commit is contained in:
Vijay Janapa Reddi
2025-07-12 11:26:44 -04:00
parent 27208e3492
commit b5cd73cfb8
14 changed files with 5431 additions and 536 deletions
Download Patch File Download Diff File Expand all files Collapse all files

693
assignments/source/00_setup/00_setup.ipynb Normal file
View File

@@ -0,0 +1,693 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "cafb32ac",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"# Module 0: Setup - Tiny\ud83d\udd25Torch Development Workflow (Enhanced for NBGrader)\n",
"\n",
"Welcome to TinyTorch! This module teaches you the development workflow you'll use throughout the course.\n",
"\n",
"## Learning Goals\n",
"- Understand the nbdev notebook-to-Python workflow\n",
"- Write your first TinyTorch code\n",
"- Run tests and use the CLI tools\n",
"- Get comfortable with the development rhythm\n",
"\n",
"## The TinyTorch Development Cycle\n",
"\n",
"1. **Write code** in this notebook using `#| export` \n",
"2. **Export code** with `python bin/tito.py sync --module setup`\n",
"3. **Run tests** with `python bin/tito.py test --module setup`\n",
"4. **Check progress** with `python bin/tito.py info`\n",
"\n",
"## New: NBGrader Integration\n",
"This module is also configured for automated grading with **100 points total**:\n",
"- Basic Functions: 30 points\n",
"- SystemInfo Class: 35 points \n",
"- DeveloperProfile Class: 35 points\n",
"\n",
"Let's get started!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "91c9132b",
"metadata": {},
"outputs": [],
"source": [
"#| default_exp core.utils"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "efc50115",
"metadata": {},
"outputs": [],
"source": [
"#| export\n",
"# Setup imports and environment\n",
"import sys\n",
"import platform\n",
"from datetime import datetime\n",
"import os\n",
"from pathlib import Path\n",
"\n",
"print(\"\ud83d\udd25 TinyTorch Development Environment\")\n",
"print(f\"Python {sys.version}\")\n",
"print(f\"Platform: {platform.system()} {platform.release()}\")\n",
"print(f\"Started: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\")"
]
},
{
"cell_type": "markdown",
"id": "da9605e8",
"metadata": {
"cell_marker": "\"\"\"",
"lines_to_next_cell": 1
},
"source": [
"## Step 1: Basic Functions (30 Points)\n",
"\n",
"Let's start with simple functions that form the foundation of TinyTorch."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f2834c3a",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"#| export\n",
"def hello_tinytorch():\n",
" \"\"\"\n",
" A simple hello world function for TinyTorch.\n",
" \n",
" Display TinyTorch ASCII art and welcome message.\n",
" Load the flame art from tinytorch_flame.txt file with graceful fallback.\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Load ASCII art from tinytorch_flame.txt file with graceful fallback\n",
" #| solution_test: Function should display ASCII art and welcome message\n",
" #| difficulty: easy\n",
" #| points: 10\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
"\n",
"def add_numbers(a, b):\n",
" \"\"\"\n",
" Add two numbers together.\n",
" \n",
" This is the foundation of all mathematical operations in ML.\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Use the + operator to add two numbers\n",
" #| solution_test: add_numbers(2, 3) should return 5\n",
" #| difficulty: easy\n",
" #| points: 10\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end"
]
},
{
"cell_type": "markdown",
"id": "b36d3f2a",
"metadata": {
"cell_marker": "\"\"\"",
"lines_to_next_cell": 1
},
"source": [
"## Hidden Tests: Basic Functions (10 Points)\n",
"\n",
"These tests verify the basic functionality and award points automatically."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8c6da938",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"### BEGIN HIDDEN TESTS\n",
"def test_hello_tinytorch():\n",
" \"\"\"Test hello_tinytorch function (5 points)\"\"\"\n",
" import io\n",
" import sys\n",
" \n",
" # Capture output\n",
" captured_output = io.StringIO()\n",
" sys.stdout = captured_output\n",
" \n",
" try:\n",
" hello_tinytorch()\n",
" output = captured_output.getvalue()\n",
" \n",
" # Check that some output was produced\n",
" assert len(output) > 0, \"Function should produce output\"\n",
" assert \"TinyTorch\" in output, \"Output should contain 'TinyTorch'\"\n",
" \n",
" finally:\n",
" sys.stdout = sys.__stdout__\n",
"\n",
"def test_add_numbers():\n",
" \"\"\"Test add_numbers function (5 points)\"\"\"\n",
" # Test basic addition\n",
" assert add_numbers(2, 3) == 5, \"add_numbers(2, 3) should return 5\"\n",
" assert add_numbers(0, 0) == 0, \"add_numbers(0, 0) should return 0\"\n",
" assert add_numbers(-1, 1) == 0, \"add_numbers(-1, 1) should return 0\"\n",
" \n",
" # Test with floats\n",
" assert add_numbers(2.5, 3.5) == 6.0, \"add_numbers(2.5, 3.5) should return 6.0\"\n",
" \n",
" # Test with negative numbers\n",
" assert add_numbers(-5, -3) == -8, \"add_numbers(-5, -3) should return -8\"\n",
"### END HIDDEN TESTS"
]
},
{
"cell_type": "markdown",
"id": "d0d424bd",
"metadata": {
"cell_marker": "\"\"\"",
"lines_to_next_cell": 1
},
"source": [
"## Step 2: SystemInfo Class (35 Points)\n",
"\n",
"Let's create a class that collects and displays system information."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "67f3ace6",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"#| export\n",
"class SystemInfo:\n",
" \"\"\"\n",
" Simple system information class.\n",
" \n",
" Collects and displays Python version, platform, and machine information.\n",
" \"\"\"\n",
" \n",
" def __init__(self):\n",
" \"\"\"\n",
" Initialize system information collection.\n",
" \n",
" Collect Python version, platform, and machine information.\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Use sys.version_info, platform.system(), and platform.machine()\n",
" #| solution_test: Should store Python version, platform, and machine info\n",
" #| difficulty: medium\n",
" #| points: 15\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
" \n",
" def __str__(self):\n",
" \"\"\"\n",
" Return human-readable system information.\n",
" \n",
" Format system info as a readable string.\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Format as \"Python X.Y on Platform (Machine)\"\n",
" #| solution_test: Should return formatted string with version and platform\n",
" #| difficulty: easy\n",
" #| points: 10\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
" \n",
" def is_compatible(self):\n",
" \"\"\"\n",
" Check if system meets minimum requirements.\n",
" \n",
" Check if Python version is >= 3.8\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Compare self.python_version with (3, 8) tuple\n",
" #| solution_test: Should return True for Python >= 3.8\n",
" #| difficulty: medium\n",
" #| points: 10\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end"
]
},
{
"cell_type": "markdown",
"id": "ae39807e",
"metadata": {
"cell_marker": "\"\"\"",
"lines_to_next_cell": 1
},
"source": [
"## Hidden Tests: SystemInfo Class (35 Points)\n",
"\n",
"These tests verify the SystemInfo class implementation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1f3b932f",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"### BEGIN HIDDEN TESTS\n",
"def test_systeminfo_init():\n",
" \"\"\"Test SystemInfo initialization (15 points)\"\"\"\n",
" info = SystemInfo()\n",
" \n",
" # Check that attributes are set\n",
" assert hasattr(info, 'python_version'), \"Should have python_version attribute\"\n",
" assert hasattr(info, 'platform'), \"Should have platform attribute\"\n",
" assert hasattr(info, 'machine'), \"Should have machine attribute\"\n",
" \n",
" # Check types\n",
" assert isinstance(info.python_version, tuple), \"python_version should be tuple\"\n",
" assert isinstance(info.platform, str), \"platform should be string\"\n",
" assert isinstance(info.machine, str), \"machine should be string\"\n",
" \n",
" # Check values are reasonable\n",
" assert len(info.python_version) >= 2, \"python_version should have at least major.minor\"\n",
" assert len(info.platform) > 0, \"platform should not be empty\"\n",
"\n",
"def test_systeminfo_str():\n",
" \"\"\"Test SystemInfo string representation (10 points)\"\"\"\n",
" info = SystemInfo()\n",
" str_repr = str(info)\n",
" \n",
" # Check that the string contains expected elements\n",
" assert \"Python\" in str_repr, \"String should contain 'Python'\"\n",
" assert str(info.python_version.major) in str_repr, \"String should contain major version\"\n",
" assert str(info.python_version.minor) in str_repr, \"String should contain minor version\"\n",
" assert info.platform in str_repr, \"String should contain platform\"\n",
" assert info.machine in str_repr, \"String should contain machine\"\n",
"\n",
"def test_systeminfo_compatibility():\n",
" \"\"\"Test SystemInfo compatibility check (10 points)\"\"\"\n",
" info = SystemInfo()\n",
" compatibility = info.is_compatible()\n",
" \n",
" # Check that it returns a boolean\n",
" assert isinstance(compatibility, bool), \"is_compatible should return boolean\"\n",
" \n",
" # Check that it's reasonable (we're running Python >= 3.8)\n",
" assert compatibility == True, \"Should return True for Python >= 3.8\"\n",
"### END HIDDEN TESTS"
]
},
{
"cell_type": "markdown",
"id": "9a5e4b60",
"metadata": {
"cell_marker": "\"\"\"",
"lines_to_next_cell": 1
},
"source": [
"## Step 3: DeveloperProfile Class (35 Points)\n",
"\n",
"Let's create a personalized developer profile system."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5d5ee4df",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"#| export\n",
"class DeveloperProfile:\n",
" \"\"\"\n",
" Developer profile for personalizing TinyTorch experience.\n",
" \n",
" Stores and displays developer information with ASCII art.\n",
" \"\"\"\n",
" \n",
" @staticmethod\n",
" def _load_default_flame():\n",
" \"\"\"\n",
" Load the default TinyTorch flame ASCII art from file.\n",
" \n",
" Load from tinytorch_flame.txt with graceful fallback.\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Use Path and file operations with try/except for fallback\n",
" #| solution_test: Should load ASCII art from file or provide fallback\n",
" #| difficulty: hard\n",
" #| points: 5\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
" \n",
" def __init__(self, name=\"Vijay Janapa Reddi\", affiliation=\"Harvard University\", \n",
" email=\"vj@eecs.harvard.edu\", github_username=\"profvjreddi\", ascii_art=None):\n",
" \"\"\"\n",
" Initialize developer profile.\n",
" \n",
" Store developer information with sensible defaults.\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Store all parameters as instance attributes, use _load_default_flame for ascii_art if None\n",
" #| solution_test: Should store all developer information\n",
" #| difficulty: medium\n",
" #| points: 15\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
" \n",
" def __str__(self):\n",
" \"\"\"\n",
" Return formatted developer information.\n",
" \n",
" Format as professional signature.\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Format as \"\ud83d\udc68\u200d\ud83d\udcbb Name | Affiliation | @username\"\n",
" #| solution_test: Should return formatted string with name, affiliation, and username\n",
" #| difficulty: easy\n",
" #| points: 5\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
" \n",
" def get_signature(self):\n",
" \"\"\"\n",
" Get a short signature for code headers.\n",
" \n",
" Return concise signature like \"Built by Name (@github)\"\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Format as \"Built by Name (@username)\"\n",
" #| solution_test: Should return signature with name and username\n",
" #| difficulty: easy\n",
" #| points: 5\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
" \n",
" def get_ascii_art(self):\n",
" \"\"\"\n",
" Get ASCII art for the profile.\n",
" \n",
" Return custom ASCII art or default flame.\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Simply return self.ascii_art\n",
" #| solution_test: Should return stored ASCII art\n",
" #| difficulty: easy\n",
" #| points: 5\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
"\n",
" def get_full_profile(self):\n",
" \"\"\"\n",
" Get complete profile with ASCII art.\n",
" \n",
" Return full profile display including ASCII art and all details.\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Format with ASCII art, then developer details with emojis\n",
" #| solution_test: Should return complete profile with ASCII art and details\n",
" #| difficulty: medium\n",
" #| points: 10\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end"
]
},
{
"cell_type": "markdown",
"id": "c1a3e703",
"metadata": {
"cell_marker": "\"\"\"",
"lines_to_next_cell": 1
},
"source": [
"## Hidden Tests: DeveloperProfile Class (35 Points)\n",
"\n",
"These tests verify the DeveloperProfile class implementation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b3d109a4",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"### BEGIN HIDDEN TESTS\n",
"def test_developer_profile_init():\n",
" \"\"\"Test DeveloperProfile initialization (15 points)\"\"\"\n",
" # Test with defaults\n",
" profile = DeveloperProfile()\n",
" \n",
" assert hasattr(profile, 'name'), \"Should have name attribute\"\n",
" assert hasattr(profile, 'affiliation'), \"Should have affiliation attribute\"\n",
" assert hasattr(profile, 'email'), \"Should have email attribute\"\n",
" assert hasattr(profile, 'github_username'), \"Should have github_username attribute\"\n",
" assert hasattr(profile, 'ascii_art'), \"Should have ascii_art attribute\"\n",
" \n",
" # Check default values\n",
" assert profile.name == \"Vijay Janapa Reddi\", \"Should have default name\"\n",
" assert profile.affiliation == \"Harvard University\", \"Should have default affiliation\"\n",
" assert profile.email == \"vj@eecs.harvard.edu\", \"Should have default email\"\n",
" assert profile.github_username == \"profvjreddi\", \"Should have default username\"\n",
" assert profile.ascii_art is not None, \"Should have ASCII art\"\n",
" \n",
" # Test with custom values\n",
" custom_profile = DeveloperProfile(\n",
" name=\"Test User\",\n",
" affiliation=\"Test University\",\n",
" email=\"test@test.com\",\n",
" github_username=\"testuser\",\n",
" ascii_art=\"Custom Art\"\n",
" )\n",
" \n",
" assert custom_profile.name == \"Test User\", \"Should store custom name\"\n",
" assert custom_profile.affiliation == \"Test University\", \"Should store custom affiliation\"\n",
" assert custom_profile.email == \"test@test.com\", \"Should store custom email\"\n",
" assert custom_profile.github_username == \"testuser\", \"Should store custom username\"\n",
" assert custom_profile.ascii_art == \"Custom Art\", \"Should store custom ASCII art\"\n",
"\n",
"def test_developer_profile_str():\n",
" \"\"\"Test DeveloperProfile string representation (5 points)\"\"\"\n",
" profile = DeveloperProfile()\n",
" str_repr = str(profile)\n",
" \n",
" assert \"\ud83d\udc68\u200d\ud83d\udcbb\" in str_repr, \"Should contain developer emoji\"\n",
" assert profile.name in str_repr, \"Should contain name\"\n",
" assert profile.affiliation in str_repr, \"Should contain affiliation\"\n",
" assert f\"@{profile.github_username}\" in str_repr, \"Should contain @username\"\n",
"\n",
"def test_developer_profile_signature():\n",
" \"\"\"Test DeveloperProfile signature (5 points)\"\"\"\n",
" profile = DeveloperProfile()\n",
" signature = profile.get_signature()\n",
" \n",
" assert \"Built by\" in signature, \"Should contain 'Built by'\"\n",
" assert profile.name in signature, \"Should contain name\"\n",
" assert f\"@{profile.github_username}\" in signature, \"Should contain @username\"\n",
"\n",
"def test_developer_profile_ascii_art():\n",
" \"\"\"Test DeveloperProfile ASCII art (5 points)\"\"\"\n",
" profile = DeveloperProfile()\n",
" ascii_art = profile.get_ascii_art()\n",
" \n",
" assert isinstance(ascii_art, str), \"ASCII art should be string\"\n",
" assert len(ascii_art) > 0, \"ASCII art should not be empty\"\n",
" assert \"TinyTorch\" in ascii_art, \"ASCII art should contain 'TinyTorch'\"\n",
"\n",
"def test_default_flame_loading():\n",
" \"\"\"Test default flame loading (5 points)\"\"\"\n",
" flame_art = DeveloperProfile._load_default_flame()\n",
" \n",
" assert isinstance(flame_art, str), \"Flame art should be string\"\n",
" assert len(flame_art) > 0, \"Flame art should not be empty\"\n",
" assert \"TinyTorch\" in flame_art, \"Flame art should contain 'TinyTorch'\"\n",
"### END HIDDEN TESTS"
]
},
{
"cell_type": "markdown",
"id": "42659b3b",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"## Test Your Implementation\n",
"\n",
"Run these cells to test your implementation:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3d81e1b7",
"metadata": {},
"outputs": [],
"source": [
"# Test basic functions\n",
"print(\"Testing Basic Functions:\")\n",
"try:\n",
" hello_tinytorch()\n",
" print(f\"2 + 3 = {add_numbers(2, 3)}\")\n",
" print(\"\u2705 Basic functions working!\")\n",
"except Exception as e:\n",
" print(f\"\u274c Error: {e}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a92516e8",
"metadata": {},
"outputs": [],
"source": [
"# Test SystemInfo\n",
"print(\"\\nTesting SystemInfo:\")\n",
"try:\n",
" info = SystemInfo()\n",
" print(f\"System: {info}\")\n",
" print(f\"Compatible: {info.is_compatible()}\")\n",
" print(\"\u2705 SystemInfo working!\")\n",
"except Exception as e:\n",
" print(f\"\u274c Error: {e}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "eba2d616",
"metadata": {},
"outputs": [],
"source": [
"# Test DeveloperProfile\n",
"print(\"\\nTesting DeveloperProfile:\")\n",
"try:\n",
" profile = DeveloperProfile()\n",
" print(f\"Profile: {profile}\")\n",
" print(f\"Signature: {profile.get_signature()}\")\n",
" print(\"\u2705 DeveloperProfile working!\")\n",
"except Exception as e:\n",
" print(f\"\u274c Error: {e}\")"
]
},
{
"cell_type": "markdown",
"id": "5c25d67e",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"## \ud83c\udf89 Module Complete!\n",
"\n",
"You've successfully implemented the setup module with **100 points total**:\n",
"\n",
"### Point Breakdown:\n",
"- **hello_tinytorch()**: 10 points\n",
"- **add_numbers()**: 10 points \n",
"- **Basic function tests**: 10 points\n",
"- **SystemInfo.__init__()**: 15 points\n",
"- **SystemInfo.__str__()**: 10 points\n",
"- **SystemInfo.is_compatible()**: 10 points\n",
"- **DeveloperProfile.__init__()**: 15 points\n",
"- **DeveloperProfile methods**: 20 points\n",
"\n",
"### What's Next:\n",
"1. Export your code: `tito sync --module setup`\n",
"2. Run tests: `tito test --module setup`\n",
"3. Generate assignment: `tito nbgrader generate --module setup`\n",
"4. Move to Module 1: Tensor!\n",
"\n",
"### NBGrader Features:\n",
"- \u2705 Automatic grading with 100 points\n",
"- \u2705 Partial credit for each component\n",
"- \u2705 Hidden tests for comprehensive validation\n",
"- \u2705 Immediate feedback for students\n",
"- \u2705 Compatible with existing TinyTorch workflow\n",
"\n",
"Happy building! \ud83d\udd25"
]
}
],
"metadata": {
"jupytext": {
"main_language": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

480
assignments/source/01_tensor/01_tensor.ipynb Normal file
View File

@@ -0,0 +1,480 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "0cf257dc",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"# Module 1: Tensor - Enhanced with nbgrader Support\n",
"\n",
"This is an enhanced version of the tensor module that demonstrates dual-purpose content creation:\n",
"- **Self-learning**: Rich educational content with guided implementation\n",
"- **Auto-grading**: nbgrader-compatible assignments with hidden tests\n",
"\n",
"## Dual System Benefits\n",
"\n",
"1. **Single Source**: One file generates both learning and assignment materials\n",
"2. **Consistent Quality**: Same instructor solutions in both contexts\n",
"3. **Flexible Assessment**: Choose between self-paced learning or formal grading\n",
"4. **Scalable**: Handle large courses with automated feedback\n",
"\n",
"## How It Works\n",
"\n",
"- **TinyTorch markers**: `#| exercise_start/end` for educational content\n",
"- **nbgrader markers**: `### BEGIN/END SOLUTION` for auto-grading\n",
"- **Hidden tests**: `### BEGIN/END HIDDEN TESTS` for automatic verification\n",
"- **Dual generation**: One command creates both student notebooks and assignments"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dbe77981",
"metadata": {},
"outputs": [],
"source": [
"#| default_exp core.tensor"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7dc4f1a0",
"metadata": {},
"outputs": [],
"source": [
"#| export\n",
"import numpy as np\n",
"from typing import Union, List, Tuple, Optional"
]
},
{
"cell_type": "markdown",
"id": "1765d8cb",
"metadata": {
"cell_marker": "\"\"\"",
"lines_to_next_cell": 1
},
"source": [
"## Enhanced Tensor Class\n",
"\n",
"This implementation shows how to create dual-purpose educational content:\n",
"\n",
"### For Self-Learning Students\n",
"- Rich explanations and step-by-step guidance\n",
"- Detailed hints and examples\n",
"- Progressive difficulty with scaffolding\n",
"\n",
"### For Formal Assessment\n",
"- Auto-graded with hidden tests\n",
"- Immediate feedback on correctness\n",
"- Partial credit for complex methods"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "aff9a0f2",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"#| export\n",
"class Tensor:\n",
" \"\"\"\n",
" TinyTorch Tensor: N-dimensional array with ML operations.\n",
" \n",
" This enhanced version demonstrates dual-purpose educational content\n",
" suitable for both self-learning and formal assessment.\n",
" \"\"\"\n",
" \n",
" def __init__(self, data: Union[int, float, List, np.ndarray], dtype: Optional[str] = None):\n",
" \"\"\"\n",
" Create a new tensor from data.\n",
" \n",
" Args:\n",
" data: Input data (scalar, list, or numpy array)\n",
" dtype: Data type ('float32', 'int32', etc.). Defaults to auto-detect.\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Use np.array() to convert input data to numpy array\n",
" #| solution_test: tensor.shape should match input shape\n",
" #| difficulty: easy\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" if isinstance(data, (int, float)):\n",
" self._data = np.array(data)\n",
" elif isinstance(data, list):\n",
" self._data = np.array(data)\n",
" elif isinstance(data, np.ndarray):\n",
" self._data = data.copy()\n",
" else:\n",
" self._data = np.array(data)\n",
" \n",
" # Apply dtype conversion if specified\n",
" if dtype is not None:\n",
" self._data = self._data.astype(dtype)\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
" \n",
" @property\n",
" def data(self) -> np.ndarray:\n",
" \"\"\"Access underlying numpy array.\"\"\"\n",
" #| exercise_start\n",
" #| hint: Return the stored numpy array (_data attribute)\n",
" #| solution_test: tensor.data should return numpy array\n",
" #| difficulty: easy\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
" \n",
" @property\n",
" def shape(self) -> Tuple[int, ...]:\n",
" \"\"\"Get tensor shape.\"\"\"\n",
" #| exercise_start\n",
" #| hint: Use the .shape attribute of the numpy array\n",
" #| solution_test: tensor.shape should return tuple of dimensions\n",
" #| difficulty: easy\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
" \n",
" @property\n",
" def size(self) -> int:\n",
" \"\"\"Get total number of elements.\"\"\"\n",
" #| exercise_start\n",
" #| hint: Use the .size attribute of the numpy array\n",
" #| solution_test: tensor.size should return total element count\n",
" #| difficulty: easy\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
" \n",
" @property\n",
" def dtype(self) -> np.dtype:\n",
" \"\"\"Get data type as numpy dtype.\"\"\"\n",
" #| exercise_start\n",
" #| hint: Use the .dtype attribute of the numpy array\n",
" #| solution_test: tensor.dtype should return numpy dtype\n",
" #| difficulty: easy\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
" \n",
" def __repr__(self) -> str:\n",
" \"\"\"String representation of the tensor.\"\"\"\n",
" #| exercise_start\n",
" #| hint: Format as \"Tensor([data], shape=shape, dtype=dtype)\"\n",
" #| solution_test: repr should include data, shape, and dtype\n",
" #| difficulty: medium\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" return f\"Tensor({data_str}, shape={self.shape}, dtype={self.dtype})\"\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
" \n",
" def add(self, other: 'Tensor') -> 'Tensor':\n",
" \"\"\"\n",
" Add two tensors element-wise.\n",
" \n",
" Args:\n",
" other: Another tensor to add\n",
" \n",
" Returns:\n",
" New tensor with element-wise sum\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Use numpy's + operator for element-wise addition\n",
" #| solution_test: result should be new Tensor with correct values\n",
" #| difficulty: medium\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" return Tensor(result_data)\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
" \n",
" def multiply(self, other: 'Tensor') -> 'Tensor':\n",
" \"\"\"\n",
" Multiply two tensors element-wise.\n",
" \n",
" Args:\n",
" other: Another tensor to multiply\n",
" \n",
" Returns:\n",
" New tensor with element-wise product\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Use numpy's * operator for element-wise multiplication\n",
" #| solution_test: result should be new Tensor with correct values\n",
" #| difficulty: medium\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" return Tensor(result_data)\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end\n",
" \n",
" def matmul(self, other: 'Tensor') -> 'Tensor':\n",
" \"\"\"\n",
" Matrix multiplication of two tensors.\n",
" \n",
" Args:\n",
" other: Another tensor for matrix multiplication\n",
" \n",
" Returns:\n",
" New tensor with matrix product\n",
" \n",
" Raises:\n",
" ValueError: If shapes are incompatible for matrix multiplication\n",
" \"\"\"\n",
" #| exercise_start\n",
" #| hint: Use np.dot() for matrix multiplication, check shapes first\n",
" #| solution_test: result should handle shape validation and matrix multiplication\n",
" #| difficulty: hard\n",
" \n",
" ### BEGIN SOLUTION\n",
" # YOUR CODE HERE\n",
" raise NotImplementedError()\n",
" if len(self.shape) != 2 or len(other.shape) != 2:\n",
" raise ValueError(\"Matrix multiplication requires 2D tensors\")\n",
" \n",
" if self.shape[1] != other.shape[0]:\n",
" raise ValueError(f\"Cannot multiply shapes {self.shape} and {other.shape}\")\n",
" \n",
" result_data = np.dot(self._data, other._data)\n",
" return Tensor(result_data)\n",
" ### END SOLUTION\n",
" \n",
" #| exercise_end"
]
},
{
"cell_type": "markdown",
"id": "90c887d9",
"metadata": {
"cell_marker": "\"\"\"",
"lines_to_next_cell": 1
},
"source": [
"## Hidden Tests for Auto-Grading\n",
"\n",
"These tests are hidden from students but used for automatic grading.\n",
"They provide comprehensive coverage and immediate feedback."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "67d0055f",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"### BEGIN HIDDEN TESTS\n",
"def test_tensor_creation_basic():\n",
" \"\"\"Test basic tensor creation (2 points)\"\"\"\n",
" t = Tensor([1, 2, 3])\n",
" assert t.shape == (3,)\n",
" assert t.data.tolist() == [1, 2, 3]\n",
" assert t.size == 3\n",
"\n",
"def test_tensor_creation_scalar():\n",
" \"\"\"Test scalar tensor creation (2 points)\"\"\"\n",
" t = Tensor(5)\n",
" assert t.shape == ()\n",
" assert t.data.item() == 5\n",
" assert t.size == 1\n",
"\n",
"def test_tensor_creation_2d():\n",
" \"\"\"Test 2D tensor creation (2 points)\"\"\"\n",
" t = Tensor([[1, 2], [3, 4]])\n",
" assert t.shape == (2, 2)\n",
" assert t.data.tolist() == [[1, 2], [3, 4]]\n",
" assert t.size == 4\n",
"\n",
"def test_tensor_dtype():\n",
" \"\"\"Test dtype handling (2 points)\"\"\"\n",
" t = Tensor([1, 2, 3], dtype='float32')\n",
" assert t.dtype == np.float32\n",
" assert t.data.dtype == np.float32\n",
"\n",
"def test_tensor_properties():\n",
" \"\"\"Test tensor properties (2 points)\"\"\"\n",
" t = Tensor([[1, 2, 3], [4, 5, 6]])\n",
" assert t.shape == (2, 3)\n",
" assert t.size == 6\n",
" assert isinstance(t.data, np.ndarray)\n",
"\n",
"def test_tensor_repr():\n",
" \"\"\"Test string representation (2 points)\"\"\"\n",
" t = Tensor([1, 2, 3])\n",
" repr_str = repr(t)\n",
" assert \"Tensor\" in repr_str\n",
" assert \"shape\" in repr_str\n",
" assert \"dtype\" in repr_str\n",
"\n",
"def test_tensor_add():\n",
" \"\"\"Test tensor addition (3 points)\"\"\"\n",
" t1 = Tensor([1, 2, 3])\n",
" t2 = Tensor([4, 5, 6])\n",
" result = t1.add(t2)\n",
" assert result.data.tolist() == [5, 7, 9]\n",
" assert result.shape == (3,)\n",
"\n",
"def test_tensor_multiply():\n",
" \"\"\"Test tensor multiplication (3 points)\"\"\"\n",
" t1 = Tensor([1, 2, 3])\n",
" t2 = Tensor([4, 5, 6])\n",
" result = t1.multiply(t2)\n",
" assert result.data.tolist() == [4, 10, 18]\n",
" assert result.shape == (3,)\n",
"\n",
"def test_tensor_matmul():\n",
" \"\"\"Test matrix multiplication (4 points)\"\"\"\n",
" t1 = Tensor([[1, 2], [3, 4]])\n",
" t2 = Tensor([[5, 6], [7, 8]])\n",
" result = t1.matmul(t2)\n",
" expected = [[19, 22], [43, 50]]\n",
" assert result.data.tolist() == expected\n",
" assert result.shape == (2, 2)\n",
"\n",
"def test_tensor_matmul_error():\n",
" \"\"\"Test matrix multiplication error handling (2 points)\"\"\"\n",
" t1 = Tensor([[1, 2, 3]]) # Shape (1, 3)\n",
" t2 = Tensor([[4, 5]]) # Shape (1, 2)\n",
" \n",
" try:\n",
" t1.matmul(t2)\n",
" assert False, \"Should have raised ValueError\"\n",
" except ValueError as e:\n",
" assert \"Cannot multiply shapes\" in str(e)\n",
"\n",
"def test_tensor_immutability():\n",
" \"\"\"Test that operations create new tensors (2 points)\"\"\"\n",
" t1 = Tensor([1, 2, 3])\n",
" t2 = Tensor([4, 5, 6])\n",
" original_data = t1.data.copy()\n",
" \n",
" result = t1.add(t2)\n",
" \n",
" # Original tensor should be unchanged\n",
" assert np.array_equal(t1.data, original_data)\n",
" # Result should be different object\n",
" assert result is not t1\n",
" assert result.data is not t1.data\n",
"\n",
"### END HIDDEN TESTS"
]
},
{
"cell_type": "markdown",
"id": "636ac01d",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"## Usage Examples\n",
"\n",
"### Self-Learning Mode\n",
"Students work through the educational content step by step:\n",
"\n",
"```python\n",
"# Create tensors\n",
"t1 = Tensor([1, 2, 3])\n",
"t2 = Tensor([4, 5, 6])\n",
"\n",
"# Basic operations\n",
"result = t1.add(t2)\n",
"print(f\"Addition: {result}\")\n",
"\n",
"# Matrix operations\n",
"matrix1 = Tensor([[1, 2], [3, 4]])\n",
"matrix2 = Tensor([[5, 6], [7, 8]])\n",
"product = matrix1.matmul(matrix2)\n",
"print(f\"Matrix multiplication: {product}\")\n",
"```\n",
"\n",
"### Assignment Mode\n",
"Students submit implementations that are automatically graded:\n",
"\n",
"1. **Immediate feedback**: Know if implementation is correct\n",
"2. **Partial credit**: Earn points for each working method\n",
"3. **Hidden tests**: Comprehensive coverage beyond visible examples\n",
"4. **Error handling**: Points for proper edge case handling\n",
"\n",
"### Benefits of Dual System\n",
"\n",
"1. **Single source**: One implementation serves both purposes\n",
"2. **Consistent quality**: Same instructor solutions everywhere\n",
"3. **Flexible assessment**: Choose the right tool for each situation\n",
"4. **Scalable**: Handle large courses with automated feedback\n",
"\n",
"This approach transforms TinyTorch from a learning framework into a complete course management solution."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cd296b25",
"metadata": {},
"outputs": [],
"source": [
"# Test the implementation\n",
"if __name__ == \"__main__\":\n",
" # Basic testing\n",
" t1 = Tensor([1, 2, 3])\n",
" t2 = Tensor([4, 5, 6])\n",
" \n",
" print(f\"t1: {t1}\")\n",
" print(f\"t2: {t2}\")\n",
" print(f\"t1 + t2: {t1.add(t2)}\")\n",
" print(f\"t1 * t2: {t1.multiply(t2)}\")\n",
" \n",
" # Matrix multiplication\n",
" m1 = Tensor([[1, 2], [3, 4]])\n",
" m2 = Tensor([[5, 6], [7, 8]])\n",
" print(f\"Matrix multiplication: {m1.matmul(m2)}\")\n",
" \n",
" print(\"\u2705 Enhanced tensor module working!\") "
]
}
],
"metadata": {
"jupytext": {
"main_language": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

1143
assignments/source/02_activations/02_activations.ipynb Normal file
View File

File diff suppressed because it is too large Load Diff

797
assignments/source/03_layers/03_layers.ipynb Normal file
View File

@@ -0,0 +1,797 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "0a3df1fa",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"# Module 2: Layers - Neural Network Building Blocks\n",
"\n",
"Welcome to the Layers module! This is where neural networks begin. You'll implement the fundamental building blocks that transform tensors.\n",
"\n",
"## Learning Goals\n",
"- Understand layers as functions that transform tensors: `y = f(x)`\n",
"- Implement Dense layers with linear transformations: `y = Wx + b`\n",
"- Use activation functions from the activations module for nonlinearity\n",
"- See how neural networks are just function composition\n",
"- Build intuition before diving into training\n",
"\n",
"## Build \u2192 Use \u2192 Understand\n",
"1. **Build**: Dense layers using activation functions as building blocks\n",
"2. **Use**: Transform tensors and see immediate results\n",
"3. **Understand**: How neural networks transform information\n",
"\n",
"## Module Dependencies\n",
"This module builds on the **activations** module:\n",
"- **activations** \u2192 **layers** \u2192 **networks**\n",
"- Clean separation of concerns: math functions \u2192 layer building blocks \u2192 full networks"
]
},
{
"cell_type": "markdown",
"id": "7ad0cde1",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"## \ud83d\udce6 Where This Code Lives in the Final Package\n",
"\n",
"**Learning Side:** You work in `modules/03_layers/layers_dev.py` \n",
"**Building Side:** Code exports to `tinytorch.core.layers`\n",
"\n",
"```python\n",
"# Final package structure:\n",
"from tinytorch.core.layers import Dense, Conv2D # All layers together!\n",
"from tinytorch.core.activations import ReLU, Sigmoid, Tanh\n",
"from tinytorch.core.tensor import Tensor\n",
"```\n",
"\n",
"**Why this matters:**\n",
"- **Learning:** Focused modules for deep understanding\n",
"- **Production:** Proper organization like PyTorch's `torch.nn`\n",
"- **Consistency:** All layers (Dense, Conv2D) live together in `core.layers`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5e2b163c",
"metadata": {},
"outputs": [],
"source": [
"#| default_exp core.layers\n",
"\n",
"# Setup and imports\n",
"import numpy as np\n",
"import sys\n",
"from typing import Union, Optional, Callable\n",
"import math"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "75eb63f1",
"metadata": {},
"outputs": [],
"source": [
"#| export\n",
"import numpy as np\n",
"import math\n",
"import sys\n",
"from typing import Union, Optional, Callable\n",
"\n",
"# Import from the main package (rock solid foundation)\n",
"from tinytorch.core.tensor import Tensor\n",
"from tinytorch.core.activations import ReLU, Sigmoid, Tanh\n",
"\n",
"# print(\"\ud83d\udd25 TinyTorch Layers Module\")\n",
"# print(f\"NumPy version: {np.__version__}\")\n",
"# print(f\"Python version: {sys.version_info.major}.{sys.version_info.minor}\")\n",
"# print(\"Ready to build neural network layers!\")"
]
},
{
"cell_type": "markdown",
"id": "0d8689a4",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"## Step 1: What is a Layer?\n",
"\n",
"### Definition\n",
"A **layer** is a function that transforms tensors. Think of it as a mathematical operation that takes input data and produces output data:\n",
"\n",
"```\n",
"Input Tensor \u2192 Layer \u2192 Output Tensor\n",
"```\n",
"\n",
"### Why Layers Matter in Neural Networks\n",
"Layers are the fundamental building blocks of all neural networks because:\n",
"- **Modularity**: Each layer has a specific job (linear transformation, nonlinearity, etc.)\n",
"- **Composability**: Layers can be combined to create complex functions\n",
"- **Learnability**: Each layer has parameters that can be learned from data\n",
"- **Interpretability**: Different layers learn different features\n",
"\n",
"### The Fundamental Insight\n",
"**Neural networks are just function composition!**\n",
"```\n",
"x \u2192 Layer1 \u2192 Layer2 \u2192 Layer3 \u2192 y\n",
"```\n",
"\n",
"Each layer transforms the data, and the final output is the composition of all these transformations.\n",
"\n",
"### Real-World Examples\n",
"- **Dense Layer**: Learns linear relationships between features\n",
"- **Convolutional Layer**: Learns spatial patterns in images\n",
"- **Recurrent Layer**: Learns temporal patterns in sequences\n",
"- **Activation Layer**: Adds nonlinearity to make networks powerful\n",
"\n",
"### Visual Intuition\n",
"```\n",
"Input: [1, 2, 3] (3 features)\n",
"Dense Layer: y = Wx + b\n",
"Weights W: [[0.1, 0.2, 0.3],\n",
" [0.4, 0.5, 0.6]] (2\u00d73 matrix)\n",
"Bias b: [0.1, 0.2] (2 values)\n",
"Output: [0.1*1 + 0.2*2 + 0.3*3 + 0.1,\n",
" 0.4*1 + 0.5*2 + 0.6*3 + 0.2] = [1.4, 3.2]\n",
"```\n",
"\n",
"Let's start with the most important layer: **Dense** (also called Linear or Fully Connected)."
]
},
{
"cell_type": "markdown",
"id": "16017609",
"metadata": {
"cell_marker": "\"\"\"",
"lines_to_next_cell": 1
},
"source": [
"## Step 2: Understanding Matrix Multiplication\n",
"\n",
"Before we build layers, let's understand the core operation: **matrix multiplication**. This is what powers all neural network computations.\n",
"\n",
"### Why Matrix Multiplication Matters\n",
"- **Efficiency**: Process multiple inputs at once\n",
"- **Parallelization**: GPU acceleration works great with matrix operations\n",
"- **Batch processing**: Handle multiple samples simultaneously\n",
"- **Mathematical foundation**: Linear algebra is the language of neural networks\n",
"\n",
"### The Math Behind It\n",
"For matrices A (m\u00d7n) and B (n\u00d7p), the result C (m\u00d7p) is:\n",
"```\n",
"C[i,j] = sum(A[i,k] * B[k,j] for k in range(n))\n",
"```\n",
"\n",
"### Visual Example\n",
"```\n",
"A = [[1, 2], B = [[5, 6],\n",
" [3, 4]] [7, 8]]\n",
"\n",
"C = A @ B = [[1*5 + 2*7, 1*6 + 2*8],\n",
" [3*5 + 4*7, 3*6 + 4*8]]\n",
" = [[19, 22],\n",
" [43, 50]]\n",
"```\n",
"\n",
"Let's implement this step by step!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "40630d5d",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"#| export\n",
"def matmul_naive(A: np.ndarray, B: np.ndarray) -> np.ndarray:\n",
" \"\"\"\n",
" Naive matrix multiplication using explicit for-loops.\n",
" \n",
" This helps you understand what matrix multiplication really does!\n",
" \n",
" Args:\n",
" A: Matrix of shape (m, n)\n",
" B: Matrix of shape (n, p)\n",
" \n",
" Returns:\n",
" Matrix of shape (m, p) where C[i,j] = sum(A[i,k] * B[k,j] for k in range(n))\n",
" \n",
" TODO: Implement matrix multiplication using three nested for-loops.\n",
" \n",
" APPROACH:\n",
" 1. Get the dimensions: m, n from A and n2, p from B\n",
" 2. Check that n == n2 (matrices must be compatible)\n",
" 3. Create output matrix C of shape (m, p) filled with zeros\n",
" 4. Use three nested loops:\n",
" - i loop: rows of A (0 to m-1)\n",
" - j loop: columns of B (0 to p-1) \n",
" - k loop: shared dimension (0 to n-1)\n",
" 5. For each (i,j), compute: C[i,j] += A[i,k] * B[k,j]\n",
" \n",
" EXAMPLE:\n",
" A = [[1, 2], B = [[5, 6],\n",
" [3, 4]] [7, 8]]\n",
" \n",
" C[0,0] = A[0,0]*B[0,0] + A[0,1]*B[1,0] = 1*5 + 2*7 = 19\n",
" C[0,1] = A[0,0]*B[0,1] + A[0,1]*B[1,1] = 1*6 + 2*8 = 22\n",
" C[1,0] = A[1,0]*B[0,0] + A[1,1]*B[1,0] = 3*5 + 4*7 = 43\n",
" C[1,1] = A[1,0]*B[0,1] + A[1,1]*B[1,1] = 3*6 + 4*8 = 50\n",
" \n",
" HINTS:\n",
" - Start with C = np.zeros((m, p))\n",
" - Use three nested for loops: for i in range(m): for j in range(p): for k in range(n):\n",
" - Accumulate the sum: C[i,j] += A[i,k] * B[k,j]\n",
" \"\"\"\n",
" raise NotImplementedError(\"Student implementation required\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "445593e1",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"#| hide\n",
"#| export\n",
"def matmul_naive(A: np.ndarray, B: np.ndarray) -> np.ndarray:\n",
" \"\"\"\n",
" Naive matrix multiplication using explicit for-loops.\n",
" \n",
" This helps you understand what matrix multiplication really does!\n",
" \"\"\"\n",
" m, n = A.shape\n",
" n2, p = B.shape\n",
" assert n == n2, f\"Matrix shapes don't match: A({m},{n}) @ B({n2},{p})\"\n",
" \n",
" C = np.zeros((m, p))\n",
" for i in range(m):\n",
" for j in range(p):\n",
" for k in range(n):\n",
" C[i, j] += A[i, k] * B[k, j]\n",
" return C"
]
},
{
"cell_type": "markdown",
"id": "e23b8269",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"### \ud83e\uddea Test Your Matrix Multiplication"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "48fadbe0",
"metadata": {},
"outputs": [],
"source": [
"# Test matrix multiplication\n",
"print(\"Testing matrix multiplication...\")\n",
"\n",
"try:\n",
" # Test case 1: Simple 2x2 matrices\n",
" A = np.array([[1, 2], [3, 4]], dtype=np.float32)\n",
" B = np.array([[5, 6], [7, 8]], dtype=np.float32)\n",
" \n",
" result = matmul_naive(A, B)\n",
" expected = np.array([[19, 22], [43, 50]], dtype=np.float32)\n",
" \n",
" print(f\"\u2705 Matrix A:\\n{A}\")\n",
" print(f\"\u2705 Matrix B:\\n{B}\")\n",
" print(f\"\u2705 Your result:\\n{result}\")\n",
" print(f\"\u2705 Expected:\\n{expected}\")\n",
" \n",
" assert np.allclose(result, expected), \"\u274c Result doesn't match expected!\"\n",
" print(\"\ud83c\udf89 Matrix multiplication works!\")\n",
" \n",
" # Test case 2: Compare with NumPy\n",
" numpy_result = A @ B\n",
" assert np.allclose(result, numpy_result), \"\u274c Doesn't match NumPy result!\"\n",
" print(\"\u2705 Matches NumPy implementation!\")\n",
" \n",
"except Exception as e:\n",
" print(f\"\u274c Error: {e}\")\n",
" print(\"Make sure to implement matmul_naive above!\")"
]
},
{
"cell_type": "markdown",
"id": "3df7433e",
"metadata": {
"cell_marker": "\"\"\"",
"lines_to_next_cell": 1
},
"source": [
"## Step 3: Building the Dense Layer\n",
"\n",
"Now let's build the **Dense layer**, the most fundamental building block of neural networks. A Dense layer performs a linear transformation: `y = Wx + b`\n",
"\n",
"### What is a Dense Layer?\n",
"- **Linear transformation**: `y = Wx + b`\n",
"- **W**: Weight matrix (learnable parameters)\n",
"- **x**: Input tensor\n",
"- **b**: Bias vector (learnable parameters)\n",
"- **y**: Output tensor\n",
"\n",
"### Why Dense Layers Matter\n",
"- **Universal approximation**: Can approximate any function with enough neurons\n",
"- **Feature learning**: Each neuron learns a different feature\n",
"- **Nonlinearity**: When combined with activation functions, becomes very powerful\n",
"- **Foundation**: All other layers build on this concept\n",
"\n",
"### The Math\n",
"For input x of shape (batch_size, input_size):\n",
"- **W**: Weight matrix of shape (input_size, output_size)\n",
"- **b**: Bias vector of shape (output_size)\n",
"- **y**: Output of shape (batch_size, output_size)\n",
"\n",
"### Visual Example\n",
"```\n",
"Input: x = [1, 2, 3] (3 features)\n",
"Weights: W = [[0.1, 0.2], Bias: b = [0.1, 0.2]\n",
" [0.3, 0.4],\n",
" [0.5, 0.6]]\n",
"\n",
"Step 1: Wx = [0.1*1 + 0.3*2 + 0.5*3, 0.2*1 + 0.4*2 + 0.6*3]\n",
" = [2.2, 3.2]\n",
"\n",
"Step 2: y = Wx + b = [2.2 + 0.1, 3.2 + 0.2] = [2.3, 3.4]\n",
"```\n",
"\n",
"Let's implement this!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c98c433e",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"#| export\n",
"class Dense:\n",
" \"\"\"\n",
" Dense (Linear) Layer: y = Wx + b\n",
" \n",
" The fundamental building block of neural networks.\n",
" Performs linear transformation: matrix multiplication + bias addition.\n",
" \n",
" Args:\n",
" input_size: Number of input features\n",
" output_size: Number of output features\n",
" use_bias: Whether to include bias term (default: True)\n",
" use_naive_matmul: Whether to use naive matrix multiplication (for learning)\n",
" \n",
" TODO: Implement the Dense layer with weight initialization and forward pass.\n",
" \n",
" APPROACH:\n",
" 1. Store layer parameters (input_size, output_size, use_bias, use_naive_matmul)\n",
" 2. Initialize weights with small random values (Xavier/Glorot initialization)\n",
" 3. Initialize bias to zeros (if use_bias=True)\n",
" 4. Implement forward pass using matrix multiplication and bias addition\n",
" \n",
" EXAMPLE:\n",
" layer = Dense(input_size=3, output_size=2)\n",
" x = Tensor([[1, 2, 3]]) # batch_size=1, input_size=3\n",
" y = layer(x) # shape: (1, 2)\n",
" \n",
" HINTS:\n",
" - Use np.random.randn() for random initialization\n",
" - Scale weights by sqrt(2/(input_size + output_size)) for Xavier init\n",
" - Store weights and bias as numpy arrays\n",
" - Use matmul_naive or @ operator based on use_naive_matmul flag\n",
" \"\"\"\n",
" \n",
" def __init__(self, input_size: int, output_size: int, use_bias: bool = True, \n",
" use_naive_matmul: bool = False):\n",
" \"\"\"\n",
" Initialize Dense layer with random weights.\n",
" \n",
" Args:\n",
" input_size: Number of input features\n",
" output_size: Number of output features\n",
" use_bias: Whether to include bias term\n",
" use_naive_matmul: Use naive matrix multiplication (for learning)\n",
" \n",
" TODO: \n",
" 1. Store layer parameters (input_size, output_size, use_bias, use_naive_matmul)\n",
" 2. Initialize weights with small random values\n",
" 3. Initialize bias to zeros (if use_bias=True)\n",
" \n",
" STEP-BY-STEP:\n",
" 1. Store the parameters as instance variables\n",
" 2. Calculate scale factor for Xavier initialization: sqrt(2/(input_size + output_size))\n",
" 3. Initialize weights: np.random.randn(input_size, output_size) * scale\n",
" 4. If use_bias=True, initialize bias: np.zeros(output_size)\n",
" 5. If use_bias=False, set bias to None\n",
" \n",
" EXAMPLE:\n",
" Dense(3, 2) creates:\n",
" - weights: shape (3, 2) with small random values\n",
" - bias: shape (2,) with zeros\n",
" \"\"\"\n",
" raise NotImplementedError(\"Student implementation required\")\n",
" \n",
" def forward(self, x: Tensor) -> Tensor:\n",
" \"\"\"\n",
" Forward pass: y = Wx + b\n",
" \n",
" Args:\n",
" x: Input tensor of shape (batch_size, input_size)\n",
" \n",
" Returns:\n",
" Output tensor of shape (batch_size, output_size)\n",
" \n",
" TODO: Implement matrix multiplication and bias addition\n",
" - Use self.use_naive_matmul to choose between NumPy and naive implementation\n",
" - If use_naive_matmul=True, use matmul_naive(x.data, self.weights)\n",
" - If use_naive_matmul=False, use x.data @ self.weights\n",
" - Add bias if self.use_bias=True\n",
" \n",
" STEP-BY-STEP:\n",
" 1. Perform matrix multiplication: Wx\n",
" - If use_naive_matmul: result = matmul_naive(x.data, self.weights)\n",
" - Else: result = x.data @ self.weights\n",
" 2. Add bias if use_bias: result += self.bias\n",
" 3. Return Tensor(result)\n",
" \n",
" EXAMPLE:\n",
" Input x: Tensor([[1, 2, 3]]) # shape (1, 3)\n",
" Weights: shape (3, 2)\n",
" Output: Tensor([[val1, val2]]) # shape (1, 2)\n",
" \n",
" HINTS:\n",
" - x.data gives you the numpy array\n",
" - self.weights is your weight matrix\n",
" - Use broadcasting for bias addition: result + self.bias\n",
" - Return Tensor(result) to wrap the result\n",
" \"\"\"\n",
" raise NotImplementedError(\"Student implementation required\")\n",
" \n",
" def __call__(self, x: Tensor) -> Tensor:\n",
" \"\"\"Make layer callable: layer(x) same as layer.forward(x)\"\"\"\n",
" return self.forward(x)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2afc2026",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"#| hide\n",
"#| export\n",
"class Dense:\n",
" \"\"\"\n",
" Dense (Linear) Layer: y = Wx + b\n",
" \n",
" The fundamental building block of neural networks.\n",
" Performs linear transformation: matrix multiplication + bias addition.\n",
" \"\"\"\n",
" \n",
" def __init__(self, input_size: int, output_size: int, use_bias: bool = True, \n",
" use_naive_matmul: bool = False):\n",
" \"\"\"\n",
" Initialize Dense layer with random weights.\n",
" \n",
" Args:\n",
" input_size: Number of input features\n",
" output_size: Number of output features\n",
" use_bias: Whether to include bias term\n",
" use_naive_matmul: Use naive matrix multiplication (for learning)\n",
" \"\"\"\n",
" # Store parameters\n",
" self.input_size = input_size\n",
" self.output_size = output_size\n",
" self.use_bias = use_bias\n",
" self.use_naive_matmul = use_naive_matmul\n",
" \n",
" # Xavier/Glorot initialization\n",
" scale = np.sqrt(2.0 / (input_size + output_size))\n",
" self.weights = np.random.randn(input_size, output_size).astype(np.float32) * scale\n",
" \n",
" # Initialize bias\n",
" if use_bias:\n",
" self.bias = np.zeros(output_size, dtype=np.float32)\n",
" else:\n",
" self.bias = None\n",
" \n",
" def forward(self, x: Tensor) -> Tensor:\n",
" \"\"\"\n",
" Forward pass: y = Wx + b\n",
" \n",
" Args:\n",
" x: Input tensor of shape (batch_size, input_size)\n",
" \n",
" Returns:\n",
" Output tensor of shape (batch_size, output_size)\n",
" \"\"\"\n",
" # Matrix multiplication\n",
" if self.use_naive_matmul:\n",
" result = matmul_naive(x.data, self.weights)\n",
" else:\n",
" result = x.data @ self.weights\n",
" \n",
" # Add bias\n",
" if self.use_bias:\n",
" result += self.bias\n",
" \n",
" return Tensor(result)\n",
" \n",
" def __call__(self, x: Tensor) -> Tensor:\n",
" \"\"\"Make layer callable: layer(x) same as layer.forward(x)\"\"\"\n",
" return self.forward(x)"
]
},
{
"cell_type": "markdown",
"id": "81d084d3",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"### \ud83e\uddea Test Your Dense Layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "24a4e96b",
"metadata": {},
"outputs": [],
"source": [
"# Test Dense layer\n",
"print(\"Testing Dense layer...\")\n",
"\n",
"try:\n",
" # Test basic Dense layer\n",
" layer = Dense(input_size=3, output_size=2, use_bias=True)\n",
" x = Tensor([[1, 2, 3]]) # batch_size=1, input_size=3\n",
" \n",
" print(f\"\u2705 Input shape: {x.shape}\")\n",
" print(f\"\u2705 Layer weights shape: {layer.weights.shape}\")\n",
" print(f\"\u2705 Layer bias shape: {layer.bias.shape}\")\n",
" \n",
" y = layer(x)\n",
" print(f\"\u2705 Output shape: {y.shape}\")\n",
" print(f\"\u2705 Output: {y}\")\n",
" \n",
" # Test without bias\n",
" layer_no_bias = Dense(input_size=2, output_size=1, use_bias=False)\n",
" x2 = Tensor([[1, 2]])\n",
" y2 = layer_no_bias(x2)\n",
" print(f\"\u2705 No bias output: {y2}\")\n",
" \n",
" # Test naive matrix multiplication\n",
" layer_naive = Dense(input_size=2, output_size=2, use_naive_matmul=True)\n",
" x3 = Tensor([[1, 2]])\n",
" y3 = layer_naive(x3)\n",
" print(f\"\u2705 Naive matmul output: {y3}\")\n",
" \n",
" print(\"\\n\ud83c\udf89 All Dense layer tests passed!\")\n",
" \n",
"except Exception as e:\n",
" print(f\"\u274c Error: {e}\")\n",
" print(\"Make sure to implement the Dense layer above!\")"
]
},
{
"cell_type": "markdown",
"id": "a527c61e",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"## Step 4: Composing Layers with Activations\n",
"\n",
"Now let's see how layers work together! A neural network is just layers composed with activation functions.\n",
"\n",
"### Why Layer Composition Matters\n",
"- **Nonlinearity**: Activation functions make networks powerful\n",
"- **Feature learning**: Each layer learns different levels of features\n",
"- **Universal approximation**: Can approximate any function\n",
"- **Modularity**: Easy to experiment with different architectures\n",
"\n",
"### The Pattern\n",
"```\n",
"Input \u2192 Dense \u2192 Activation \u2192 Dense \u2192 Activation \u2192 Output\n",
"```\n",
"\n",
"### Real-World Example\n",
"```\n",
"Input: [1, 2, 3] (3 features)\n",
"Dense(3\u21922): [1.4, 2.8] (linear transformation)\n",
"ReLU: [1.4, 2.8] (nonlinearity)\n",
"Dense(2\u21921): [3.2] (final prediction)\n",
"```\n",
"\n",
"Let's build a simple network!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "db3611ff",
"metadata": {},
"outputs": [],
"source": [
"# Test layer composition\n",
"print(\"Testing layer composition...\")\n",
"\n",
"try:\n",
" # Create a simple network: Dense \u2192 ReLU \u2192 Dense\n",
" dense1 = Dense(input_size=3, output_size=2)\n",
" relu = ReLU()\n",
" dense2 = Dense(input_size=2, output_size=1)\n",
" \n",
" # Test input\n",
" x = Tensor([[1, 2, 3]])\n",
" print(f\"\u2705 Input: {x}\")\n",
" \n",
" # Forward pass through the network\n",
" h1 = dense1(x)\n",
" print(f\"\u2705 After Dense1: {h1}\")\n",
" \n",
" h2 = relu(h1)\n",
" print(f\"\u2705 After ReLU: {h2}\")\n",
" \n",
" y = dense2(h2)\n",
" print(f\"\u2705 Final output: {y}\")\n",
" \n",
" print(\"\\n\ud83c\udf89 Layer composition works!\")\n",
" print(\"This is how neural networks work: layers + activations!\")\n",
" \n",
"except Exception as e:\n",
" print(f\"\u274c Error: {e}\")\n",
" print(\"Make sure all your layers and activations are working!\")"
]
},
{
"cell_type": "markdown",
"id": "69f75a1f",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"## Step 5: Performance Comparison\n",
"\n",
"Let's compare our naive matrix multiplication with NumPy's optimized version to understand why optimization matters in ML.\n",
"\n",
"### Why Performance Matters\n",
"- **Training time**: Neural networks train for hours/days\n",
"- **Inference speed**: Real-time applications need fast predictions\n",
"- **GPU utilization**: Optimized operations use hardware efficiently\n",
"- **Scalability**: Large models need efficient implementations"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "25fc59d6",
"metadata": {},
"outputs": [],
"source": [
"# Performance comparison\n",
"print(\"Comparing naive vs NumPy matrix multiplication...\")\n",
"\n",
"try:\n",
" import time\n",
" \n",
" # Create test matrices\n",
" A = np.random.randn(100, 100).astype(np.float32)\n",
" B = np.random.randn(100, 100).astype(np.float32)\n",
" \n",
" # Time naive implementation\n",
" start_time = time.time()\n",
" result_naive = matmul_naive(A, B)\n",
" naive_time = time.time() - start_time\n",
" \n",
" # Time NumPy implementation\n",
" start_time = time.time()\n",
" result_numpy = A @ B\n",
" numpy_time = time.time() - start_time\n",
" \n",
" print(f\"\u2705 Naive time: {naive_time:.4f} seconds\")\n",
" print(f\"\u2705 NumPy time: {numpy_time:.4f} seconds\")\n",
" print(f\"\u2705 Speedup: {naive_time/numpy_time:.1f}x faster\")\n",
" \n",
" # Verify correctness\n",
" assert np.allclose(result_naive, result_numpy), \"Results don't match!\"\n",
" print(\"\u2705 Results are identical!\")\n",
" \n",
" print(\"\\n\ud83d\udca1 This is why we use optimized libraries in production!\")\n",
" \n",
"except Exception as e:\n",
" print(f\"\u274c Error: {e}\")"
]
},
{
"cell_type": "markdown",
"id": "ca2216d4",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"## \ud83c\udfaf Module Summary\n",
"\n",
"Congratulations! You've built the foundation of neural network layers:\n",
"\n",
"### What You've Accomplished\n",
"\u2705 **Matrix Multiplication**: Understanding the core operation \n",
"\u2705 **Dense Layer**: Linear transformation with weights and bias \n",
"\u2705 **Layer Composition**: Combining layers with activations \n",
"\u2705 **Performance Awareness**: Understanding optimization importance \n",
"\u2705 **Testing**: Immediate feedback on your implementations \n",
"\n",
"### Key Concepts You've Learned\n",
"- **Layers** are functions that transform tensors\n",
"- **Matrix multiplication** powers all neural network computations\n",
"- **Dense layers** perform linear transformations: `y = Wx + b`\n",
"- **Layer composition** creates complex functions from simple building blocks\n",
"- **Performance** matters for real-world ML applications\n",
"\n",
"### What's Next\n",
"In the next modules, you'll build on this foundation:\n",
"- **Networks**: Compose layers into complete models\n",
"- **Training**: Learn parameters with gradients and optimization\n",
"- **Convolutional layers**: Process spatial data like images\n",
"- **Recurrent layers**: Process sequential data like text\n",
"\n",
"### Real-World Connection\n",
"Your Dense layer is now ready to:\n",
"- Learn patterns in data through weight updates\n",
"- Transform features for classification and regression\n",
"- Serve as building blocks for complex architectures\n",
"- Integrate with the rest of the TinyTorch ecosystem\n",
"\n",
"**Ready for the next challenge?** Let's move on to building complete neural networks!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b8fef297",
"metadata": {},
"outputs": [],
"source": [
"# Final verification\n",
"print(\"\\n\" + \"=\"*50)\n",
"print(\"\ud83c\udf89 LAYERS MODULE COMPLETE!\")\n",
"print(\"=\"*50)\n",
"print(\"\u2705 Matrix multiplication understanding\")\n",
"print(\"\u2705 Dense layer implementation\")\n",
"print(\"\u2705 Layer composition with activations\")\n",
"print(\"\u2705 Performance awareness\")\n",
"print(\"\u2705 Comprehensive testing\")\n",
"print(\"\\n\ud83d\ude80 Ready to build networks in the next module!\") "
]
}
],
"metadata": {
"jupytext": {
"main_language": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

1437
assignments/source/04_networks/04_networks.ipynb Normal file
View File

File diff suppressed because it is too large Load Diff

816
assignments/source/05_cnn/05_cnn.ipynb Normal file
View File

@@ -0,0 +1,816 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "ca53839c",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"# Module X: CNN - Convolutional Neural Networks\n",
"\n",
"Welcome to the CNN module! Here you'll implement the core building block of modern computer vision: the convolutional layer.\n",
"\n",
"## Learning Goals\n",
"- Understand the convolution operation (sliding window, local connectivity, weight sharing)\n",
"- Implement Conv2D with explicit for-loops\n",
"- Visualize how convolution builds feature maps\n",
"- Compose Conv2D with other layers to build a simple ConvNet\n",
"- (Stretch) Explore stride, padding, pooling, and multi-channel input\n",
"\n",
"## Build \u2192 Use \u2192 Understand\n",
"1. **Build**: Conv2D layer using sliding window convolution\n",
"2. **Use**: Transform images and see feature maps\n",
"3. **Understand**: How CNNs learn spatial patterns"
]
},
{
"cell_type": "markdown",
"id": "9e0d8f02",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"## \ud83d\udce6 Where This Code Lives in the Final Package\n",
"\n",
"**Learning Side:** You work in `modules/cnn/cnn_dev.py` \n",
"**Building Side:** Code exports to `tinytorch.core.layers`\n",
"\n",
"```python\n",
"# Final package structure:\n",
"from tinytorch.core.layers import Dense, Conv2D # Both layers together!\n",
"from tinytorch.core.activations import ReLU\n",
"from tinytorch.core.tensor import Tensor\n",
"```\n",
"\n",
"**Why this matters:**\n",
"- **Learning:** Focused modules for deep understanding\n",
"- **Production:** Proper organization like PyTorch's `torch.nn`\n",
"- **Consistency:** All layers (Dense, Conv2D) live together in `core.layers`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fbd717db",
"metadata": {},
"outputs": [],
"source": [
"#| default_exp core.cnn"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7f22e530",
"metadata": {},
"outputs": [],
"source": [
"#| export\n",
"import numpy as np\n",
"from typing import List, Tuple, Optional\n",
"from tinytorch.core.tensor import Tensor\n",
"\n",
"# Setup and imports (for development)\n",
"import matplotlib.pyplot as plt\n",
"from tinytorch.core.layers import Dense\n",
"from tinytorch.core.activations import ReLU"
]
},
{
"cell_type": "markdown",
"id": "f99723c8",
"metadata": {
"cell_marker": "\"\"\"",
"lines_to_next_cell": 1
},
"source": [
"## Step 1: What is Convolution?\n",
"\n",
"### Definition\n",
"A **convolutional layer** applies a small filter (kernel) across the input, producing a feature map. This operation captures local patterns and is the foundation of modern vision models.\n",
"\n",
"### Why Convolution Matters in Computer Vision\n",
"- **Local connectivity**: Each output value depends only on a small region of the input\n",
"- **Weight sharing**: The same filter is applied everywhere (translation invariance)\n",
"- **Spatial hierarchy**: Multiple layers build increasingly complex features\n",
"- **Parameter efficiency**: Much fewer parameters than fully connected layers\n",
"\n",
"### The Fundamental Insight\n",
"**Convolution is pattern matching!** The kernel learns to detect specific patterns:\n",
"- **Edge detectors**: Find boundaries between objects\n",
"- **Texture detectors**: Recognize surface patterns\n",
"- **Shape detectors**: Identify geometric forms\n",
"- **Feature detectors**: Combine simple patterns into complex features\n",
"\n",
"### Real-World Examples\n",
"- **Image processing**: Detect edges, blur, sharpen\n",
"- **Computer vision**: Recognize objects, faces, text\n",
"- **Medical imaging**: Detect tumors, analyze scans\n",
"- **Autonomous driving**: Identify traffic signs, pedestrians\n",
"\n",
"### Visual Intuition\n",
"```\n",
"Input Image: Kernel: Output Feature Map:\n",
"[1, 2, 3] [1, 0] [1*1+2*0+4*0+5*(-1), 2*1+3*0+5*0+6*(-1)]\n",
"[4, 5, 6] [0, -1] [4*1+5*0+7*0+8*(-1), 5*1+6*0+8*0+9*(-1)]\n",
"[7, 8, 9]\n",
"```\n",
"\n",
"The kernel slides across the input, computing dot products at each position.\n",
"\n",
"### The Math Behind It\n",
"For input I (H\u00d7W) and kernel K (kH\u00d7kW), the output O (out_H\u00d7out_W) is:\n",
"```\n",
"O[i,j] = sum(I[i+di, j+dj] * K[di, dj] for di in range(kH), dj in range(kW))\n",
"```\n",
"\n",
"Let's implement this step by step!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "aa4af055",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"#| export\n",
"def conv2d_naive(input: np.ndarray, kernel: np.ndarray) -> np.ndarray:\n",
" \"\"\"\n",
" Naive 2D convolution (single channel, no stride, no padding).\n",
" \n",
" Args:\n",
" input: 2D input array (H, W)\n",
" kernel: 2D filter (kH, kW)\n",
" Returns:\n",
" 2D output array (H-kH+1, W-kW+1)\n",
" \n",
" TODO: Implement the sliding window convolution using for-loops.\n",
" \n",
" APPROACH:\n",
" 1. Get input dimensions: H, W = input.shape\n",
" 2. Get kernel dimensions: kH, kW = kernel.shape\n",
" 3. Calculate output dimensions: out_H = H - kH + 1, out_W = W - kW + 1\n",
" 4. Create output array: np.zeros((out_H, out_W))\n",
" 5. Use nested loops to slide the kernel:\n",
" - i loop: output rows (0 to out_H-1)\n",
" - j loop: output columns (0 to out_W-1)\n",
" - di loop: kernel rows (0 to kH-1)\n",
" - dj loop: kernel columns (0 to kW-1)\n",
" 6. For each (i,j), compute: output[i,j] += input[i+di, j+dj] * kernel[di, dj]\n",
" \n",
" EXAMPLE:\n",
" Input: [[1, 2, 3], Kernel: [[1, 0],\n",
" [4, 5, 6], [0, -1]]\n",
" [7, 8, 9]]\n",
" \n",
" Output[0,0] = 1*1 + 2*0 + 4*0 + 5*(-1) = 1 - 5 = -4\n",
" Output[0,1] = 2*1 + 3*0 + 5*0 + 6*(-1) = 2 - 6 = -4\n",
" Output[1,0] = 4*1 + 5*0 + 7*0 + 8*(-1) = 4 - 8 = -4\n",
" Output[1,1] = 5*1 + 6*0 + 8*0 + 9*(-1) = 5 - 9 = -4\n",
" \n",
" HINTS:\n",
" - Start with output = np.zeros((out_H, out_W))\n",
" - Use four nested loops: for i in range(out_H): for j in range(out_W): for di in range(kH): for dj in range(kW):\n",
" - Accumulate the sum: output[i,j] += input[i+di, j+dj] * kernel[di, dj]\n",
" \"\"\"\n",
" raise NotImplementedError(\"Student implementation required\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d83b2c10",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"#| hide\n",
"#| export\n",
"def conv2d_naive(input: np.ndarray, kernel: np.ndarray) -> np.ndarray:\n",
" H, W = input.shape\n",
" kH, kW = kernel.shape\n",
" out_H, out_W = H - kH + 1, W - kW + 1\n",
" output = np.zeros((out_H, out_W), dtype=input.dtype)\n",
" for i in range(out_H):\n",
" for j in range(out_W):\n",
" for di in range(kH):\n",
" for dj in range(kW):\n",
" output[i, j] += input[i + di, j + dj] * kernel[di, dj]\n",
" return output"
]
},
{
"cell_type": "markdown",
"id": "454a6bad",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"### \ud83e\uddea Test Your Conv2D Implementation\n",
"\n",
"Try your function on this simple example:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7705032a",
"metadata": {},
"outputs": [],
"source": [
"# Test case for conv2d_naive\n",
"input = np.array([\n",
" [1, 2, 3],\n",
" [4, 5, 6],\n",
" [7, 8, 9]\n",
"], dtype=np.float32)\n",
"kernel = np.array([\n",
" [1, 0],\n",
" [0, -1]\n",
"], dtype=np.float32)\n",
"\n",
"expected = np.array([\n",
" [1*1+2*0+4*0+5*(-1), 2*1+3*0+5*0+6*(-1)],\n",
" [4*1+5*0+7*0+8*(-1), 5*1+6*0+8*0+9*(-1)]\n",
"], dtype=np.float32)\n",
"\n",
"try:\n",
" output = conv2d_naive(input, kernel)\n",
" print(\"\u2705 Input:\\n\", input)\n",
" print(\"\u2705 Kernel:\\n\", kernel)\n",
" print(\"\u2705 Your output:\\n\", output)\n",
" print(\"\u2705 Expected:\\n\", expected)\n",
" assert np.allclose(output, expected), \"\u274c Output does not match expected!\"\n",
" print(\"\ud83c\udf89 conv2d_naive works!\")\n",
"except Exception as e:\n",
" print(f\"\u274c Error: {e}\")\n",
" print(\"Make sure to implement conv2d_naive above!\")"
]
},
{
"cell_type": "markdown",
"id": "53449e22",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"## Step 2: Understanding What Convolution Does\n",
"\n",
"Let's visualize how different kernels detect different patterns:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "05a1ce2c",
"metadata": {},
"outputs": [],
"source": [
"# Visualize different convolution kernels\n",
"print(\"Visualizing different convolution kernels...\")\n",
"\n",
"try:\n",
" # Test different kernels\n",
" test_input = np.array([\n",
" [1, 1, 1, 0, 0],\n",
" [1, 1, 1, 0, 0],\n",
" [1, 1, 1, 0, 0],\n",
" [0, 0, 0, 0, 0],\n",
" [0, 0, 0, 0, 0]\n",
" ], dtype=np.float32)\n",
" \n",
" # Edge detection kernel (horizontal)\n",
" edge_kernel = np.array([\n",
" [1, 1, 1],\n",
" [0, 0, 0],\n",
" [-1, -1, -1]\n",
" ], dtype=np.float32)\n",
" \n",
" # Sharpening kernel\n",
" sharpen_kernel = np.array([\n",
" [0, -1, 0],\n",
" [-1, 5, -1],\n",
" [0, -1, 0]\n",
" ], dtype=np.float32)\n",
" \n",
" # Test edge detection\n",
" edge_output = conv2d_naive(test_input, edge_kernel)\n",
" print(\"\u2705 Edge detection kernel:\")\n",
" print(\" Detects horizontal edges (boundaries between light and dark)\")\n",
" print(\" Output:\\n\", edge_output)\n",
" \n",
" # Test sharpening\n",
" sharpen_output = conv2d_naive(test_input, sharpen_kernel)\n",
" print(\"\u2705 Sharpening kernel:\")\n",
" print(\" Enhances edges and details\")\n",
" print(\" Output:\\n\", sharpen_output)\n",
" \n",
" print(\"\\n\ud83d\udca1 Different kernels detect different patterns!\")\n",
" print(\" Neural networks learn these kernels automatically!\")\n",
" \n",
"except Exception as e:\n",
" print(f\"\u274c Error: {e}\")"
]
},
{
"cell_type": "markdown",
"id": "0b33791b",
"metadata": {
"cell_marker": "\"\"\"",
"lines_to_next_cell": 1
},
"source": [
"## Step 3: Conv2D Layer Class\n",
"\n",
"Now let's wrap your convolution function in a layer class for use in networks. This makes it consistent with other layers like Dense.\n",
"\n",
"### Why Layer Classes Matter\n",
"- **Consistent API**: Same interface as Dense layers\n",
"- **Learnable parameters**: Kernels can be learned from data\n",
"- **Composability**: Can be combined with other layers\n",
"- **Integration**: Works seamlessly with the rest of TinyTorch\n",
"\n",
"### The Pattern\n",
"```\n",
"Input Tensor \u2192 Conv2D \u2192 Output Tensor\n",
"```\n",
"\n",
"Just like Dense layers, but with spatial operations instead of linear transformations."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "118ba687",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"#| export\n",
"class Conv2D:\n",
" \"\"\"\n",
" 2D Convolutional Layer (single channel, single filter, no stride/pad).\n",
" \n",
" Args:\n",
" kernel_size: (kH, kW) - size of the convolution kernel\n",
" \n",
" TODO: Initialize a random kernel and implement the forward pass using conv2d_naive.\n",
" \n",
" APPROACH:\n",
" 1. Store kernel_size as instance variable\n",
" 2. Initialize random kernel with small values\n",
" 3. Implement forward pass using conv2d_naive function\n",
" 4. Return Tensor wrapped around the result\n",
" \n",
" EXAMPLE:\n",
" layer = Conv2D(kernel_size=(2, 2))\n",
" x = Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) # shape (3, 3)\n",
" y = layer(x) # shape (2, 2)\n",
" \n",
" HINTS:\n",
" - Store kernel_size as (kH, kW)\n",
" - Initialize kernel with np.random.randn(kH, kW) * 0.1 (small values)\n",
" - Use conv2d_naive(x.data, self.kernel) in forward pass\n",
" - Return Tensor(result) to wrap the result\n",
" \"\"\"\n",
" def __init__(self, kernel_size: Tuple[int, int]):\n",
" \"\"\"\n",
" Initialize Conv2D layer with random kernel.\n",
" \n",
" Args:\n",
" kernel_size: (kH, kW) - size of the convolution kernel\n",
" \n",
" TODO: \n",
" 1. Store kernel_size as instance variable\n",
" 2. Initialize random kernel with small values\n",
" 3. Scale kernel values to prevent large outputs\n",
" \n",
" STEP-BY-STEP:\n",
" 1. Store kernel_size as self.kernel_size\n",
" 2. Unpack kernel_size into kH, kW\n",
" 3. Initialize kernel: np.random.randn(kH, kW) * 0.1\n",
" 4. Convert to float32 for consistency\n",
" \n",
" EXAMPLE:\n",
" Conv2D((2, 2)) creates:\n",
" - kernel: shape (2, 2) with small random values\n",
" \"\"\"\n",
" raise NotImplementedError(\"Student implementation required\")\n",
" \n",
" def forward(self, x: Tensor) -> Tensor:\n",
" \"\"\"\n",
" Forward pass: apply convolution to input.\n",
" \n",
" Args:\n",
" x: Input tensor of shape (H, W)\n",
" \n",
" Returns:\n",
" Output tensor of shape (H-kH+1, W-kW+1)\n",
" \n",
" TODO: Implement convolution using conv2d_naive function.\n",
" \n",
" STEP-BY-STEP:\n",
" 1. Use conv2d_naive(x.data, self.kernel)\n",
" 2. Return Tensor(result)\n",
" \n",
" EXAMPLE:\n",
" Input x: Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) # shape (3, 3)\n",
" Kernel: shape (2, 2)\n",
" Output: Tensor([[val1, val2], [val3, val4]]) # shape (2, 2)\n",
" \n",
" HINTS:\n",
" - x.data gives you the numpy array\n",
" - self.kernel is your learned kernel\n",
" - Use conv2d_naive(x.data, self.kernel)\n",
" - Return Tensor(result) to wrap the result\n",
" \"\"\"\n",
" raise NotImplementedError(\"Student implementation required\")\n",
" \n",
" def __call__(self, x: Tensor) -> Tensor:\n",
" \"\"\"Make layer callable: layer(x) same as layer.forward(x)\"\"\"\n",
" return self.forward(x)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3e18c382",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"#| hide\n",
"#| export\n",
"class Conv2D:\n",
" def __init__(self, kernel_size: Tuple[int, int]):\n",
" self.kernel_size = kernel_size\n",
" kH, kW = kernel_size\n",
" # Initialize with small random values\n",
" self.kernel = np.random.randn(kH, kW).astype(np.float32) * 0.1\n",
" \n",
" def forward(self, x: Tensor) -> Tensor:\n",
" return Tensor(conv2d_naive(x.data, self.kernel))\n",
" \n",
" def __call__(self, x: Tensor) -> Tensor:\n",
" return self.forward(x)"
]
},
{
"cell_type": "markdown",
"id": "e288fb18",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"### \ud83e\uddea Test Your Conv2D Layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2f1a4a6a",
"metadata": {},
"outputs": [],
"source": [
"# Test Conv2D layer\n",
"print(\"Testing Conv2D layer...\")\n",
"\n",
"try:\n",
" # Test basic Conv2D layer\n",
" conv = Conv2D(kernel_size=(2, 2))\n",
" x = Tensor(np.array([\n",
" [1, 2, 3],\n",
" [4, 5, 6],\n",
" [7, 8, 9]\n",
" ], dtype=np.float32))\n",
" \n",
" print(f\"\u2705 Input shape: {x.shape}\")\n",
" print(f\"\u2705 Kernel shape: {conv.kernel.shape}\")\n",
" print(f\"\u2705 Kernel values:\\n{conv.kernel}\")\n",
" \n",
" y = conv(x)\n",
" print(f\"\u2705 Output shape: {y.shape}\")\n",
" print(f\"\u2705 Output: {y}\")\n",
" \n",
" # Test with different kernel size\n",
" conv2 = Conv2D(kernel_size=(3, 3))\n",
" y2 = conv2(x)\n",
" print(f\"\u2705 3x3 kernel output shape: {y2.shape}\")\n",
" \n",
" print(\"\\n\ud83c\udf89 Conv2D layer works!\")\n",
" \n",
"except Exception as e:\n",
" print(f\"\u274c Error: {e}\")\n",
" print(\"Make sure to implement the Conv2D layer above!\")"
]
},
{
"cell_type": "markdown",
"id": "97939763",
"metadata": {
"cell_marker": "\"\"\"",
"lines_to_next_cell": 1
},
"source": [
"## Step 4: Building a Simple ConvNet\n",
"\n",
"Now let's compose Conv2D layers with other layers to build a complete convolutional neural network!\n",
"\n",
"### Why ConvNets Matter\n",
"- **Spatial hierarchy**: Each layer learns increasingly complex features\n",
"- **Parameter sharing**: Same kernel applied everywhere (efficiency)\n",
"- **Translation invariance**: Can recognize objects regardless of position\n",
"- **Real-world success**: Power most modern computer vision systems\n",
"\n",
"### The Architecture\n",
"```\n",
"Input Image \u2192 Conv2D \u2192 ReLU \u2192 Flatten \u2192 Dense \u2192 Output\n",
"```\n",
"\n",
"This simple architecture can learn to recognize patterns in images!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "51631fe6",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"#| export\n",
"def flatten(x: Tensor) -> Tensor:\n",
" \"\"\"\n",
" Flatten a 2D tensor to 1D (for connecting to Dense).\n",
" \n",
" TODO: Implement flattening operation.\n",
" \n",
" APPROACH:\n",
" 1. Get the numpy array from the tensor\n",
" 2. Use .flatten() to convert to 1D\n",
" 3. Add batch dimension with [None, :]\n",
" 4. Return Tensor wrapped around the result\n",
" \n",
" EXAMPLE:\n",
" Input: Tensor([[1, 2], [3, 4]]) # shape (2, 2)\n",
" Output: Tensor([[1, 2, 3, 4]]) # shape (1, 4)\n",
" \n",
" HINTS:\n",
" - Use x.data.flatten() to get 1D array\n",
" - Add batch dimension: result[None, :]\n",
" - Return Tensor(result)\n",
" \"\"\"\n",
" raise NotImplementedError(\"Student implementation required\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7e8f2b50",
"metadata": {
"lines_to_next_cell": 1
},
"outputs": [],
"source": [
"#| hide\n",
"#| export\n",
"def flatten(x: Tensor) -> Tensor:\n",
" \"\"\"Flatten a 2D tensor to 1D (for connecting to Dense).\"\"\"\n",
" return Tensor(x.data.flatten()[None, :])"
]
},
{
"cell_type": "markdown",
"id": "7bdb9f80",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"### \ud83e\uddea Test Your Flatten Function"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c6d92ebc",
"metadata": {},
"outputs": [],
"source": [
"# Test flatten function\n",
"print(\"Testing flatten function...\")\n",
"\n",
"try:\n",
" # Test flattening\n",
" x = Tensor([[1, 2, 3], [4, 5, 6]]) # shape (2, 3)\n",
" flattened = flatten(x)\n",
" \n",
" print(f\"\u2705 Input shape: {x.shape}\")\n",
" print(f\"\u2705 Flattened shape: {flattened.shape}\")\n",
" print(f\"\u2705 Flattened values: {flattened}\")\n",
" \n",
" # Verify the flattening worked correctly\n",
" expected = np.array([[1, 2, 3, 4, 5, 6]])\n",
" assert np.allclose(flattened.data, expected), \"\u274c Flattening incorrect!\"\n",
" print(\"\u2705 Flattening works correctly!\")\n",
" \n",
"except Exception as e:\n",
" print(f\"\u274c Error: {e}\")\n",
" print(\"Make sure to implement the flatten function above!\")"
]
},
{
"cell_type": "markdown",
"id": "9804128d",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"## Step 5: Composing a Complete ConvNet\n",
"\n",
"Now let's build a simple convolutional neural network that can process images!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d60d05b9",
"metadata": {},
"outputs": [],
"source": [
"# Compose a simple ConvNet\n",
"print(\"Building a simple ConvNet...\")\n",
"\n",
"try:\n",
" # Create network components\n",
" conv = Conv2D((2, 2))\n",
" relu = ReLU()\n",
" dense = Dense(input_size=4, output_size=1) # 4 features from 2x2 output\n",
" \n",
" # Test input (small 3x3 \"image\")\n",
" x = Tensor(np.random.randn(3, 3).astype(np.float32))\n",
" print(f\"\u2705 Input shape: {x.shape}\")\n",
" print(f\"\u2705 Input: {x}\")\n",
" \n",
" # Forward pass through the network\n",
" conv_out = conv(x)\n",
" print(f\"\u2705 After Conv2D: {conv_out}\")\n",
" \n",
" relu_out = relu(conv_out)\n",
" print(f\"\u2705 After ReLU: {relu_out}\")\n",
" \n",
" flattened = flatten(relu_out)\n",
" print(f\"\u2705 After flatten: {flattened}\")\n",
" \n",
" final_out = dense(flattened)\n",
" print(f\"\u2705 Final output: {final_out}\")\n",
" \n",
" print(\"\\n\ud83c\udf89 Simple ConvNet works!\")\n",
" print(\"This network can learn to recognize patterns in images!\")\n",
" \n",
"except Exception as e:\n",
" print(f\"\u274c Error: {e}\")\n",
" print(\"Check your Conv2D, flatten, and Dense implementations!\")"
]
},
{
"cell_type": "markdown",
"id": "9fe4faf0",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"## Step 6: Understanding the Power of Convolution\n",
"\n",
"Let's see how convolution captures different types of patterns:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "434133c2",
"metadata": {},
"outputs": [],
"source": [
"# Demonstrate pattern detection\n",
"print(\"Demonstrating pattern detection...\")\n",
"\n",
"try:\n",
" # Create a simple \"image\" with a pattern\n",
" image = np.array([\n",
" [0, 0, 0, 0, 0],\n",
" [0, 1, 1, 1, 0],\n",
" [0, 1, 1, 1, 0],\n",
" [0, 1, 1, 1, 0],\n",
" [0, 0, 0, 0, 0]\n",
" ], dtype=np.float32)\n",
" \n",
" # Different kernels detect different patterns\n",
" edge_kernel = np.array([\n",
" [1, 1, 1],\n",
" [1, -8, 1],\n",
" [1, 1, 1]\n",
" ], dtype=np.float32)\n",
" \n",
" blur_kernel = np.array([\n",
" [1/9, 1/9, 1/9],\n",
" [1/9, 1/9, 1/9],\n",
" [1/9, 1/9, 1/9]\n",
" ], dtype=np.float32)\n",
" \n",
" # Test edge detection\n",
" edge_result = conv2d_naive(image, edge_kernel)\n",
" print(\"\u2705 Edge detection:\")\n",
" print(\" Detects boundaries around the white square\")\n",
" print(\" Result:\\n\", edge_result)\n",
" \n",
" # Test blurring\n",
" blur_result = conv2d_naive(image, blur_kernel)\n",
" print(\"\u2705 Blurring:\")\n",
" print(\" Smooths the image\")\n",
" print(\" Result:\\n\", blur_result)\n",
" \n",
" print(\"\\n\ud83d\udca1 Different kernels = different feature detectors!\")\n",
" print(\" Neural networks learn these automatically from data!\")\n",
" \n",
"except Exception as e:\n",
" print(f\"\u274c Error: {e}\")"
]
},
{
"cell_type": "markdown",
"id": "80938b52",
"metadata": {
"cell_marker": "\"\"\""
},
"source": [
"## \ud83c\udfaf Module Summary\n",
"\n",
"Congratulations! You've built the foundation of convolutional neural networks:\n",
"\n",
"### What You've Accomplished\n",
"\u2705 **Convolution Operation**: Understanding the sliding window mechanism \n",
"\u2705 **Conv2D Layer**: Learnable convolutional layer implementation \n",
"\u2705 **Pattern Detection**: Visualizing how kernels detect different features \n",
"\u2705 **ConvNet Architecture**: Composing Conv2D with other layers \n",
"\u2705 **Real-world Applications**: Understanding computer vision applications \n",
"\n",
"### Key Concepts You've Learned\n",
"- **Convolution** is pattern matching with sliding windows\n",
"- **Local connectivity** means each output depends on a small input region\n",
"- **Weight sharing** makes CNNs parameter-efficient\n",
"- **Spatial hierarchy** builds complex features from simple patterns\n",
"- **Translation invariance** allows recognition regardless of position\n",
"\n",
"### What's Next\n",
"In the next modules, you'll build on this foundation:\n",
"- **Advanced CNN features**: Stride, padding, pooling\n",
"- **Multi-channel convolution**: RGB images, multiple filters\n",
"- **Training**: Learning kernels from data\n",
"- **Real applications**: Image classification, object detection\n",
"\n",
"### Real-World Connection\n",
"Your Conv2D layer is now ready to:\n",
"- Learn edge detectors, texture recognizers, and shape detectors\n",
"- Process real images for computer vision tasks\n",
"- Integrate with the rest of the TinyTorch ecosystem\n",
"- Scale to complex architectures like ResNet, VGG, etc.\n",
"\n",
"**Ready for the next challenge?** Let's move on to training these networks!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "03f153f1",
"metadata": {},
"outputs": [],
"source": [
"# Final verification\n",
"print(\"\\n\" + \"=\"*50)\n",
"print(\"\ud83c\udf89 CNN MODULE COMPLETE!\")\n",
"print(\"=\"*50)\n",
"print(\"\u2705 Convolution operation understanding\")\n",
"print(\"\u2705 Conv2D layer implementation\")\n",
"print(\"\u2705 Pattern detection visualization\")\n",
"print(\"\u2705 ConvNet architecture composition\")\n",
"print(\"\u2705 Real-world computer vision context\")\n",
"print(\"\\n\ud83d\ude80 Ready to train networks in the next module!\") "
]
}
],
"metadata": {
"jupytext": {
"main_language": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}