diff --git a/COMPREHENSIVE_MODULE_REVIEW.md b/COMPREHENSIVE_MODULE_REVIEW.md new file mode 100644 index 00000000..f1814338 --- /dev/null +++ b/COMPREHENSIVE_MODULE_REVIEW.md @@ -0,0 +1,453 @@ +# Comprehensive Module Review: Educational Framework Analysis + +**Date**: [Current] +**Reviewer**: Expert PyTorch Developer + Educational Materials Specialist +**Scope**: All 20 modules (01-20) + archived modules + +--- + +## Executive Summary + +Overall, the TinyTorch framework demonstrates **strong progressive building** and **educational clarity**. However, several **anti-patterns** from AI code generation ("reward hacking") have been identified that undermine the educational experience. This review identifies issues and provides actionable fixes. + +### Key Findings + +✅ **Strengths**: +- Clear progressive building from Module 01 → 20 +- Explicit imports in most modules (no star imports) +- Good dependency documentation in recent modules (13, 16, 18) +- Consistent module structure and organization + +⚠️ **Issues Found**: +1. **`import_previous_module` helper function** - Used in Modules 02, 04, 06 (problematic pattern) +2. **Try/except fallbacks** - Some modules silently handle missing dependencies +3. **Inconsistent error handling** - Some modules raise educational errors, others fail silently +4. **Module 20 capstone** - Optional imports handled with None assignments (should be explicit) + +--- + +## Detailed Module-by-Module Review + +### ✅ Module 01: Tensor (Foundation) +**Status**: ✅ EXCELLENT +- **Imports**: Clean, no dependencies (foundation module) +- **Progressive Building**: N/A (foundation) +- **Educational Clarity**: Excellent documentation +- **Issues**: None + +**Pattern**: +```python +import numpy as np +# No dependencies - this is the foundation +``` + +--- + +### ⚠️ Module 02: Activations +**Status**: ⚠️ NEEDS FIX +- **Imports**: Uses `import_previous_module` helper function +- **Progressive Building**: Should import Tensor directly +- **Educational Clarity**: Good, but helper function obscures dependencies +- **Issues**: + - `import_previous_module()` function uses `sys.path.append()` (line 828) + - Used in test code, but pattern is problematic + +**Current Pattern** (PROBLEMATIC): +```python +def import_previous_module(module_name: str, component_name: str): + import sys + import os + sys.path.append(os.path.join(os.path.dirname(__file__), '..', module_name)) + module = __import__(f"{module_name.split('_')[1]}_dev") + return getattr(module, component_name) +``` + +**Should Be**: +```python +from tinytorch.core.tensor import Tensor +``` + +**Recommendation**: Remove `import_previous_module` helper. Use direct imports from `tinytorch` package. + +--- + +### ✅ Module 03: Layers +**Status**: ✅ GOOD (with minor note) +- **Imports**: Direct imports from `tinytorch.core.tensor` and `tinytorch.core.activations` +- **Progressive Building**: Clear dependencies +- **Educational Clarity**: Excellent +- **Issues**: + - Commented-out `import_previous_module` function (line 750) - should be removed entirely + - Uses `import_previous_module` in test code (line 781) - should use direct imports + +**Current Pattern**: +```python +from tinytorch.core.tensor import Tensor +from tinytorch.core.activations import ReLU, Sigmoid +``` + +**Recommendation**: Remove commented-out `import_previous_module` code and update test code to use direct imports. + +--- + +### ⚠️ Module 04: Losses +**Status**: ⚠️ NEEDS FIX +- **Imports**: Uses `import_previous_module` helper function +- **Progressive Building**: Should import Tensor, Linear, ReLU directly +- **Educational Clarity**: Good documentation, but helper obscures dependencies +- **Issues**: + - `import_previous_module()` function defined (line 84) + - Uses `sys.path.append()` pattern + - However, ALSO has direct imports (lines 92-94) - inconsistent! + +**Current Pattern** (INCONSISTENT): +```python +def import_previous_module(module_name: str, component_name: str): + # ... sys.path.append pattern ... + +# Import from tinytorch package +from tinytorch.core.tensor import Tensor +from tinytorch.core.layers import Linear +from tinytorch.core.activations import ReLU +``` + +**Recommendation**: Remove `import_previous_module` function entirely. Keep only direct imports. + +--- + +### ✅ Module 05: Autograd +**Status**: ✅ GOOD (with acceptable exception) +- **Imports**: Direct imports from tinytorch package +- **Progressive Building**: Clear dependencies +- **Educational Clarity**: Excellent +- **Issues**: + - Try/except ImportError that silently passes (line 1149) - **ACCEPTABLE** for optional dependencies during development + - This is fine because it's handling optional monkey-patching of loss functions + +**Current Pattern** (ACCEPTABLE): +```python +try: + # Monkey-patch loss functions for autograd tracking + BinaryCrossEntropyLoss.forward = tracked_bce_forward + # ... +except ImportError: + # Activations/losses not yet available (happens during module development) + pass +``` + +**Recommendation**: Keep as-is. This is acceptable for optional dependencies during development. + +--- + +### ⚠️ Module 06: Optimizers +**Status**: ⚠️ NEEDS FIX +- **Imports**: Direct imports in main code, but uses `import_previous_module` in test code +- **Progressive Building**: Clear in main code +- **Educational Clarity**: Good +- **Issues**: + - Uses `import_previous_module` in test code (lines 1261-1264) + - Commented-out function definition (line 1228) + +**Current Pattern** (TEST CODE ISSUE): +```python +# In test code: +Tensor = import_previous_module('01_tensor', 'Tensor') +Linear = import_previous_module('03_layers', 'Linear') +ReLU = import_previous_module('02_activations', 'ReLU') +MSELoss = import_previous_module('04_losses', 'MSELoss') +``` + +**Recommendation**: Update test code to use direct imports: +```python +from tinytorch.core.tensor import Tensor +from tinytorch.core.layers import Linear +from tinytorch.core.activations import ReLU +from tinytorch.core.losses import MSELoss +``` + +--- + +### ✅ Module 07: Training +**Status**: ✅ EXCELLENT +- **Imports**: Direct imports, clean +- **Progressive Building**: Clear dependencies +- **Educational Clarity**: Excellent +- **Issues**: None (commented-out `import_previous_module` should be removed) + +**Pattern**: +```python +from tinytorch.core.tensor import Tensor +from tinytorch.core.layers import Linear +from tinytorch.core.losses import MSELoss, CrossEntropyLoss +from tinytorch.core.optimizers import SGD, AdamW +``` + +--- + +### ✅ Module 08: DataLoader +**Status**: ✅ EXCELLENT +- **Imports**: Direct imports, clean +- **Progressive Building**: Clear (only needs Tensor) +- **Educational Clarity**: Excellent +- **Issues**: None + +--- + +### ✅ Module 09: Spatial +**Status**: ✅ EXCELLENT +- **Imports**: Direct imports, clean +- **Progressive Building**: Clear (only needs Tensor) +- **Educational Clarity**: Excellent +- **Issues**: None + +--- + +### ✅ Module 10: Tokenization +**Status**: ✅ EXCELLENT +- **Imports**: Direct imports, clean +- **Progressive Building**: Clear (only needs Tensor) +- **Educational Clarity**: Excellent +- **Issues**: None + +--- + +### ✅ Module 11: Embeddings +**Status**: ✅ EXCELLENT +- **Imports**: Direct imports, clean +- **Progressive Building**: Clear (only needs Tensor) +- **Educational Clarity**: Excellent +- **Issues**: None + +--- + +### ✅ Module 12: Attention +**Status**: ✅ EXCELLENT +- **Imports**: Direct imports, clean +- **Progressive Building**: Clear dependencies (Tensor, Linear) +- **Educational Clarity**: Excellent +- **Issues**: None + +--- + +### ✅ Module 13: Transformers +**Status**: ✅ EXCELLENT (RECENTLY FIXED) +- **Imports**: Direct imports with educational error messages +- **Progressive Building**: Excellent dependency documentation +- **Educational Clarity**: Excellent +- **Issues**: None (fixed in recent hot fixes) + +**Pattern** (EXEMPLARY): +```python +try: + from tinytorch.core.tensor import Tensor # Module 01: Foundation +except ImportError as e: + raise ImportError( + "❌ Module 13 (Transformers) requires Module 01 (Tensor) to be completed first.\n" + " Please complete Module 01 first, then run 'tito module complete 01'.\n" + " Original error: " + str(e) + ) from e +``` + +--- + +### ✅ Module 14: Profiling +**Status**: ✅ EXCELLENT +- **Imports**: Direct imports, clean +- **Progressive Building**: Clear dependencies +- **Educational Clarity**: Excellent +- **Issues**: None + +--- + +### ✅ Module 15: Quantization +**Status**: ✅ EXCELLENT +- **Imports**: Direct imports, clean +- **Progressive Building**: Clear dependencies +- **Educational Clarity**: Excellent +- **Issues**: None + +--- + +### ✅ Module 16: Compression +**Status**: ✅ EXCELLENT (RECENTLY FIXED) +- **Imports**: Direct imports with educational error messages +- **Progressive Building**: Excellent dependency documentation +- **Educational Clarity**: Excellent +- **Issues**: None (fixed in recent hot fixes) +- **Note**: Sequential class defined locally as testing utility (correct) + +--- + +### ✅ Module 17: Memoization +**Status**: ✅ EXCELLENT +- **Imports**: Direct imports, clean +- **Progressive Building**: Clear dependencies +- **Educational Clarity**: Excellent +- **Issues**: None + +--- + +### ✅ Module 18: Acceleration +**Status**: ✅ EXCELLENT (RECENTLY FIXED) +- **Imports**: Direct imports with educational error messages +- **Progressive Building**: Excellent dependency documentation +- **Educational Clarity**: Excellent +- **Issues**: None (fixed in recent hot fixes) + +--- + +### ⚠️ Module 19: Benchmarking +**Status**: ⚠️ ACCEPTABLE (with note) +- **Imports**: Direct imports + optional external dependencies +- **Progressive Building**: Clear +- **Educational Clarity**: Good +- **Issues**: + - Try/except for optional external dependencies (pandas, matplotlib, psutil) - **ACCEPTABLE** + - These are external libraries, not TinyTorch modules + - Creates fallback classes when pandas not available (line 177) - acceptable for optional dependency + +**Current Pattern** (ACCEPTABLE): +```python +try: + import pandas as pd + PANDAS_AVAILABLE = True +except ImportError: + PANDAS_AVAILABLE = False + # Create a simple DataFrame-like class for when pandas is not available + class pd: + class DataFrame: + # ... minimal fallback +``` + +**Recommendation**: Keep as-is. This is acceptable for optional external dependencies. + +--- + +### ⚠️ Module 20: Capstone +**Status**: ⚠️ NEEDS IMPROVEMENT +- **Imports**: Direct imports + try/except for optional features +- **Progressive Building**: Clear +- **Educational Clarity**: Good, but could be more explicit +- **Issues**: + - Try/except that sets things to None (lines 254-266) + - Should raise educational errors instead of silently setting to None + +**Current Pattern** (COULD BE BETTER): +```python +try: + from tinytorch.optimization.quantization import QuantizedLinear +except ImportError: + QuantizedLinear = None # Not available + +try: + from tinytorch.optimization.compression import magnitude_prune, structured_prune +except ImportError: + magnitude_prune = None + structured_prune = None +``` + +**Recommendation**: Either: +1. Make these required dependencies (raise educational errors) +2. Or document clearly that these are optional features + +--- + +## Critical Issues Summary + +### 🔴 HIGH PRIORITY + +1. **Remove `import_previous_module` helper function** + - **Modules**: 02, 04, 06 + - **Impact**: Obscures dependencies, uses non-portable sys.path manipulation + - **Fix**: Replace with direct imports from `tinytorch` package + +2. **Update test code to use direct imports** + - **Modules**: 02, 03, 06 + - **Impact**: Test code should follow same patterns as main code + - **Fix**: Replace `import_previous_module` calls with direct imports + +### 🟡 MEDIUM PRIORITY + +3. **Module 20: Make optional dependencies explicit** + - **Impact**: Students might not understand why features are None + - **Fix**: Either make required or document clearly as optional + +4. **Remove commented-out code** + - **Modules**: 01, 03, 06, 07 + - **Impact**: Clutters codebase + - **Fix**: Remove commented-out `import_previous_module` functions + +--- + +## Recommended Fixes + +### Fix 1: Remove `import_previous_module` from Module 02 +**File**: `modules/02_activations/activations_dev.py` +- Remove function definition (line 828) +- Update test code to use direct imports + +### Fix 2: Remove `import_previous_module` from Module 04 +**File**: `modules/04_losses/losses_dev.py` +- Remove function definition (line 84) +- Keep only direct imports (already present) + +### Fix 3: Update Module 06 test code +**File**: `modules/06_optimizers/optimizers_dev.py` +- Remove commented-out function (line 1228) +- Update test code (lines 1261-1264) to use direct imports + +### Fix 4: Improve Module 20 optional dependencies +**File**: `modules/20_capstone/capstone.py` +- Either make dependencies required (raise educational errors) +- Or document clearly as optional features with clear messaging + +### Fix 5: Clean up commented code +**Files**: Multiple modules +- Remove all commented-out `import_previous_module` functions + +--- + +## Educational Principles Validation + +### ✅ Progressive Building +- **Status**: EXCELLENT +- **Evidence**: Clear dependency chains, modules build on each other logically +- **Note**: Some modules use helper functions that obscure this + +### ✅ Explicit Imports +- **Status**: EXCELLENT (after fixes) +- **Evidence**: No star imports found, direct imports used throughout +- **Note**: Helper functions need to be removed + +### ✅ Educational Clarity +- **Status**: EXCELLENT +- **Evidence**: Good documentation, clear learning objectives +- **Note**: Recent modules (13, 16, 18) have exemplary dependency documentation + +### ✅ No Reward Hacking +- **Status**: GOOD (after recent fixes) +- **Evidence**: Recent fixes removed duplicate classes, fallbacks, hardcoded paths +- **Note**: `import_previous_module` pattern is a form of reward hacking + +--- + +## Conclusion + +The TinyTorch framework is **well-structured and educational**, with recent fixes addressing major issues. The remaining issues are primarily: + +1. **Legacy helper functions** (`import_previous_module`) that should be removed +2. **Test code** that should follow same patterns as main code +3. **Optional dependencies** that should be more explicitly documented + +**Overall Grade**: A- (Excellent, with minor improvements needed) + +**Priority Actions**: +1. Remove `import_previous_module` from Modules 02, 04 +2. Update test code in Modules 02, 03, 06 +3. Improve Module 20 optional dependency handling +4. Clean up commented code + +--- + +**Next Steps**: Implement fixes for high-priority issues. + diff --git a/paper/paper.aux b/paper/paper.aux index 64f0f10b..52eb6b95 100644 --- a/paper/paper.aux +++ b/paper/paper.aux @@ -41,7 +41,7 @@ \newlabel{fig:module-flow@cref}{{[figure][2][]2}{[1][2][]4}{}{}{}} \@writefile{toc}{\contentsline {section}{\numberline {2}Related Work}{4}{section.2}\protected@file@percent } \newlabel{sec:related}{{2}{4}{Related Work}{section.2}{}} -\newlabel{sec:related@cref}{{[section][2][]2}{[1][3][]4}{}{}{}} +\newlabel{sec:related@cref}{{[section][2][]2}{[1][4][]4}{}{}{}} \@writefile{toc}{\contentsline {subsection}{\numberline {2.1}Educational ML Frameworks}{4}{subsection.2.1}\protected@file@percent } \citation{zhang2021dive} \citation{howard2020fastai} @@ -72,16 +72,15 @@ \newlabel{tab:objectives@cref}{{[table][2][]2}{[1][7][]8}{}{}{}} \citation{reddi2020mlperf} \citation{collins1989cognitive} -\citation{perkins1992transfer} \@writefile{toc}{\contentsline {subsection}{\numberline {3.3}Module Structure}{9}{subsection.3.3}\protected@file@percent } \newlabel{subsec:module-pedagogy}{{3.3}{9}{Module Structure}{subsection.3.3}{}} \newlabel{subsec:module-pedagogy@cref}{{[subsection][3][3]3.3}{[1][9][]9}{}{}{}} +\citation{perkins1992transfer} \citation{bruner1960process} \citation{krizhevsky2009cifar,lecun1998gradient} \citation{reddi2020mlperf} \@writefile{toc}{\contentsline {subsection}{\numberline {3.4}Milestone Arcs}{10}{subsection.3.4}\protected@file@percent } \citation{sweller1988cognitive} -\citation{meyer2003threshold} \newlabel{lst:dormant-tensor}{{2}{11}{Module 01: Dormant gradient features}{lstlisting.2}{}} \newlabel{lst:dormant-tensor@cref}{{[listing][2][]2}{[1][11][]11}{}{}{}} \@writefile{lol}{\contentsline {lstlisting}{\numberline {2}{\ignorespaces Module 01: Dormant gradient features.}}{11}{lstlisting.2}\protected@file@percent } @@ -89,128 +88,136 @@ \newlabel{sec:progressive}{{4}{11}{Progressive Disclosure}{section.4}{}} \newlabel{sec:progressive@cref}{{[section][4][]4}{[1][11][]11}{}{}{}} \@writefile{toc}{\contentsline {subsection}{\numberline {4.1}Pattern Implementation}{11}{subsection.4.1}\protected@file@percent } -\newlabel{lst:activation}{{3}{11}{Module 05: Autograd activation}{lstlisting.3}{}} -\newlabel{lst:activation@cref}{{[listing][3][]3}{[1][11][]11}{}{}{}} -\@writefile{lol}{\contentsline {lstlisting}{\numberline {3}{\ignorespaces Module 05: Autograd activation.}}{11}{lstlisting.3}\protected@file@percent } \@writefile{toc}{\contentsline {subsection}{\numberline {4.2}Pedagogical Justification}{11}{subsection.4.2}\protected@file@percent } +\citation{meyer2003threshold} \citation{pytorch04release} -\citation{tensorflow20} -\citation{lave1991situated} \@writefile{lof}{\contentsline {figure}{\numberline {3}{\ignorespaces Progressive activation of Tensor gradient features. Dormant placeholders (gray, dashed) exist from Module 01. Module 05 activates full autograd functionality (orange, solid) via runtime method enhancement.}}{12}{figure.caption.8}\protected@file@percent } \newlabel{fig:progressive-timeline}{{3}{12}{Progressive activation of Tensor gradient features. Dormant placeholders (gray, dashed) exist from Module 01. Module 05 activates full autograd functionality (orange, solid) via runtime method enhancement}{figure.caption.8}{}} \newlabel{fig:progressive-timeline@cref}{{[figure][3][]3}{[1][11][]12}{}{}{}} +\newlabel{lst:activation}{{3}{12}{Module 05: Autograd activation}{lstlisting.3}{}} +\newlabel{lst:activation@cref}{{[listing][3][]3}{[1][11][]12}{}{}{}} +\@writefile{lol}{\contentsline {lstlisting}{\numberline {3}{\ignorespaces Module 05: Autograd activation.}}{12}{lstlisting.3}\protected@file@percent } \@writefile{toc}{\contentsline {subsection}{\numberline {4.3}Production Framework Alignment}{12}{subsection.4.3}\protected@file@percent } -\@writefile{toc}{\contentsline {section}{\numberline {5}Systems-First Integration}{12}{section.5}\protected@file@percent } -\newlabel{sec:systems}{{5}{12}{Systems-First Integration}{section.5}{}} -\newlabel{sec:systems@cref}{{[section][5][]5}{[1][12][]12}{}{}{}} -\@writefile{toc}{\contentsline {subsection}{\numberline {5.1}Phase 1: Understanding Memory Through Profiling}{12}{subsection.5.1}\protected@file@percent } +\citation{tensorflow20} +\citation{lave1991situated} \citation{kapur2008productive} +\@writefile{toc}{\contentsline {section}{\numberline {5}Systems-First Integration}{13}{section.5}\protected@file@percent } +\newlabel{sec:systems}{{5}{13}{Systems-First Integration}{section.5}{}} +\newlabel{sec:systems@cref}{{[section][5][]5}{[1][13][]13}{}{}{}} +\@writefile{toc}{\contentsline {subsection}{\numberline {5.1}Phase 1: Understanding Memory Through Profiling}{13}{subsection.5.1}\protected@file@percent } \newlabel{lst:conv-explicit}{{4}{13}{Explicit convolution showing 7-nested complexity}{lstlisting.4}{}} \newlabel{lst:conv-explicit@cref}{{[listing][4][]4}{[1][13][]13}{}{}{}} \@writefile{lol}{\contentsline {lstlisting}{\numberline {4}{\ignorespaces Explicit convolution showing 7-nested complexity.}}{13}{lstlisting.4}\protected@file@percent } \@writefile{toc}{\contentsline {subsection}{\numberline {5.2}Phase 2: Analyzing Complexity Through Transparent Implementations}{13}{subsection.5.2}\protected@file@percent } -\@writefile{lot}{\contentsline {table}{\numberline {3}{\ignorespaces Runtime comparison: TinyTorch vs PyTorch (CPU). Pure Python implementations with explicit loops are 100--10,000$\times $ slower, making performance costs visible and optimization meaningful.}}{13}{table.caption.10}\protected@file@percent } -\newlabel{tab:performance}{{3}{13}{Runtime comparison: TinyTorch vs PyTorch (CPU). Pure Python implementations with explicit loops are 100--10,000$\times $ slower, making performance costs visible and optimization meaningful}{table.caption.10}{}} -\newlabel{tab:performance@cref}{{[table][3][]3}{[1][13][]13}{}{}{}} -\@writefile{toc}{\contentsline {subsection}{\numberline {5.3}Phase 3: Optimizing Systems Through Measurement-Driven Iteration}{13}{subsection.5.3}\protected@file@percent } -\citation{banbury2021widening} +\@writefile{lot}{\contentsline {table}{\numberline {3}{\ignorespaces Runtime comparison: TinyTorch vs PyTorch (CPU). Pure Python implementations with explicit loops are 100--10,000$\times $ slower, making performance costs visible and optimization meaningful.}}{14}{table.caption.10}\protected@file@percent } +\newlabel{tab:performance}{{3}{14}{Runtime comparison: TinyTorch vs PyTorch (CPU). Pure Python implementations with explicit loops are 100--10,000$\times $ slower, making performance costs visible and optimization meaningful}{table.caption.10}{}} +\newlabel{tab:performance@cref}{{[table][3][]3}{[1][14][]14}{}{}{}} +\@writefile{toc}{\contentsline {subsection}{\numberline {5.3}Phase 3: Optimizing Systems Through Measurement-Driven Iteration}{14}{subsection.5.3}\protected@file@percent } \@writefile{toc}{\contentsline {section}{\numberline {6}Deployment and Infrastructure}{14}{section.6}\protected@file@percent } \newlabel{sec:deployment}{{6}{14}{Deployment and Infrastructure}{section.6}{}} \newlabel{sec:deployment@cref}{{[section][6][]6}{[1][14][]14}{}{}{}} \@writefile{toc}{\contentsline {subsection}{\numberline {6.1}Integration Models}{14}{subsection.6.1}\protected@file@percent } \newlabel{subsec:integration}{{6.1}{14}{Integration Models}{subsection.6.1}{}} \newlabel{subsec:integration@cref}{{[subsection][1][6]6.1}{[1][14][]14}{}{}{}} -\@writefile{toc}{\contentsline {subsection}{\numberline {6.2}Infrastructure and Accessibility}{14}{subsection.6.2}\protected@file@percent } -\newlabel{subsec:infrastructure}{{6.2}{14}{Infrastructure and Accessibility}{subsection.6.2}{}} -\newlabel{subsec:infrastructure@cref}{{[subsection][2][6]6.2}{[1][14][]14}{}{}{}} -\citation{blank2019nbgrader} +\citation{banbury2021widening} +\@writefile{toc}{\contentsline {subsection}{\numberline {6.2}Infrastructure and Accessibility}{15}{subsection.6.2}\protected@file@percent } +\newlabel{subsec:infrastructure}{{6.2}{15}{Infrastructure and Accessibility}{subsection.6.2}{}} +\newlabel{subsec:infrastructure@cref}{{[subsection][2][6]6.2}{[1][15][]15}{}{}{}} \newlabel{lst:nbgrader-example}{{5}{15}{NBGrader cell metadata and solution structure}{lstlisting.5}{}} \newlabel{lst:nbgrader-example@cref}{{[listing][5][]5}{[1][15][]15}{}{}{}} \@writefile{lol}{\contentsline {lstlisting}{\numberline {5}{\ignorespaces NBGrader cell metadata and solution structure.}}{15}{lstlisting.5}\protected@file@percent } \@writefile{toc}{\contentsline {subsubsection}{\numberline {6.2.1}Jupyter Environment Options}{15}{subsubsection.6.2.1}\protected@file@percent } \@writefile{toc}{\contentsline {subsubsection}{\numberline {6.2.2}NBGrader Autograding Workflow}{15}{subsubsection.6.2.2}\protected@file@percent } -\@writefile{toc}{\contentsline {subsection}{\numberline {6.3}Automated Assessment Infrastructure}{15}{subsection.6.3}\protected@file@percent } -\@writefile{toc}{\contentsline {subsection}{\numberline {6.4}Package Organization}{15}{subsection.6.4}\protected@file@percent } -\newlabel{subsec:package}{{6.4}{15}{Package Organization}{subsection.6.4}{}} -\newlabel{subsec:package@cref}{{[subsection][4][6]6.4}{[1][15][]15}{}{}{}} +\citation{blank2019nbgrader} \citation{howard2020fastai} \citation{collins1989cognitive} +\@writefile{toc}{\contentsline {subsection}{\numberline {6.3}Automated Assessment Infrastructure}{16}{subsection.6.3}\protected@file@percent } +\@writefile{toc}{\contentsline {subsection}{\numberline {6.4}Package Organization}{16}{subsection.6.4}\protected@file@percent } +\newlabel{subsec:package}{{6.4}{16}{Package Organization}{subsection.6.4}{}} +\newlabel{subsec:package@cref}{{[subsection][4][6]6.4}{[1][16][]16}{}{}{}} \newlabel{lst:progressive-imports}{{6}{16}{Progressive imports: Framework capabilities grow module-by-module}{lstlisting.6}{}} \newlabel{lst:progressive-imports@cref}{{[listing][6][]6}{[1][16][]16}{}{}{}} \@writefile{lol}{\contentsline {lstlisting}{\numberline {6}{\ignorespaces Progressive imports: Framework capabilities grow module-by-module.}}{16}{lstlisting.6}\protected@file@percent } -\@writefile{toc}{\contentsline {subsection}{\numberline {6.5}Connection Maps and Knowledge Integration}{16}{subsection.6.5}\protected@file@percent } \newlabel{lst:connection-map}{{7}{16}{Module 05 connection map}{lstlisting.7}{}} \newlabel{lst:connection-map@cref}{{[listing][7][]7}{[1][16][]16}{}{}{}} \@writefile{lol}{\contentsline {lstlisting}{\numberline {7}{\ignorespaces Module 05 connection map.}}{16}{lstlisting.7}\protected@file@percent } -\@writefile{toc}{\contentsline {subsection}{\numberline {6.6}Open Source Infrastructure}{16}{subsection.6.6}\protected@file@percent } -\newlabel{subsec:opensource}{{6.6}{16}{Open Source Infrastructure}{subsection.6.6}{}} -\newlabel{subsec:opensource@cref}{{[subsection][6][6]6.6}{[1][16][]16}{}{}{}} -\@writefile{toc}{\contentsline {subsection}{\numberline {6.7}Teaching Assistant Support}{16}{subsection.6.7}\protected@file@percent } -\newlabel{subsec:ta-support}{{6.7}{16}{Teaching Assistant Support}{subsection.6.7}{}} -\newlabel{subsec:ta-support@cref}{{[subsection][7][6]6.7}{[1][16][]16}{}{}{}} +\@writefile{toc}{\contentsline {subsection}{\numberline {6.5}Connection Maps and Knowledge Integration}{16}{subsection.6.5}\protected@file@percent } \citation{kapur2008productive} +\@writefile{toc}{\contentsline {subsection}{\numberline {6.6}Open Source Infrastructure}{17}{subsection.6.6}\protected@file@percent } +\newlabel{subsec:opensource}{{6.6}{17}{Open Source Infrastructure}{subsection.6.6}{}} +\newlabel{subsec:opensource@cref}{{[subsection][6][6]6.6}{[1][17][]17}{}{}{}} +\@writefile{toc}{\contentsline {subsection}{\numberline {6.7}Teaching Assistant Support}{17}{subsection.6.7}\protected@file@percent } +\newlabel{subsec:ta-support}{{6.7}{17}{Teaching Assistant Support}{subsection.6.7}{}} +\newlabel{subsec:ta-support@cref}{{[subsection][7][6]6.7}{[1][17][]17}{}{}{}} \@writefile{toc}{\contentsline {subsection}{\numberline {6.8}Student Learning Support}{17}{subsection.6.8}\protected@file@percent } \newlabel{subsec:student-support}{{6.8}{17}{Student Learning Support}{subsection.6.8}{}} \newlabel{subsec:student-support@cref}{{[subsection][8][6]6.8}{[1][17][]17}{}{}{}} -\@writefile{toc}{\contentsline {section}{\numberline {7}Discussion and Limitations}{17}{section.7}\protected@file@percent } -\newlabel{sec:discussion}{{7}{17}{Discussion and Limitations}{section.7}{}} -\newlabel{sec:discussion@cref}{{[section][7][]7}{[1][17][]17}{}{}{}} -\@writefile{toc}{\contentsline {subsection}{\numberline {7.1}Scope: What's NOT Covered}{17}{subsection.7.1}\protected@file@percent } -\newlabel{subsec:scope}{{7.1}{17}{Scope: What's NOT Covered}{subsection.7.1}{}} -\newlabel{subsec:scope@cref}{{[subsection][1][7]7.1}{[1][17][]17}{}{}{}} +\@writefile{toc}{\contentsline {section}{\numberline {7}Discussion and Limitations}{18}{section.7}\protected@file@percent } +\newlabel{sec:discussion}{{7}{18}{Discussion and Limitations}{section.7}{}} +\newlabel{sec:discussion@cref}{{[section][7][]7}{[1][18][]18}{}{}{}} +\@writefile{toc}{\contentsline {subsection}{\numberline {7.1}Scope: What's NOT Covered}{18}{subsection.7.1}\protected@file@percent } +\newlabel{subsec:scope}{{7.1}{18}{Scope: What's NOT Covered}{subsection.7.1}{}} +\newlabel{subsec:scope@cref}{{[subsection][1][7]7.1}{[1][18][]18}{}{}{}} +\@writefile{toc}{\contentsline {subsection}{\numberline {7.2}Limitations: Understanding Scope}{18}{subsection.7.2}\protected@file@percent } \citation{williams2009roofline} \citation{micikevicius2018mixed} -\@writefile{toc}{\contentsline {subsection}{\numberline {7.2}Limitations: Understanding Scope}{18}{subsection.7.2}\protected@file@percent } -\@writefile{toc}{\contentsline {section}{\numberline {8}Future Work}{18}{section.8}\protected@file@percent } -\newlabel{sec:future-work}{{8}{18}{Future Work}{section.8}{}} -\newlabel{sec:future-work@cref}{{[section][8][]8}{[1][18][]18}{}{}{}} -\@writefile{toc}{\contentsline {subsection}{\numberline {8.1}Systems Extensions: Analytical Models and Simulators}{18}{subsection.8.1}\protected@file@percent } \citation{chakkaravarthy2023astrasim,astrasimsim2020} \citation{strubell2019energy,patterson2021carbon} \citation{banbury2021benchmarking} -\@writefile{toc}{\contentsline {subsection}{\numberline {8.2}Empirical Validation}{19}{subsection.8.2}\protected@file@percent } +\@writefile{toc}{\contentsline {section}{\numberline {8}Future Work}{19}{section.8}\protected@file@percent } +\newlabel{sec:future-work}{{8}{19}{Future Work}{section.8}{}} +\newlabel{sec:future-work@cref}{{[section][8][]8}{[1][19][]19}{}{}{}} +\@writefile{toc}{\contentsline {subsection}{\numberline {8.1}Systems Extensions: Analytical Models and Simulators}{19}{subsection.8.1}\protected@file@percent } +\citation{sweller1988cognitive} +\citation{collins1989cognitive} +\citation{paas1992training} +\citation{sorva2012visual} +\citation{chen2022dlsyscourse} +\@writefile{toc}{\contentsline {subsection}{\numberline {8.2}Empirical Validation Roadmap}{20}{subsection.8.2}\protected@file@percent } +\@writefile{toc}{\contentsline {subsection}{\numberline {8.3}Curriculum Extensions: Fundamentals vs. Production Scope}{21}{subsection.8.3}\protected@file@percent } +\@writefile{toc}{\contentsline {subsection}{\numberline {8.4}Community Building and Adoption}{21}{subsection.8.4}\protected@file@percent } +\@writefile{toc}{\contentsline {section}{\numberline {9}Conclusion}{21}{section.9}\protected@file@percent } +\newlabel{sec:conclusion}{{9}{21}{Conclusion}{section.9}{}} +\newlabel{sec:conclusion@cref}{{[section][9][]9}{[1][21][]21}{}{}{}} \bibstyle{plainnat} \bibdata{references} \bibcite{aho2006compilers}{{1}{2006}{{Aho et~al.}}{{Aho, Lam, Sethi, and Ullman}}} -\@writefile{toc}{\contentsline {subsection}{\numberline {8.3}Curriculum Extensions: Fundamentals vs. Production Scope}{20}{subsection.8.3}\protected@file@percent } -\@writefile{toc}{\contentsline {subsection}{\numberline {8.4}Community Building and Adoption}{20}{subsection.8.4}\protected@file@percent } -\@writefile{toc}{\contentsline {section}{\numberline {9}Conclusion}{20}{section.9}\protected@file@percent } -\newlabel{sec:conclusion}{{9}{20}{Conclusion}{section.9}{}} -\newlabel{sec:conclusion@cref}{{[section][9][]9}{[1][20][]20}{}{}{}} \bibcite{banbury2021benchmarking}{{2}{}{{Banbury et~al.}}{{Banbury, Reddi, Lam, Fu, Fazel, Holleman, Huang, Hurtado, Kanter, Lokhmotov, Patterson, Pau, Seo, Sieracki, Thakker, Verhelst, and Yadav}}} -\bibcite{perkins1992transfer}{{3}{1992}{{Burstein et~al.}}{{Burstein, Henry, Collison, Marczak, Sligar, Watson, Marquez, Abbasalizad-Farhangi, Abbasi, Abd-Allah, Abdoli, Abdollahi, Abdollahpour, Abdulkader, Abrigo, Acharya, Adebayo, Adekanmbi, Adham, Afshari, Aghaali, Ahmadi, Ahmadi, Ahmadpour, Ahmed, Akal, Akinyemi, Alahdab, Alam, Alamene, Alene, Alijanzadeh, Alinia, Alipour, Aljunid, Almalki, Al-Mekhlafi, Altirkawi, Alvis-Guzman, Amegah, Amini, Amit, Anbari, Androudi, Anjomshoa, Ansari, Antonio, Arabloo, Arefi, Aremu, Armoon, Arora, Artaman, Asadi, Asadi-Aliabadi, Ashraf-Ganjouei, Assadi, Ataeinia, Atre, Quintanilla, Ayanore, Azari, Babaee, Babazadeh, Badawi, Bagheri, Bagherzadeh, Baheiraei, Balouchi, Barac, Bassat, Baune, Bayati, Bedi, Beghi, Behzadifar, Behzadifar, Belay, Bell, Bell, Berbada, Bernstein, Bhattacharjee, Bhattarai, Bhutta, Bijani, Bohlouli, Breitborde, Britton, Browne, Nagaraja, Busse, Butt, Car, Cárdenas, Castañeda-Orjuela, Cerin, Chanie, Chatterjee, Chu, Cooper, Costa, Dalal, Dandona, Dandona, Daoud, Daryani, Das~Gupta, Davis, Davis~Weaver, Davitoiu, De~Neve, Demeke, Demoz, Deribe, Desai, Deshpande, Desyibelew, Dey, Dharmaratne, Dhimal, Diaz, Doshmangir, Duraes, Dwyer-Lindgren, Earl, Ebrahimi, Ebrahimpour, Effiong, Eftekhari, Ehsani-Chimeh, El~Sayed, El~Sayed~Zaki, El~Tantawi, El-Khatib, Emamian, Enany, Eskandarieh, Eyawo, Ezalarab, Faramarzi, Fareed, Faridnia, Faro, Fazaeli, Fazlzadeh, Fentahun, Fereshtehnejad, Fernandes, Filip, Fischer, Foigt, Foroutan, Francis, Fukumoto, Fullman, Gallus, Gebre, Gebrehiwot, Gebremeskel, Gessner, Geta, Gething, Ghadimi, Ghadiri, Ghajarzadeh, Ghashghaee, Gill, Gill, Golding, Gomes, Gona, Gopalani, Gorini, Goulart, Graetz, Greaves, Green, Guo, Haj-Mirzaian, Haj-Mirzaian, Hall, Hamidi, Haririan, Haro, Hasankhani, Hasanpoor, Hasanzadeh, Hassankhani, Hassen, Hegazy, Hendrie, Heydarpour, Hird, Hoang, Hollerich, Rad, Hoseini-Ghahfarokhi, Hossain, Hosseini, Hosseinzadeh, Hostiuc, Hostiuc, Househ, Hsairi, Ilesanmi, Imani-Nasab, Iqbal, Irvani, Islam, Islam, Jürisson, Balalami, Jalali, Javidnia, Jayatilleke, Jenabi, Ji, Jobanputra, Johnson, Jonas, Shushtari, Jozwiak, Kabir, Kahsay, Kalani, Kalhor, Karami, Karki, Kasaeian, Kassebaum, Keiyoro, Kemp, Khabiri, Khader, Khafaie, Khan, Khan, Khan, Khang, Khatab, Khater, Khater, Khatony, Khazaei, Khazaei, Khazaei-Pool, Khubchandani, Kianipour, Kim, Kimokoti, Kinyoki, Kisa, Kisa, Kolola, Kosen, Koul, Koyanagi, Kraemer, Krishan, Krohn, Kugbey, Kumar, Kumar, Kumar, Kuupiel, Lacey, Lad, Lami, Larsson, Lee, Leili, Levine, Li, Lim, Listl, Longbottom, Lopez, Lorkowski, Magdeldin, Abd El~Razek, Abd El~Razek, Majeed, Maleki, Malekzadeh, Malta, Mamun, Manafi, Manda, Mansourian, Martins-Melo, Masaka, Massenburg, Maulik, Mayala, Mazidi, McKee, Mehrotra, Mehta, Meles, Mendoza, Menezes, Meretoja, Meretoja, Mestrovic, Miller, Miller-Petrie, Mills, Milne, Mini, Mir, Mirjalali, Mirrakhimov, Mohamadi, Mohammad, Darwesh, Mezerji, Mohammed, Mohammed, Mokdad, Molokhia, Monasta, Moodley, Moosazadeh, Moradi, Moradi, Moradi, Moradi-Lakeh, Moradinazar, Moraga, Morawska, Mosapour, Mousavi, Mueller, Muluneh, Mustafa, Nabavizadeh, Naderi, Nagarajan, Nahvijou, Najafi, Nangia, Ndwandwe, Neamati, Negoi, Negoi, Ngunjiri, Thi~Nguyen, Nguyen, Nguyen, Nielsen, Ningrum, Nirayo, Nixon, Nnaji, Nojomi, Noroozi, Nosratnejad, Noubiap, Motlagh, Ofori-Asenso, Ogbo, Oladimeji, Olagunju, Olfatifar, Olum, Olusanya, Oluwasanu, Onwujekwe, Oren, Ortega-Altamirano, Ortiz, Osarenotor, Osei, Osgood-Zimmerman, Otstavnov, Owolabi, ~, Pagheh, Pakhale, Panda-Jonas, Pandey, Park, Parsian, Pashaei, Patel, Pepito, Pereira, Perkins, Pickering, Pilgrim, Pirestani, Piroozi, Pirsaheb, Plana-Ripoll, Pourjafar, Puri, Qorbani, Quintana, Rabiee, Rabiee, Radfar, Rafiei, Rahim, Rahimi, Rahimi-Movaghar, Rahimzadeh, Rajati, Raju, Ramezankhani, Ranabhat, Rasella, Rashedi, Rawal, Reiner, Renzaho, Rezaei, Rezapour, Riahi, Ribeiro, Roever, Roro, Roser, Roshandel, Roshani, Rostami, Rubagotti, Rubino, Sabour, Sadat, Sadeghi, Saeedi, Safari, Safari-Faramani, Safdarian, Sahebkar, Salahshoor, Salam, Salamati, Salehi, Zahabi, Salimi, Salimzadeh, Salomon, Sambala, Samy, Santric~Milicevic, Jose, Saraswathy, Sarmiento-Suárez, Sartorius, Sathian, Saxena, Sbarra, Schaeffer, Schwebel, Sepanlou, Seyedmousavi, Shaahmadi, Shaikh, Shams-Beyranvand, Shamshirian, Shamsizadeh, Sharafi, Sharif, Sharif-Alhoseini, Sharifi, Sharma, Sharma, Sheikh, Shields, Shigematsu, Shiri, Shiue, Shuval, Siddiqi, Silva, Singh, Sinha, Sisay, Sisay, Sliwa, Smith, Somayaji, Soofi, Soriano, Sreeramareddy, Sudaryanto, Sufiyan, Sykes, Sylaja, Tabarés-Seisdedos, Tabb, Tabuchi, Taveira, Temsah, Terkawi, Tessema, Thankappan, Thirunavukkarasu, To, Tovani-Palone, Tran, Tran, Ullah, Usman, Uthman, Vahedian-Azimi, Valdez, van Boven, Vasankari, Vasseghian, Veisani, Venketasubramanian, Violante, Vladimirov, Vlassov, Vos, Vu, Vujcic, Waheed, Wakefield, Wang, Wang, Wang, Ward, Weintraub, Weldegwergs, Weldesamuel, Westerman, Wiysonge, Wondafrash, Woyczynski, Wu, Xu, Yadegar, Yamada, Yazdi-Feyzabadi, Yilgwan, Yip, Yonemoto, Lebni, Younis, Yousefifard, Yousof, Yu, Yusefzadeh, Zabeh, Moghadam, Bin~Zaman, Zamani, Zandian, Zangeneh, Zerfu, Zhang, Ziapour, Zodpey, Murray, and Hay}}} +\bibcite{bruner1960process}{{3}{1960}{{Bruner}}{{}}} \bibcite{chen2022dlsyscourse}{{4}{2022}{{Chen and Zheng}}{{}}} \bibcite{collins1989cognitive}{{5}{}{{Collins et~al.}}{{Collins, Brown, and Newman}}} -\bibcite{bruner1960process}{{6}{1960}{{Frolli et~al.}}{{Frolli, Cerciello, Ciotola, Ricci, Esposito, and Sica}}} -\bibcite{roberthalf2024talent}{{7}{}{{Heffernan}}{{}}} -\bibcite{hotz2023tinygrad}{{8}{2023}{{Hotz and contributors}}{{}}} -\bibcite{howard2020fastai}{{9}{}{{Howard and Gugger}}{{}}} -\bibcite{johnson2016cs231n}{{10}{2016}{{Johnson et~al.}}{{Johnson, Karpathy, and Fei-Fei}}} -\bibcite{blank2019nbgrader}{{11}{}{{Jupyter et~al.}}{{Jupyter, Blank, Bourgin, Brown, Bussonnier, Frederic, Granger, Griffiths, Hamrick, Kelley, Pacer, Page, Pérez, Ragan-Kelley, Suchow, and Willing}}} -\bibcite{kapur2008productive}{{12}{}{{Kapur}}{{}}} -\bibcite{karpathy2022micrograd}{{13}{2022}{{Karpathy}}{{}}} -\bibcite{krizhevsky2009cifar}{{14}{2009}{{Krizhevsky and Hinton}}{{}}} -\bibcite{lave1991situated}{{15}{}{{Lave and Wenger}}{{}}} -\bibcite{lecun1998gradient}{{16}{}{{Lecun et~al.}}{{Lecun, Bottou, Bengio, and Haffner}}} -\bibcite{meadows2008thinking}{{17}{2008}{{Meadows}}{{}}} -\bibcite{meyer2003threshold}{{18}{2003}{{Meyer and Land}}{{}}} -\bibcite{micikevicius2018mixed}{{19}{}{{Micikevicius et~al.}}{{Micikevicius, Narang, Alben, Diamos, Elsen, Garcia, Ginsburg, Houston, Kuchaiev, Venkatesh, and Wu}}} +\bibcite{roberthalf2024talent}{{6}{}{{Heffernan}}{{}}} +\bibcite{hotz2023tinygrad}{{7}{2023}{{Hotz and contributors}}{{}}} +\bibcite{howard2020fastai}{{8}{}{{Howard and Gugger}}{{}}} +\bibcite{johnson2016cs231n}{{9}{2016}{{Johnson et~al.}}{{Johnson, Karpathy, and Fei-Fei}}} +\bibcite{blank2019nbgrader}{{10}{}{{Jupyter et~al.}}{{Jupyter, Blank, Bourgin, Brown, Bussonnier, Frederic, Granger, Griffiths, Hamrick, Kelley, Pacer, Page, Pérez, Ragan-Kelley, Suchow, and Willing}}} +\bibcite{kapur2008productive}{{11}{}{{Kapur}}{{}}} +\bibcite{karpathy2022micrograd}{{12}{2022}{{Karpathy}}{{}}} +\bibcite{krizhevsky2009cifar}{{13}{2009}{{Krizhevsky and Hinton}}{{}}} +\bibcite{lave1991situated}{{14}{}{{Lave and Wenger}}{{}}} +\bibcite{lecun1998gradient}{{15}{}{{Lecun et~al.}}{{Lecun, Bottou, Bengio, and Haffner}}} +\bibcite{meadows2008thinking}{{16}{2008}{{Meadows}}{{}}} +\bibcite{meyer2003threshold}{{17}{2003}{{Meyer and Land}}{{}}} +\bibcite{micikevicius2018mixed}{{18}{}{{Micikevicius et~al.}}{{Micikevicius, Narang, Alben, Diamos, Elsen, Garcia, Ginsburg, Houston, Kuchaiev, Venkatesh, and Wu}}} +\bibcite{paas1992training}{{19}{1992}{{Paas}}{{}}} \bibcite{patterson2021carbon}{{20}{}{{Patterson et~al.}}{{Patterson, Gonzalez, Le, Liang, Munguia, Rothchild, So, Texier, and Dean}}} -\bibcite{chakkaravarthy2023astrasim}{{21}{}{{Rashidi et~al.}}{{Rashidi, Sridharan, Srinivasan, and Krishna}}} -\bibcite{reddi2024mlsysbook}{{22}{}{{Reddi}}{{}}} -\bibcite{reddi2020mlperf}{{23}{a}{{Reddi et~al.}}{{Reddi, Cheng, Kanter, Mattson, Schmuelling, Wu, Anderson, Breughe, Charlebois, Chou, Chukka, Coleman, Davis, Deng, Diamos, Duke, Fick, Gardner, Hubara, Idgunji, Jablin, Jiao, John, Kanwar, Lee, Liao, Lokhmotov, Massa, Meng, Micikevicius, Osborne, Pekhimenko, Rajan, Sequeira, Sirasao, Sun, Tang, Thomson, Wei, Wu, Xu, Yamada, Yu, Yuan, Zhong, Zhang, and Zhou}}} -\bibcite{banbury2021widening}{{24}{b}{{Reddi et~al.}}{{Reddi, Plancher, Kennedy, Moroney, Warden, Agarwal, Banbury, Banzi, Bennett, Brown, Chitlangia, Ghosal, Grafman, Jaeger, Krishnan, Lam, Leiker, Mann, Mazumder, Pajak, Ramaprasad, Smith, Stewart, and Tingley}}} -\bibcite{rosenblatt1958perceptron}{{25}{}{{Rosenblatt}}{{}}} -\bibcite{rumelhart1986learning}{{26}{}{{Rumelhart et~al.}}{{Rumelhart, Hinton, and Williams}}} -\bibcite{schneider2020minitorch}{{27}{2020}{{Rush}}{{}}} -\bibcite{astrasimsim2020}{{28}{}{{Samajdar et~al.}}{{Samajdar, Joseph, Zhu, Whatmough, Mattina, and Krishna}}} -\bibcite{keller2025ai}{{29}{2025}{{Search}}{{}}} -\bibcite{strubell2019energy}{{30}{}{{Strubell et~al.}}{{Strubell, Ganesh, and McCallum}}} -\bibcite{sweller1988cognitive}{{31}{}{{Sweller}}{{}}} -\bibcite{pytorch04release}{{32}{2018}{{Team}}{{}}} -\bibcite{tensorflow20}{{33}{2019}{{Team}}{{}}} -\bibcite{vaswani2017attention}{{34}{}{{Vaswani et~al.}}{{Vaswani, Shazeer, Parmar, Uszkoreit, Jones, N.Gomez, Kaiser, and Polosukhin}}} -\bibcite{williams2009roofline}{{35}{}{{Williams et~al.}}{{Williams, Waterman, and Patterson}}} -\bibcite{papert1980mindstorms}{{36}{}{{Wooster and Papert}}{{}}} -\bibcite{zhang2021dive}{{37}{}{{Zhang et~al.}}{{Zhang, Lipton, Li, and Smola}}} +\bibcite{perkins1992transfer}{{21}{1992}{{Perkins and Salomon}}{{}}} +\bibcite{chakkaravarthy2023astrasim}{{22}{}{{Rashidi et~al.}}{{Rashidi, Sridharan, Srinivasan, and Krishna}}} +\bibcite{reddi2024mlsysbook}{{23}{}{{Reddi}}{{}}} +\bibcite{reddi2020mlperf}{{24}{a}{{Reddi et~al.}}{{Reddi, Cheng, Kanter, Mattson, Schmuelling, Wu, Anderson, Breughe, Charlebois, Chou, Chukka, Coleman, Davis, Deng, Diamos, Duke, Fick, Gardner, Hubara, Idgunji, Jablin, Jiao, John, Kanwar, Lee, Liao, Lokhmotov, Massa, Meng, Micikevicius, Osborne, Pekhimenko, Rajan, Sequeira, Sirasao, Sun, Tang, Thomson, Wei, Wu, Xu, Yamada, Yu, Yuan, Zhong, Zhang, and Zhou}}} +\bibcite{banbury2021widening}{{25}{b}{{Reddi et~al.}}{{Reddi, Plancher, Kennedy, Moroney, Warden, Agarwal, Banbury, Banzi, Bennett, Brown, Chitlangia, Ghosal, Grafman, Jaeger, Krishnan, Lam, Leiker, Mann, Mazumder, Pajak, Ramaprasad, Smith, Stewart, and Tingley}}} +\bibcite{rosenblatt1958perceptron}{{26}{}{{Rosenblatt}}{{}}} +\bibcite{rumelhart1986learning}{{27}{}{{Rumelhart et~al.}}{{Rumelhart, Hinton, and Williams}}} +\bibcite{schneider2020minitorch}{{28}{2020}{{Rush}}{{}}} +\bibcite{astrasimsim2020}{{29}{}{{Samajdar et~al.}}{{Samajdar, Joseph, Zhu, Whatmough, Mattina, and Krishna}}} +\bibcite{keller2025ai}{{30}{2025}{{Search}}{{}}} +\bibcite{sorva2012visual}{{31}{2012}{{Sorva}}{{}}} +\bibcite{strubell2019energy}{{32}{}{{Strubell et~al.}}{{Strubell, Ganesh, and McCallum}}} +\bibcite{sweller1988cognitive}{{33}{}{{Sweller}}{{}}} +\bibcite{pytorch04release}{{34}{2018}{{Team}}{{}}} +\bibcite{tensorflow20}{{35}{2019}{{Team}}{{}}} +\bibcite{vaswani2017attention}{{36}{}{{Vaswani et~al.}}{{Vaswani, Shazeer, Parmar, Uszkoreit, Jones, N.Gomez, Kaiser, and Polosukhin}}} +\bibcite{williams2009roofline}{{37}{}{{Williams et~al.}}{{Williams, Waterman, and Patterson}}} +\bibcite{papert1980mindstorms}{{38}{}{{Wooster and Papert}}{{}}} +\bibcite{zhang2021dive}{{39}{}{{Zhang et~al.}}{{Zhang, Lipton, Li, and Smola}}} \gdef \@abspage@last{24} diff --git a/paper/paper.bbl b/paper/paper.bbl index b1c4a797..be448467 100644 --- a/paper/paper.bbl +++ b/paper/paper.bbl @@ -1,4 +1,4 @@ -\begin{thebibliography}{37} +\begin{thebibliography}{39} \providecommand{\natexlab}[1]{#1} \providecommand{\url}[1]{\texttt{#1}} \expandafter\ifx\csname urlstyle\endcsname\relax @@ -21,173 +21,11 @@ Colby~R. Banbury, Vijay~Janapa Reddi, Max Lam, William Fu, Amin Fazel, Jeremy \newblock \emph{arXiv preprint arXiv:2003.04821}. \newblock URL \url{http://arxiv.org/abs/2003.04821v4}. -\bibitem[Burstein et~al.(1992)Burstein, Henry, Collison, Marczak, Sligar, - Watson, Marquez, Abbasalizad-Farhangi, Abbasi, Abd-Allah, Abdoli, Abdollahi, - Abdollahpour, Abdulkader, Abrigo, Acharya, Adebayo, Adekanmbi, Adham, - Afshari, Aghaali, Ahmadi, Ahmadi, Ahmadpour, Ahmed, Akal, Akinyemi, Alahdab, - Alam, Alamene, Alene, Alijanzadeh, Alinia, Alipour, Aljunid, Almalki, - Al-Mekhlafi, Altirkawi, Alvis-Guzman, Amegah, Amini, Amit, Anbari, Androudi, - Anjomshoa, Ansari, Antonio, Arabloo, Arefi, Aremu, Armoon, Arora, Artaman, - Asadi, Asadi-Aliabadi, Ashraf-Ganjouei, Assadi, Ataeinia, Atre, Quintanilla, - Ayanore, Azari, Babaee, Babazadeh, Badawi, Bagheri, Bagherzadeh, Baheiraei, - Balouchi, Barac, Bassat, Baune, Bayati, Bedi, Beghi, Behzadifar, Behzadifar, - Belay, Bell, Bell, Berbada, Bernstein, Bhattacharjee, Bhattarai, Bhutta, - Bijani, Bohlouli, Breitborde, Britton, Browne, Nagaraja, Busse, Butt, Car, - Cárdenas, Castañeda-Orjuela, Cerin, Chanie, Chatterjee, Chu, Cooper, Costa, - Dalal, Dandona, Dandona, Daoud, Daryani, Das~Gupta, Davis, Davis~Weaver, - Davitoiu, De~Neve, Demeke, Demoz, Deribe, Desai, Deshpande, Desyibelew, Dey, - Dharmaratne, Dhimal, Diaz, Doshmangir, Duraes, Dwyer-Lindgren, Earl, - Ebrahimi, Ebrahimpour, Effiong, Eftekhari, Ehsani-Chimeh, El~Sayed, - El~Sayed~Zaki, El~Tantawi, El-Khatib, Emamian, Enany, Eskandarieh, Eyawo, - Ezalarab, Faramarzi, Fareed, Faridnia, Faro, Fazaeli, Fazlzadeh, Fentahun, - Fereshtehnejad, Fernandes, Filip, Fischer, Foigt, Foroutan, Francis, - Fukumoto, Fullman, Gallus, Gebre, Gebrehiwot, Gebremeskel, Gessner, Geta, - Gething, Ghadimi, Ghadiri, Ghajarzadeh, Ghashghaee, Gill, Gill, Golding, - Gomes, Gona, Gopalani, Gorini, Goulart, Graetz, Greaves, Green, Guo, - Haj-Mirzaian, Haj-Mirzaian, Hall, Hamidi, Haririan, Haro, Hasankhani, - Hasanpoor, Hasanzadeh, Hassankhani, Hassen, Hegazy, Hendrie, Heydarpour, - Hird, Hoang, Hollerich, Rad, Hoseini-Ghahfarokhi, Hossain, Hosseini, - Hosseinzadeh, Hostiuc, Hostiuc, Househ, Hsairi, Ilesanmi, Imani-Nasab, Iqbal, - Irvani, Islam, Islam, Jürisson, Balalami, Jalali, Javidnia, Jayatilleke, - Jenabi, Ji, Jobanputra, Johnson, Jonas, Shushtari, Jozwiak, Kabir, Kahsay, - Kalani, Kalhor, Karami, Karki, Kasaeian, Kassebaum, Keiyoro, Kemp, Khabiri, - Khader, Khafaie, Khan, Khan, Khan, Khang, Khatab, Khater, Khater, Khatony, - Khazaei, Khazaei, Khazaei-Pool, Khubchandani, Kianipour, Kim, Kimokoti, - Kinyoki, Kisa, Kisa, Kolola, Kosen, Koul, Koyanagi, Kraemer, Krishan, Krohn, - Kugbey, Kumar, Kumar, Kumar, Kuupiel, Lacey, Lad, Lami, Larsson, Lee, Leili, - Levine, Li, Lim, Listl, Longbottom, Lopez, Lorkowski, Magdeldin, Abd - El~Razek, Abd El~Razek, Majeed, Maleki, Malekzadeh, Malta, Mamun, Manafi, - Manda, Mansourian, Martins-Melo, Masaka, Massenburg, Maulik, Mayala, Mazidi, - McKee, Mehrotra, Mehta, Meles, Mendoza, Menezes, Meretoja, Meretoja, - Mestrovic, Miller, Miller-Petrie, Mills, Milne, Mini, Mir, Mirjalali, - Mirrakhimov, Mohamadi, Mohammad, Darwesh, Mezerji, Mohammed, Mohammed, - Mokdad, Molokhia, Monasta, Moodley, Moosazadeh, Moradi, Moradi, Moradi, - Moradi-Lakeh, Moradinazar, Moraga, Morawska, Mosapour, Mousavi, Mueller, - Muluneh, Mustafa, Nabavizadeh, Naderi, Nagarajan, Nahvijou, Najafi, Nangia, - Ndwandwe, Neamati, Negoi, Negoi, Ngunjiri, Thi~Nguyen, Nguyen, Nguyen, - Nielsen, Ningrum, Nirayo, Nixon, Nnaji, Nojomi, Noroozi, Nosratnejad, - Noubiap, Motlagh, Ofori-Asenso, Ogbo, Oladimeji, Olagunju, Olfatifar, Olum, - Olusanya, Oluwasanu, Onwujekwe, Oren, Ortega-Altamirano, Ortiz, Osarenotor, - Osei, Osgood-Zimmerman, Otstavnov, Owolabi, ~, Pagheh, Pakhale, Panda-Jonas, - Pandey, Park, Parsian, Pashaei, Patel, Pepito, Pereira, Perkins, Pickering, - Pilgrim, Pirestani, Piroozi, Pirsaheb, Plana-Ripoll, Pourjafar, Puri, - Qorbani, Quintana, Rabiee, Rabiee, Radfar, Rafiei, Rahim, Rahimi, - Rahimi-Movaghar, Rahimzadeh, Rajati, Raju, Ramezankhani, Ranabhat, Rasella, - Rashedi, Rawal, Reiner, Renzaho, Rezaei, Rezapour, Riahi, Ribeiro, Roever, - Roro, Roser, Roshandel, Roshani, Rostami, Rubagotti, Rubino, Sabour, Sadat, - Sadeghi, Saeedi, Safari, Safari-Faramani, Safdarian, Sahebkar, Salahshoor, - Salam, Salamati, Salehi, Zahabi, Salimi, Salimzadeh, Salomon, Sambala, Samy, - Santric~Milicevic, Jose, Saraswathy, Sarmiento-Suárez, Sartorius, Sathian, - Saxena, Sbarra, Schaeffer, Schwebel, Sepanlou, Seyedmousavi, Shaahmadi, - Shaikh, Shams-Beyranvand, Shamshirian, Shamsizadeh, Sharafi, Sharif, - Sharif-Alhoseini, Sharifi, Sharma, Sharma, Sheikh, Shields, Shigematsu, - Shiri, Shiue, Shuval, Siddiqi, Silva, Singh, Sinha, Sisay, Sisay, Sliwa, - Smith, Somayaji, Soofi, Soriano, Sreeramareddy, Sudaryanto, Sufiyan, Sykes, - Sylaja, Tabarés-Seisdedos, Tabb, Tabuchi, Taveira, Temsah, Terkawi, Tessema, - Thankappan, Thirunavukkarasu, To, Tovani-Palone, Tran, Tran, Ullah, Usman, - Uthman, Vahedian-Azimi, Valdez, van Boven, Vasankari, Vasseghian, Veisani, - Venketasubramanian, Violante, Vladimirov, Vlassov, Vos, Vu, Vujcic, Waheed, - Wakefield, Wang, Wang, Wang, Ward, Weintraub, Weldegwergs, Weldesamuel, - Westerman, Wiysonge, Wondafrash, Woyczynski, Wu, Xu, Yadegar, Yamada, - Yazdi-Feyzabadi, Yilgwan, Yip, Yonemoto, Lebni, Younis, Yousefifard, Yousof, - Yu, Yusefzadeh, Zabeh, Moghadam, Bin~Zaman, Zamani, Zandian, Zangeneh, Zerfu, - Zhang, Ziapour, Zodpey, Murray, and Hay]{perkins1992transfer} -R~Burstein, NJ~Henry, ML~Collison, LB~Marczak, A~Sligar, S~Watson, N~Marquez, - M~Abbasalizad-Farhangi, M~Abbasi, F~Abd-Allah, A~Abdoli, M~Abdollahi, - I~Abdollahpour, RS~Abdulkader, MRM Abrigo, D~Acharya, OM~Adebayo, - V~Adekanmbi, D~Adham, M~Afshari, M~Aghaali, K~Ahmadi, M~Ahmadi, E~Ahmadpour, - R~Ahmed, CG~Akal, JO~Akinyemi, F~Alahdab, N~Alam, GM~Alamene, KA~Alene, - M~Alijanzadeh, C~Alinia, V~Alipour, SM~Aljunid, MJ~Almalki, HM~Al-Mekhlafi, - K~Altirkawi, N~Alvis-Guzman, AK~Amegah, S~Amini, AML Amit, Z~Anbari, - S~Androudi, M~Anjomshoa, F~Ansari, CAT Antonio, J~Arabloo, Z~Arefi, O~Aremu, - B~Armoon, A~Arora, A~Artaman, A~Asadi, M~Asadi-Aliabadi, A~Ashraf-Ganjouei, - R~Assadi, B~Ataeinia, SR~Atre, BPA Quintanilla, MA~Ayanore, S~Azari, - E~Babaee, A~Babazadeh, A~Badawi, S~Bagheri, M~Bagherzadeh, N~Baheiraei, - A~Balouchi, A~Barac, Q~Bassat, BT~Baune, M~Bayati, N~Bedi, E~Beghi, - M~Behzadifar, M~Behzadifar, YB~Belay, B~Bell, ML~Bell, DA~Berbada, - RS~Bernstein, NV~Bhattacharjee, S~Bhattarai, ZA~Bhutta, A~Bijani, S~Bohlouli, - NJK Breitborde, G~Britton, AJ~Browne, SB~Nagaraja, R~Busse, ZA~Butt, J~Car, - R~Cárdenas, CA~Castañeda-Orjuela, E~Cerin, WF~Chanie, P~Chatterjee, DT~Chu, - C~Cooper, VM~Costa, K~Dalal, L~Dandona, R~Dandona, F~Daoud, A~Daryani, - R~Das~Gupta, I~Davis, N~Davis~Weaver, DV~Davitoiu, JW~De~Neve, FM~Demeke, - GT~Demoz, K~Deribe, R~Desai, A~Deshpande, HD~Desyibelew, S~Dey, - SD~Dharmaratne, M~Dhimal, D~Diaz, L~Doshmangir, AR~Duraes, L~Dwyer-Lindgren, - L~Earl, R~Ebrahimi, S~Ebrahimpour, A~Effiong, A~Eftekhari, E~Ehsani-Chimeh, - I~El~Sayed, M~El~Sayed~Zaki, M~El~Tantawi, Z~El-Khatib, MH~Emamian, S~Enany, - S~Eskandarieh, O~Eyawo, M~Ezalarab, M~Faramarzi, M~Fareed, R~Faridnia, - A~Faro, AA~Fazaeli, M~Fazlzadeh, N~Fentahun, SM~Fereshtehnejad, JC~Fernandes, - I~Filip, F~Fischer, NA~Foigt, M~Foroutan, JM~Francis, T~Fukumoto, N~Fullman, - S~Gallus, DG~Gebre, TT~Gebrehiwot, GG~Gebremeskel, BD~Gessner, B~Geta, - PW~Gething, R~Ghadimi, K~Ghadiri, M~Ghajarzadeh, A~Ghashghaee, PS~Gill, - TK~Gill, N~Golding, NGM Gomes, PN~Gona, SV~Gopalani, G~Gorini, BNG Goulart, - N~Graetz, F~Greaves, MS~Green, Y~Guo, A~Haj-Mirzaian, A~Haj-Mirzaian, - BJ~Hall, S~Hamidi, H~Haririan, JM~Haro, M~Hasankhani, E~Hasanpoor, - A~Hasanzadeh, H~Hassankhani, HY~Hassen, MI~Hegazy, D~Hendrie, F~Heydarpour, - TR~Hird, CL~Hoang, G~Hollerich, EH~Rad, M~Hoseini-Ghahfarokhi, N~Hossain, - M~Hosseini, M~Hosseinzadeh, M~Hostiuc, S~Hostiuc, M~Househ, M~Hsairi, - OS~Ilesanmi, MH~Imani-Nasab, U~Iqbal, SSN Irvani, N~Islam, SMS Islam, - M~Jürisson, NJ~Balalami, A~Jalali, J~Javidnia, AU~Jayatilleke, E~Jenabi, - JS~Ji, YB~Jobanputra, K~Johnson, JB~Jonas, ZJ~Shushtari, JJ~Jozwiak, A~Kabir, - A~Kahsay, H~Kalani, R~Kalhor, M~Karami, S~Karki, A~Kasaeian, NJ~Kassebaum, - PN~Keiyoro, GR~Kemp, R~Khabiri, YS~Khader, MA~Khafaie, EA~Khan, J~Khan, - MS~Khan, YH~Khang, K~Khatab, A~Khater, MM~Khater, A~Khatony, M~Khazaei, - S~Khazaei, M~Khazaei-Pool, J~Khubchandani, N~Kianipour, YJ~Kim, RW~Kimokoti, - DK~Kinyoki, A~Kisa, S~Kisa, T~Kolola, S~Kosen, PA~Koul, A~Koyanagi, MUG - Kraemer, K~Krishan, KJ~Krohn, N~Kugbey, GA~Kumar, M~Kumar, P~Kumar, - D~Kuupiel, B~Lacey, SD~Lad, FH~Lami, AO~Larsson, PH~Lee, M~Leili, AJ~Levine, - S~Li, LL~Lim, S~Listl, J~Longbottom, JCF Lopez, S~Lorkowski, S~Magdeldin, - HM~Abd El~Razek, MM~Abd El~Razek, A~Majeed, A~Maleki, R~Malekzadeh, DC~Malta, - AA~Mamun, N~Manafi, AL~Manda, M~Mansourian, FR~Martins-Melo, A~Masaka, - BB~Massenburg, PK~Maulik, BK~Mayala, M~Mazidi, M~McKee, R~Mehrotra, KM~Mehta, - GG~Meles, W~Mendoza, RG~Menezes, A~Meretoja, TJ~Meretoja, T~Mestrovic, - TR~Miller, MK~Miller-Petrie, EJ~Mills, GJ~Milne, GK~Mini, SM~Mir, - H~Mirjalali, EM~Mirrakhimov, E~Mohamadi, DK~Mohammad, AM~Darwesh, NMG - Mezerji, AS~Mohammed, S~Mohammed, AH~Mokdad, M~Molokhia, L~Monasta, - Y~Moodley, M~Moosazadeh, G~Moradi, M~Moradi, Y~Moradi, M~Moradi-Lakeh, - M~Moradinazar, P~Moraga, L~Morawska, A~Mosapour, SM~Mousavi, UO~Mueller, - AG~Muluneh, G~Mustafa, B~Nabavizadeh, M~Naderi, AJ~Nagarajan, A~Nahvijou, - F~Najafi, V~Nangia, DE~Ndwandwe, N~Neamati, I~Negoi, RI~Negoi, JW~Ngunjiri, - HL~Thi~Nguyen, LH~Nguyen, SH~Nguyen, KR~Nielsen, DNA Ningrum, YL~Nirayo, - MR~Nixon, CA~Nnaji, M~Nojomi, M~Noroozi, S~Nosratnejad, JJ~Noubiap, - SN~Motlagh, R~Ofori-Asenso, FA~Ogbo, KE~Oladimeji, AT~Olagunju, M~Olfatifar, - S~Olum, BO~Olusanya, MM~Oluwasanu, OE~Onwujekwe, E~Oren, DDV - Ortega-Altamirano, A~Ortiz, O~Osarenotor, FB~Osei, AE~Osgood-Zimmerman, - SS~Otstavnov, MO~Owolabi, A~M ~, P, AS~Pagheh, S~Pakhale, S~Panda-Jonas, - A~Pandey, EK~Park, H~Parsian, T~Pashaei, SK~Patel, VCF Pepito, A~Pereira, - S~Perkins, BV~Pickering, T~Pilgrim, M~Pirestani, B~Piroozi, M~Pirsaheb, - O~Plana-Ripoll, H~Pourjafar, P~Puri, M~Qorbani, H~Quintana, M~Rabiee, - N~Rabiee, A~Radfar, A~Rafiei, F~Rahim, Z~Rahimi, V~Rahimi-Movaghar, - S~Rahimzadeh, F~Rajati, SB~Raju, A~Ramezankhani, CL~Ranabhat, D~Rasella, - V~Rashedi, L~Rawal, Jr~Reiner, RC, AMN Renzaho, S~Rezaei, A~Rezapour, - SM~Riahi, AI~Ribeiro, L~Roever, EM~Roro, M~Roser, G~Roshandel, D~Roshani, - A~Rostami, E~Rubagotti, S~Rubino, S~Sabour, N~Sadat, E~Sadeghi, R~Saeedi, - Y~Safari, R~Safari-Faramani, M~Safdarian, A~Sahebkar, MR~Salahshoor, N~Salam, - P~Salamati, F~Salehi, SS~Zahabi, Y~Salimi, H~Salimzadeh, JA~Salomon, - EZ~Sambala, AM~Samy, MM~Santric~Milicevic, BPS Jose, SYI Saraswathy, - R~Sarmiento-Suárez, B~Sartorius, B~Sathian, S~Saxena, AN~Sbarra, - LE~Schaeffer, DC~Schwebel, SG~Sepanlou, S~Seyedmousavi, F~Shaahmadi, - MA~Shaikh, M~Shams-Beyranvand, A~Shamshirian, M~Shamsizadeh, K~Sharafi, - M~Sharif, M~Sharif-Alhoseini, H~Sharifi, J~Sharma, R~Sharma, A~Sheikh, - C~Shields, M~Shigematsu, R~Shiri, I~Shiue, K~Shuval, TJ~Siddiqi, JP~Silva, - JA~Singh, DN~Sinha, MM~Sisay, S~Sisay, K~Sliwa, DL~Smith, R~Somayaji, - M~Soofi, JB~Soriano, CT~Sreeramareddy, A~Sudaryanto, MB~Sufiyan, BL~Sykes, - PN~Sylaja, R~Tabarés-Seisdedos, KM~Tabb, T~Tabuchi, N~Taveira, MH~Temsah, - AS~Terkawi, ZT~Tessema, KR~Thankappan, S~Thirunavukkarasu, QG~To, - MR~Tovani-Palone, BX~Tran, KB~Tran, I~Ullah, MS~Usman, OA~Uthman, - A~Vahedian-Azimi, PR~Valdez, JFM van Boven, TJ~Vasankari, Y~Vasseghian, - Y~Veisani, N~Venketasubramanian, FS~Violante, SK~Vladimirov, V~Vlassov, - T~Vos, GT~Vu, IS~Vujcic, Y~Waheed, J~Wakefield, H~Wang, Y~Wang, YP~Wang, - JL~Ward, RG~Weintraub, KG~Weldegwergs, GT~Weldesamuel, R~Westerman, - CS~Wiysonge, DZ~Wondafrash, L~Woyczynski, AM~Wu, G~Xu, A~Yadegar, T~Yamada, - V~Yazdi-Feyzabadi, CS~Yilgwan, P~Yip, N~Yonemoto, JY~Lebni, MZ~Younis, - M~Yousefifard, HSA Yousof, C~Yu, H~Yusefzadeh, E~Zabeh, TZ~Moghadam, - S~Bin~Zaman, M~Zamani, H~Zandian, A~Zangeneh, TA~Zerfu, Y~Zhang, A~Ziapour, - S~Zodpey, CJL Murray, and SI~Hay. -\newblock Mapping 123 million neonatal, infant and child deaths between 2000 - and 2017. -\newblock \emph{Nature}, 574\penalty0 (7778):\penalty0 353--358, 1992. -\newblock ISSN 0028-0836. -\newblock \doi{10.1038/s41586-019-1545-0}. +\bibitem[Bruner(1960)]{bruner1960process} +Jerome~S. Bruner. +\newblock \emph{The Process of Education}. +\newblock Harvard University Press, Cambridge, MA, 1960. +\newblock ISBN 978-0-674-71001-6. \bibitem[Chen and Zheng(2022)]{chen2022dlsyscourse} Tianqi Chen and Zico Zheng. @@ -204,15 +42,6 @@ Allan Collins, John~Seely Brown, and Susan~E. Newman. \newblock \doi{10.4324/9781315044408-14}. \newblock URL \url{https://doi.org/10.4324/9781315044408-14}. -\bibitem[Frolli et~al.(1960)Frolli, Cerciello, Ciotola, Ricci, Esposito, and - Sica]{bruner1960process} -A~Frolli, F~Cerciello, S~Ciotola, MC~Ricci, C~Esposito, and LS~Sica. -\newblock Narrative approach and mentalization. -\newblock \emph{Behavioral sciences (Basel, Switzerland)}, 13\penalty0 (12), - 1960. -\newblock ISSN 2076-328X. -\newblock \doi{10.3390/bs13120994}. - \bibitem[Heffernan()]{roberthalf2024talent} Margaret Heffernan. \newblock Earning half the minimum wage: new report reveals pressures on @@ -310,6 +139,14 @@ Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, \newblock Mixed precision training. \newblock URL \url{http://arxiv.org/abs/1710.03740v3}. +\bibitem[Paas(1992)]{paas1992training} +Fred G. W.~C. Paas. +\newblock Training strategies for attaining transfer of problem-solving skill + in statistics: A cognitive-load approach. +\newblock \emph{Journal of Educational Psychology}, 84\penalty0 (4):\penalty0 + 429--434, 1992. +\newblock \doi{10.1037/0022-0663.84.4.429}. + \bibitem[Patterson et~al.()Patterson, Gonzalez, Le, Liang, Munguia, Rothchild, So, Texier, and Dean]{patterson2021carbon} David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, @@ -318,6 +155,11 @@ David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, \newblock \emph{arXiv preprint arXiv:2104.10350}. \newblock URL \url{http://arxiv.org/abs/2104.10350v3}. +\bibitem[Perkins and Salomon(1992)]{perkins1992transfer} +D.~N. Perkins and Gavriel Salomon. +\newblock Transfer of learning. +\newblock \emph{International Encyclopedia of Education}, 1992. + \bibitem[Rashidi et~al.()Rashidi, Sridharan, Srinivasan, and Krishna]{chakkaravarthy2023astrasim} Saeed Rashidi, Srinivas Sridharan, Sudarshan Srinivasan, and Tushar Krishna. @@ -412,6 +254,12 @@ Keller~Executive Search. \newblock URL \url{https://www.kellerexecutivesearch.com/intelligence/ai-machine-learning-talent-gap-2025/}. +\bibitem[Sorva(2012)]{sorva2012visual} +Juha Sorva. +\newblock \emph{Visual program simulation in introductory programming + education}. +\newblock Doctoral dissertation, Aalto University, Espoo, Finland, 2012. + \bibitem[Strubell et~al.()Strubell, Ganesh, and McCallum]{strubell2019energy} Emma Strubell, Ananya Ganesh, and Andrew McCallum. \newblock Energy and policy considerations for deep learning in nlp. diff --git a/paper/paper.blg b/paper/paper.blg index 8e806f09..e76c0488 100644 --- a/paper/paper.blg +++ b/paper/paper.blg @@ -49,45 +49,45 @@ Warning--empty year in papert1980mindstorms Warning--empty year in papert1980mindstorms Warning--empty year in zhang2021dive Warning--empty year in zhang2021dive -You've used 37 entries, +You've used 39 entries, 2773 wiz_defined-function locations, - 874 strings with 23762 characters, -and the built_in function-call counts, 50306 in all, are: -= -- 3560 -> -- 6924 -< -- 25 -+ -- 2316 -- -- 2278 -* -- 5403 -:= -- 8608 -add.period$ -- 165 -call.type$ -- 37 -change.case$ -- 869 -chr.to.int$ -- 36 -cite$ -- 120 -duplicate$ -- 762 -empty$ -- 1359 -format.name$ -- 2329 -if$ -- 9551 + 879 strings with 15131 characters, +and the built_in function-call counts, 22733 in all, are: += -- 1917 +> -- 1884 +< -- 29 ++ -- 638 +- -- 598 +* -- 2045 +:= -- 3643 +add.period$ -- 169 +call.type$ -- 39 +change.case$ -- 314 +chr.to.int$ -- 38 +cite$ -- 124 +duplicate$ -- 796 +empty$ -- 1410 +format.name$ -- 653 +if$ -- 4599 int.to.chr$ -- 2 int.to.str$ -- 1 -missing$ -- 29 -newline$ -- 242 -num.names$ -- 150 -pop$ -- 1821 +missing$ -- 31 +newline$ -- 250 +num.names$ -- 158 +pop$ -- 719 preamble$ -- 1 -purify$ -- 833 +purify$ -- 277 quote$ -- 0 -skip$ -- 1205 +skip$ -- 667 stack$ -- 0 -substring$ -- 441 -swap$ -- 90 +substring$ -- 445 +swap$ -- 92 text.length$ -- 7 text.prefix$ -- 0 top$ -- 0 -type$ -- 392 +type$ -- 409 warning$ -- 46 -while$ -- 147 +while$ -- 153 width$ -- 0 -write$ -- 557 +write$ -- 579 (There were 46 warnings) diff --git a/paper/paper.out b/paper/paper.out index 367c2a50..5f16cfd6 100644 --- a/paper/paper.out +++ b/paper/paper.out @@ -32,7 +32,7 @@ \BOOKMARK [2][-]{subsection.7.2}{\376\377\000L\000i\000m\000i\000t\000a\000t\000i\000o\000n\000s\000:\000\040\000U\000n\000d\000e\000r\000s\000t\000a\000n\000d\000i\000n\000g\000\040\000S\000c\000o\000p\000e}{section.7}% 32 \BOOKMARK [1][-]{section.8}{\376\377\000F\000u\000t\000u\000r\000e\000\040\000W\000o\000r\000k}{}% 33 \BOOKMARK [2][-]{subsection.8.1}{\376\377\000S\000y\000s\000t\000e\000m\000s\000\040\000E\000x\000t\000e\000n\000s\000i\000o\000n\000s\000:\000\040\000A\000n\000a\000l\000y\000t\000i\000c\000a\000l\000\040\000M\000o\000d\000e\000l\000s\000\040\000a\000n\000d\000\040\000S\000i\000m\000u\000l\000a\000t\000o\000r\000s}{section.8}% 34 -\BOOKMARK [2][-]{subsection.8.2}{\376\377\000E\000m\000p\000i\000r\000i\000c\000a\000l\000\040\000V\000a\000l\000i\000d\000a\000t\000i\000o\000n}{section.8}% 35 +\BOOKMARK [2][-]{subsection.8.2}{\376\377\000E\000m\000p\000i\000r\000i\000c\000a\000l\000\040\000V\000a\000l\000i\000d\000a\000t\000i\000o\000n\000\040\000R\000o\000a\000d\000m\000a\000p}{section.8}% 35 \BOOKMARK [2][-]{subsection.8.3}{\376\377\000C\000u\000r\000r\000i\000c\000u\000l\000u\000m\000\040\000E\000x\000t\000e\000n\000s\000i\000o\000n\000s\000:\000\040\000F\000u\000n\000d\000a\000m\000e\000n\000t\000a\000l\000s\000\040\000v\000s\000.\000\040\000P\000r\000o\000d\000u\000c\000t\000i\000o\000n\000\040\000S\000c\000o\000p\000e}{section.8}% 36 \BOOKMARK [2][-]{subsection.8.4}{\376\377\000C\000o\000m\000m\000u\000n\000i\000t\000y\000\040\000B\000u\000i\000l\000d\000i\000n\000g\000\040\000a\000n\000d\000\040\000A\000d\000o\000p\000t\000i\000o\000n}{section.8}% 37 \BOOKMARK [1][-]{section.9}{\376\377\000C\000o\000n\000c\000l\000u\000s\000i\000o\000n}{}% 38 diff --git a/paper/paper.pdf b/paper/paper.pdf index 7f2e456e..1e36c6f4 100644 Binary files a/paper/paper.pdf and b/paper/paper.pdf differ diff --git a/paper/paper.tex b/paper/paper.tex index 273f9634..a1f86338 100644 --- a/paper/paper.tex +++ b/paper/paper.tex @@ -234,11 +234,13 @@ model.fit(dataloader, epochs=10) class Linear: def __init__(self, in_features, out): + # Memory: out × in_features × 4B self.weight = Tensor.randn( out, in_features) self.bias = Tensor.zeros(out) def forward(self, x): + # O(batch × in × out) FLOPs return (x @ self.weight.T + self.bias) @@ -249,6 +251,7 @@ class Adam: self.lr = lr # 2× optimizer state: # momentum + variance + # Why 2× memory vs SGD? self.m = [Tensor.zeros_like(p) for p in params] self.v = [Tensor.zeros_like(p) @@ -257,10 +260,12 @@ class Adam: def step(self): for i, p in enumerate( self.params): + # Exponential moving avg self.m[i] = (0.9*self.m[i] + 0.1*p.grad) self.v[i] = (0.999*self.v[i] + 0.001*p.grad**2) + # Per-parameter adaptive lr p.data -= (self.lr * self.m[i] / (self.v[i].sqrt() + 1e-8)) @@ -591,6 +596,7 @@ Each milestone: (1) recreates actual breakthroughs using exclusively student cod \noindent\textbf{Quantitative Success Criteria:} To ensure reproducibility and clear learning objectives, each milestone defines explicit success thresholds balancing historical plausibility (recognizing pure-Python performance limits) with pedagogical validation (ensuring implementations are functionally correct, not just syntactically complete): + \begin{itemize}[leftmargin=*, itemsep=1pt, parsep=0pt] \item \textbf{M03 (1958 Perceptron)}: 100\% accuracy on 4-point XOR-like linearly separable problem, demonstrating basic gradient descent convergence. \item \textbf{M06 (1969 XOR Solution)}: 100\% accuracy on XOR classification within 20 epochs, proving multi-layer networks solve non-linear problems. @@ -1026,6 +1032,7 @@ TinyTorch's CPU-only design prioritizes pedagogical transparency, but students b While TinyTorch's design is grounded in established learning theory (cognitive load~\citep{sweller1988cognitive}, progressive disclosure, cognitive apprenticeship~\citep{collins1989cognitive}), its pedagogical effectiveness requires empirical validation through controlled classroom studies. We commit to the following validation roadmap: \noindent\textbf{Phase 1: Pilot Deployment (Fall 2025, $n=30$--50 students)} + \begin{itemize}[leftmargin=*, itemsep=1pt, parsep=0pt] \item \textbf{Institutions}: Deploy at 2--3 universities in introductory ML systems courses as primary hands-on framework alongside theory lectures. \item \textbf{Cognitive load measurement}: Paas Mental Effort Rating Scale~\citep{paas1992training} administered after Modules 05 (autograd) and 09 (CNNs) to test progressive disclosure hypothesis---does dormant feature activation reduce cognitive load compared to introducing autograd as separate framework? @@ -1034,6 +1041,7 @@ While TinyTorch's design is grounded in established learning theory (cognitive l \end{itemize} \noindent\textbf{Phase 2: Comparative Study (Spring 2026, $n=100$--150 students)} + \begin{itemize}[leftmargin=*, itemsep=1pt, parsep=0pt] \item \textbf{Design}: Randomized controlled trial comparing TinyTorch (systems-first, build-from-scratch) versus PyTorch-only (application-first, use-existing-frameworks) versus lecture-only (control) across 3 sections of same ML course with identical theory content. \item \textbf{Conceptual understanding}: ML systems concept inventory (adapted from program visualization assessment~\citep{sorva2012visual} for systems thinking) administered pre-course and post-course, measuring understanding of autograd mechanics, memory profiling, computational complexity, and optimization tradeoffs. @@ -1042,6 +1050,7 @@ While TinyTorch's design is grounded in established learning theory (cognitive l \end{itemize} \noindent\textbf{Phase 3: Longitudinal Tracking (2026--2027, $n=200$+ students)} + \begin{itemize}[leftmargin=*, itemsep=1pt, parsep=0pt] \item \textbf{Retention}: Re-administer concept inventory at 6 months and 12 months post-course to measure long-term retention of systems thinking---does implementation-based learning persist better than lecture-based learning? \item \textbf{Advanced course performance}: Track TinyTorch cohort versus control group performance in subsequent ML systems courses (e.g., CMU Deep Learning Systems~\citep{chen2022dlsyscourse}, distributed training courses) to measure preparation effectiveness.