github-starred/TinyTorch
2
0
Fork 0
mirror of https://github.com/MLSysBook/TinyTorch.git synced 2026-04-28 12:42:33 -05:00
Code Issues 7 Packages Projects Releases Wiki Activity

Evolve pedagogical framework: Build → Use → [Engage] patterns

Browse Source 
- Updated pedagogical principles with refined engagement patterns:
  - Build → Use → Reflect (design & systems thinking)
  - Build → Use → Analyze (technical depth & debugging)
  - Build → Use → Optimize (systems iteration & performance)

- Added pattern selection guide for module developers
- Updated development workflow to choose pattern first
- Created specific module assignments for each pattern
- Enhanced quick reference with pattern-specific activities

This evolution moves beyond passive 'understanding' to active,
specific engagement that matches professional ML engineering skills.
This commit is contained in:
Vijay Janapa Reddi
2025-07-11 18:37:00 -04:00
parent 576ed42a41
commit 53fb514918
3 changed files with 600 additions and 289 deletions
Download Patch File Download Diff File Expand all files Collapse all files

485
docs/development/module-development-guide.md
View File

@@ -1,279 +1,318 @@
# 📖 TinyTorch Module Development Guide
A complete guide for creating educational modules that automatically generate student exercise versions.
**Complete methodology for creating educational modules with real-world ML engineering practices.**
## 🎯 Philosophy
**Write once, teach everywhere.** Create complete, working implementations with embedded pedagogical markers that automatically generate student exercise notebooks.
**"Build → Use → Understand → Repeat"** with real data and immediate feedback.
## 🏗️ Module Development Workflow
Create complete, working implementations that automatically generate student exercise versions while maintaining production-quality exports.
### Step 1: Plan the Learning Journey
Before coding, decide:
- **What should students implement?** (core learning objectives)
- **What should be provided?** (utilities, complex setup, advanced features)
- **What's the difficulty progression?** (easy → medium → hard)
## 🔑 Core Principles
### **Real Data, Real Systems**
- **Use production datasets**: No mock/fake data - students work with CIFAR-10, not synthetic data
- **Show progress feedback**: Downloads, training need visual progress indicators
- **Cache for efficiency**: Download once, use repeatedly
- **Real-world scale**: Use actual dataset sizes, not toy examples
### **Immediate Visual Feedback**
- **Visual confirmation**: Students see their code working (images, plots, results)
- **Development vs. Export separation**: Rich feedback in `_dev.py`, clean exports to package
- **Progress indicators**: Status messages, progress bars for long operations
- **Real-time validation**: Students can verify each step immediately
### **Educational Excellence**
- **Progressive complexity**: Easy → Medium → Hard with clear difficulty indicators
- **Comprehensive guidance**: TODO sections with approach, examples, hints, systems thinking
- **Real-world connections**: Connect every concept to production ML engineering
- **Immediate testing**: Test each component with real inputs as you build
## 🏗️ Development Workflow
### Step 1: Choose the Learning Pattern
- **Select engagement pattern**: Reflect, Analyze, or Optimize?
- **Use the Pattern Selection Guide** from [Pedagogical Principles](../pedagogy/pedagogical-principles.md):
- **Build → Use → Reflect**: Early modules, design decisions, systems thinking
- **Build → Use → Analyze**: Middle modules, technical depth, performance
- **Build → Use → Optimize**: Advanced modules, iteration, production focus
- **Document your choice** with clear rationale
### Step 2: Plan the Learning Journey
- **Define learning objectives**: What should students implement vs. receive?
- **Choose real data**: What production dataset will they use?
- **Design progression**: How does complexity build through the module?
- **Map to production**: How does this connect to real ML systems?
- **Design pattern-specific activities**: Questions, exercises, or challenges
### Step 3: Write Complete Implementation
Create `modules/{module}/{module}_dev.py` with NBDev structure:
### Step 2: Write Complete Implementation
Create `modules/{module}/{module}_dev.py` with:
```python
# ---
# jupyter:
# jupytext:
# text_representation:
# extension: .py
# format_name: percent
# format_version: '1.3'
# jupytext_version: 1.17.1
# ---
# %% [markdown]
# # Module Title: Learning Objectives
#
# Brief description of what students will build and learn.
"""
# Module: {Title} - {Purpose}
## 🎯 Learning Pattern: Build → Use → [Pattern]
**Pattern Choice**: [Reflect/Analyze/Optimize]
**Rationale**: [Why this pattern fits the learning objectives]
**Key Activities**:
- [Pattern-specific activity 1]
- [Pattern-specific activity 2]
- [Pattern-specific activity 3]
## Learning Objectives
- ✅ Build {core_concept} from scratch
- ✅ Use it with real data ({dataset_name})
- ✅ [Engage] through {pattern_specific_activities}
- ✅ Connect to production ML systems
## What You'll Build
{description_of_what_students_build}
"""
# %%
#| keep_imports
#| default_exp core.{module}
import numpy as np
from typing import Union, List
import matplotlib.pyplot as plt
from typing import Union, List, Optional
# %%
# Complete working implementation with markers...
#| export
class MainClass:
"""
{Description of the class}
TODO: {What students need to implement}
APPROACH:
1. {Step 1 with specific guidance}
2. {Step 2 with specific guidance}
3. {Step 3 with specific guidance}
EXAMPLE:
Input: {concrete_example}
Expected: {expected_output}
HINTS:
- {Helpful hint about approach}
- {Systems thinking hint}
- {Real-world connection}
"""
def __init__(self, params):
raise NotImplementedError("Student implementation required")
# %%
#| hide
#| export
class MainClass:
"""Complete implementation (hidden from students)."""
def __init__(self, params):
# Actual working implementation
pass
# %% [markdown]
"""
## 🧪 Test Your Implementation
"""
# %%
# Test with real data
try:
# Test student implementation
result = MainClass(real_data_example)
print(f"✅ Success: {result}")
except NotImplementedError:
print("⚠️ Implement the class above first!")
# Visual feedback (development only - not exported)
def show_results(data):
"""Show visual confirmation of working code."""
plt.figure(figsize=(10, 6))
# Visualization code
plt.show()
if _should_show_plots():
show_results(real_data)
```
### Step 3: Add Pedagogical Markers
Mark what students should implement vs. what to provide.
### Step 4: Create Tests with Real Data
Create `modules/{module}/tests/test_{module}.py`:
### Step 4: Convert and Generate
```python
import pytest
import numpy as np
from {module}_dev import MainClass
def test_with_real_data():
"""Test with actual production data."""
# Use real datasets, not mocks
real_data = load_real_dataset()
instance = MainClass(real_data)
result = instance.process()
# Test real properties
assert result.shape == expected_real_shape
assert result.dtype == expected_real_dtype
# Test with actual data characteristics
```
### Step 5: Convert and Export
```bash
# Convert Python to notebook
# Convert to notebook
python bin/tito.py notebooks --module {module}
# Generate student version
python3 bin/generate_student_notebooks.py --module {module}
# Export to package
python bin/tito.py sync --module {module}
# Test everything
python bin/tito.py test --module {module}
```
## 🏷️ Marker System
## 🏷️ NBDev Directives
### Exercise Markers
```python
#| exercise_start
#| difficulty: easy|medium|hard
#| hint: Helpful guidance without giving away the solution
#| hint: Multiple hints allowed for complex methods
#| solution_test: What students should verify after implementation
def method_to_implement(self, params):
"""Complete function signature and docstring."""
# Full working implementation
return result
#| exercise_end
```
### Core Directives
- `#| default_exp core.{module}` - Sets export destination
- `#| export` - Marks code for export to package
- `#| hide` + `#| export` - Hidden implementation (instructor solution)
- `# %% [markdown]` - Markdown cells for explanations
- `# %%` - Code cells
### Preservation Markers
```python
#| keep_imports
import statements # Preserved in student version
#| keep_complete
def utility_method(self):
"""Keep entire implementation - too complex/not educational."""
return complex_implementation()
#| remove_cell
# This entire cell removed from student version
instructor_only_code()
```
### Educational Structure
- **Concept explanation** → **Implementation guidance** → **Hidden solution** → **Testing** → **Visual feedback**
## 🎨 Difficulty System
Use visual indicators that students immediately understand:
- **🟢 Easy (5-10 min)**: Constructor, properties, basic operations
- **🟡 Medium (10-20 min)**: Conditional logic, data processing, error handling
- **🔴 Hard (20+ min)**: Complex algorithms, system integration, optimization
- **🟢 Easy (5-10 min)**: Basic implementation, clear patterns
- Constructor with simple type conversion
- Property getters
- Basic arithmetic operations
## 📋 Implementation Guidelines
- **🟡 Medium (10-20 min)**: Moderate complexity, some edge cases
- Methods with conditional logic
- Shape manipulation
- Error handling
### Students Implement (Core Learning)
- **Main functionality**: Core algorithms and data structures
- **Data processing**: Loading, preprocessing, batching
- **Error handling**: Input validation, type checking
- **Basic operations**: Mathematical operations, transformations
- **🔴 Hard (20+ min)**: Complex logic, multiple concepts
- Broadcasting operations
- Advanced algorithms
- Integration between components
### Students Receive (Focus on Learning Goals)
- **Complex setup**: Download progress bars, caching systems
- **Utility functions**: Visualization, debugging helpers
- **Advanced features**: Optimization, GPU support
- **Infrastructure**: Test frameworks, import management
## 📋 What Students Should Implement vs. Receive
### ✅ **Students Implement** (Active Learning)
- **Core functionality**: Main learning objectives
- **Basic operations**: Addition, multiplication, etc.
- **Essential properties**: Shape, size, dtype
- **Error handling**: Type checking, validation
- **Simple algorithms**: Reshape, transpose, reductions
### 🎁 **Students Receive** (Focus on Learning Goals)
- **Complex setup code**: Initialization boilerplate
- **Utility functions**: String formatting, type checking helpers
- **Advanced features**: GPU support, optimization, broadcasting
- **Infrastructure**: Test frameworks, import statements
- **Edge case handling**: Complex error messages, corner cases
## 📝 Writing Guidelines
### Function Structure
Always provide complete signatures and docstrings:
### TODO Guidance Quality
```python
#| exercise_start
#| difficulty: medium
#| hint: Use numpy broadcasting for element-wise operations
#| solution_test: Result should be same shape as inputs
def __add__(self, other: Union['Tensor', int, float]) -> 'Tensor':
"""
Add this tensor with another tensor or scalar.
Args:
other: Another Tensor or scalar value
Returns:
Tensor: New tensor with element-wise addition result
Raises:
TypeError: If other is not a compatible type
"""
if isinstance(other, Tensor):
return Tensor(self._data + other._data)
else:
return Tensor(self._data + other)
#| exercise_end
```
"""
TODO: {Clear, specific task}
### Hint Guidelines
- **Be helpful, not prescriptive**: Guide thinking, don't give code
- **Progressive disclosure**: Start general, get more specific
- **Reference relevant concepts**: "Use numpy broadcasting", "Check instance type"
APPROACH:
1. {Concrete first step}
2. {Concrete second step}
3. {Concrete third step}
Good hints:
```python
#| hint: Convert input to numpy array for consistent handling
#| hint: Use isinstance() to check if input is a Tensor or scalar
#| hint: Remember to return a new Tensor object, not numpy array
```
EXAMPLE:
Input: {actual_data_example}
Expected: {concrete_expected_output}
Bad hints:
```python
#| hint: Write: return Tensor(self._data + other._data) # Too specific!
#| hint: Add the data # Too vague!
```
HINTS:
- {Helpful guidance without giving code}
- {Systems thinking consideration}
- {Real-world connection}
### Solution Tests
Help students verify their implementation:
```python
#| solution_test: Tensor([1,2]) + Tensor([3,4]) should equal Tensor([4,6])
#| solution_test: Tensor([1,2]) + 5 should equal Tensor([6,7])
#| solution_test: Result should be a Tensor object, not numpy array
SYSTEMS THINKING:
- {Performance consideration}
- {Scalability question}
- {User experience aspect}
"""
```
## 🗂️ Module Structure
```
modules/{module}/
├── {module}_dev.py # 🔧 Complete implementation (instructor)
├── {module}_dev.ipynb # 📓 Generated from .py file
├── {module}_dev_student.ipynb # 🎓 Auto-generated exercise version
├── test_{module}.py # 🧪 Comprehensive test suite
├── check_{module}.py # ✅ Manual verification script
└── README.md # 📖 Module overview and setup
├── {module}_dev.py # 🔧 Complete implementation
├── {module}_dev.ipynb # 📓 Generated notebook
├── tests/
│ └── test_{module}.py # 🧪 Real data tests
├── README.md # 📖 Module guide
└── data/ # 📊 Cached datasets (if needed)
```
## 🎯 Module Examples
## ✅ Quality Standards
### Easy Module (Tensor Basics)
Students implement:
- 🟢 Constructor (`__init__`)
- 🟢 Properties (`shape`, `size`, `dtype`)
- 🟢 Basic arithmetic (`__add__`, `__sub__`)
- 🟡 Utilities (`reshape`, `transpose`)
### Before Release
- [ ] Uses real data, not synthetic/mock data
- [ ] Includes progress feedback for long operations
- [ ] Visual feedback functions (development only)
- [ ] Tests use actual datasets at realistic scales
- [ ] TODO guidance includes systems thinking
- [ ] Clean separation between development and exports
- [ ] Follows "Build → Use → Understand" progression
### Medium Module (Autograd)
Students implement:
- 🟡 Forward pass computation
- 🟡 Gradient accumulation
- 🔴 Backward pass algorithm
- 🔴 Chain rule application
### Hard Module (Neural Networks)
Students implement:
- 🟡 Layer forward pass
- 🔴 Loss computation
- 🔴 Backpropagation
- 🔴 Parameter updates
## 🛠️ Development Tools
### Convert Python to Notebook
```bash
python bin/tito.py notebooks --module {module}
```
### Generate Student Version
```bash
python bin/generate_student_notebooks.py --module {module}
```
### Test Complete Workflow
```bash
# Test instructor version
cd modules/{module}
jupyter lab {module}_dev.ipynb
# Test student version
jupyter lab {module}_dev_student.ipynb
# Verify exports work
python bin/tito.py sync --module {module}
python bin/tito.py test --module {module}
```
## ✅ Quality Checklist
Before releasing a module:
**Instructor Version:**
- [ ] Complete implementation works and passes all tests
- [ ] All methods have proper type hints and docstrings
- [ ] Markers are correctly placed and consistent
- [ ] Code follows TinyTorch style guidelines
**Student Version:**
- [ ] Generated notebook preserves function signatures
- [ ] Hints are helpful but not prescriptive
- [ ] Difficulty progression makes sense
- [ ] Tests provide clear verification guidance
- [ ] Students can run and get feedback immediately
**Integration:**
- [ ] Module exports correctly to tinytorch package
- [ ] Tests pass with student implementation stubs
- [ ] README explains module purpose and workflow
- [ ] Follows established module naming conventions
### Integration Requirements
- [ ] Exports correctly to `tinytorch.core.{module}`
- [ ] No circular dependencies
- [ ] Consistent with existing module patterns
- [ ] Compatible with TinyTorch CLI tools
## 💡 Best Practices
### For Instructors
1. **Start with learning goals**: What should students understand after this module?
2. **Write complete first**: Get the full solution working before adding markers
3. **Think like a student**: What would be confusing? What needs guidance?
4. **Test the student path**: Try implementing following your own hints
### Development Process
1. **Start with real data**: Choose production dataset first
2. **Write complete implementation**: Get it working before adding markers
3. **Add rich feedback**: Visual confirmation, progress indicators
4. **Test the student path**: Follow your own TODO guidance
5. **Optimize user experience**: Consider performance, caching, error messages
### For Markers
1. **Be consistent**: Use same difficulty criteria across modules
2. **Progressive complexity**: Easy concepts early, hard concepts later
3. **Meaningful hints**: Guide understanding, not just implementation
4. **Clear boundaries**: Be explicit about what students implement vs. receive
### Systems Thinking
- **Performance**: How does this scale with larger datasets?
- **Caching**: How do we avoid repeated expensive operations?
- **User Experience**: How do students know the code is working?
- **Production Relevance**: How does this connect to real ML systems?
### For Generator
1. **Preserve signatures**: Students need to know the interface
2. **Keep docstrings**: Essential for understanding purpose
3. **Clean output**: Remove implementation but keep structure
4. **Test guidance**: Help students verify their work
### Educational Design
- **Immediate gratification**: Students see results quickly
- **Progressive complexity**: Build understanding step by step
- **Real-world connections**: Connect every concept to production
- **Visual confirmation**: Students see their code working
## 🔄 Iteration and Improvement
## 🔄 Continuous Improvement
After teaching with a module:
1. **Collect feedback**: What confused students? What was too easy/hard?
2. **Update markers**: Adjust difficulty, improve hints
3. **Refine scope**: Move methods between implement/provide categories
4. **Share learnings**: Update this guide with discoveries
1. **Monitor student experience**: Where do they get stuck?
2. **Improve guidance**: Better TODO instructions, clearer hints
3. **Enhance feedback**: More visual confirmation, better progress indicators
4. **Optimize performance**: Faster data loading, better caching
5. **Update documentation**: Share learnings with other developers
Remember: The goal is **active learning through hands-on implementation**, not just demonstration!
## 🎯 Success Metrics
**Students should be able to:**
- Explain what they built in simple terms
- Modify code to solve related problems
- Connect module concepts to real ML systems
- Debug issues by understanding the system
**Modules should achieve:**
- High student engagement and completion rates
- Smooth progression to next modules
- Real-world relevance and production quality
- Consistent patterns across the curriculum
---
**Remember**: We're teaching ML systems engineering, not just algorithms. Every module should reflect real-world practices and challenges while maintaining the "Build → Use → Understand" educational cycle.

95
docs/development/quick-module-reference.md
View File

@@ -1,8 +1,50 @@
# 🚀 TinyTorch Module Development - Quick Reference
## 🧠 Choose Your Learning Pattern First
**Before writing any code, decide which engagement pattern fits your module:**
### **🤔 Build → Use → Reflect** (Design & Systems Thinking)
- **Best for**: Early modules, foundational concepts, design decisions
- **Focus**: "Why did we make this choice? What are the trade-offs?"
- **Activities**: Design comparisons, trade-off analysis, systems implications
- **Examples**: Setup, Tensor, Layers, Data
### **🔍 Build → Use → Analyze** (Technical Depth)
- **Best for**: Middle modules, performance-critical components
- **Focus**: "How does this actually work? Where are the bottlenecks?"
- **Activities**: Profiling, debugging, measurement, technical investigation
- **Examples**: Training, Autograd, Profiling, Benchmarking
### **⚡ Build → Use → Optimize** (Systems Iteration)
- **Best for**: Advanced modules, production-focused components
- **Focus**: "How can we make this better? How does it scale?"
- **Activities**: Performance improvement, scaling, iteration, optimization
- **Examples**: MLOps, Kernels, Compression, Distributed
### **Decision Questions**
1. **What type of learning is most important?** (Conceptual/Technical/Optimization)
2. **Where does this fit in progression?** (Early/Middle/Advanced)
3. **What skills do students need most?** (Design thinking/Debugging/Performance)
### **Document Your Choice**
```markdown
## 🎯 Learning Pattern: Build → Use → [Pattern]
**Pattern Choice**: [Reflect/Analyze/Optimize]
**Rationale**: [Why this pattern fits the learning objectives]
**Key Activities**:
- [Specific activity 1]
- [Specific activity 2]
- [Specific activity 3]
```
---
## 📋 Development Checklist
### 1. Plan Module
- [ ] **Choose engagement pattern** (Reflect/Analyze/Optimize)
- [ ] Define learning objectives (what students will implement)
- [ ] Choose difficulty levels (🟢 easy → 🟡 medium → 🔴 hard)
- [ ] Decide what to provide vs. what students implement
@@ -12,6 +54,11 @@ Create `modules/{module}/{module}_dev.py`:
```python
# %% [markdown]
# # Module: {Title}
#
# ## 🎯 Learning Pattern: Build → Use → [Pattern]
# **Pattern Choice**: [Reflect/Analyze/Optimize]
# **Rationale**: [Why this pattern fits]
#
# Learning objectives and overview
# %%
@@ -31,7 +78,12 @@ class YourClass:
#| exercise_end
```
### 3. Convert and Generate
### 3. Design Pattern-Specific Activities
- **Reflect**: Add reflection questions and trade-off analysis
- **Analyze**: Include profiling tools and debugging exercises
- **Optimize**: Create performance challenges and iteration tasks
### 4. Convert and Generate
```bash
# Convert Python to notebook
python bin/tito.py notebooks --module {module}
@@ -40,7 +92,7 @@ python bin/tito.py notebooks --module {module}
python3 bin/generate_student_notebooks.py --module {module}
```
### 4. Test and Verify
### 5. Test and Verify
```bash
# Test both versions work
jupyter lab modules/{module}/{module}_dev.ipynb
@@ -69,9 +121,31 @@ python bin/tito.py test --module {module}
- **🟡 Medium (10-20 min)**: Conditional logic, shape manipulation
- **🔴 Hard (20+ min)**: Complex algorithms, multiple concepts
## 🎯 Pattern-Specific Activities
### **Reflection Activities**
- Design comparison questions
- Trade-off analysis exercises
- "What if we had different constraints?" scenarios
- Connection to production systems
### **Analysis Activities**
- Profiling memory usage and performance
- Debugging exercises with real issues
- Measurement and interpretation tasks
- Technical investigation challenges
### **Optimization Activities**
- Performance improvement challenges
- Scaling exercises with larger datasets
- Iteration on system design
- Benchmarking and comparison tasks
## ✅ Quality Check
**Before release:**
- [ ] **Pattern choice documented** with clear rationale
- [ ] **Pattern-specific activities** included and tested
- [ ] Complete version works and passes tests
- [ ] Student version preserves signatures and docstrings
- [ ] Hints are helpful but not prescriptive
@@ -82,20 +156,21 @@ python bin/tito.py test --module {module}
```
modules/{module}/
├── {module}_dev.py # 🔧 Write this first
├── {module}_dev.py # 🔧 Write this first (with pattern choice)
├── {module}_dev.ipynb # 📓 Generated
├── {module}_dev_student.ipynb # 🎓 Auto-generated
├── test_{module}.py # 🧪 Test suite
└── README.md # 📖 Module guide
└── README.md # 📖 Module guide (include pattern)
```
## 💡 Pro Tips
1. **Write complete implementation first** - Get it working before adding markers
2. **Test the student path** - Follow your own hints to verify they work
3. **Be generous with hints** - Better too helpful than too cryptic
4. **Preserve all signatures** - Students need to know the interface
5. **Progressive difficulty** - Start easy, build complexity
1. **Choose pattern first** - Everything else flows from this decision
2. **Write complete implementation first** - Get it working before adding markers
3. **Design pattern-specific activities** - Make them engaging and educational
4. **Test the student path** - Follow your own hints to verify they work
5. **Be generous with hints** - Better too helpful than too cryptic
6. **Document your pattern choice** - Help future developers understand your reasoning
## 🛠️ Common Commands
@@ -112,4 +187,4 @@ python bin/generate_student_notebooks.py --module {module}
python bin/tito.py test --module {module}
```
*See `MODULE_DEVELOPMENT_GUIDE.md` for complete details.*
*See [Module Development Guide](module-development-guide.md) for complete details and [Pedagogical Principles](../pedagogy/pedagogical-principles.md) for the full framework.*

309
docs/pedagogy/pedagogical-principles.md
View File

@@ -1,6 +1,6 @@
# 🎓 TinyTorch Pedagogical Principles
**The "Build → Use → Understand" Framework for ML Systems Education**
**The "Build → Use → [Engage]" Framework for ML Systems Education**
This document defines the core pedagogical principles that guide TinyTorch module development and student learning progression.
@@ -8,16 +8,94 @@ This document defines the core pedagogical principles that guide TinyTorch modul
## 🎯 Core Learning Philosophy
### **Build → Use → Understand → Repeat**
### **Build → Use → [Engage] → Repeat**
Each module follows this fundamental cycle:
Each module follows this fundamental cycle, but the third stage varies based on learning objectives:
1. **Build** the fundamental building block from scratch
2. **Use** it immediately to see concrete results
3. **Understand** how it works through experimentation
3. **[Engage]** through the appropriate cognitive process:
- **Reflect**: Metacognition about design decisions and trade-offs
- **Analyze**: Technical depth through profiling and debugging
- **Optimize**: Systems iteration and performance improvement
4. **Repeat** with the next building block
This approach ensures students get **immediate positive feedback** and **concrete understanding** before moving to abstract concepts.
This approach ensures students get **immediate positive feedback** and **specific, actionable engagement** before moving to abstract concepts.
---
## 🧠 Engagement Patterns by Learning Type
### **🤔 Build → Use → Reflect** (Design & Systems Thinking)
**When to use**: Early modules, foundational concepts, design decisions
**Focus**: Metacognition, trade-offs, systems implications
**Activities**:
- "Why did we choose this design over alternatives?"
- "What are the memory vs. speed trade-offs?"
- "How does this connect to production systems?"
- "What would we change if we had different constraints?"
**Example modules**: Setup, Tensor, Layers, Data
### **🔍 Build → Use → Analyze** (Technical Depth)
**When to use**: Middle modules, performance-critical components
**Focus**: System behavior, bottlenecks, technical understanding
**Activities**:
- Profile memory usage and compute patterns
- Debug system behavior and edge cases
- Inspect intermediate results and data flow
- Measure performance characteristics
**Example modules**: Training, Autograd, Profiling, Benchmarking
### **⚡ Build → Use → Optimize** (Systems Iteration)
**When to use**: Advanced modules, production-focused components
**Focus**: Performance improvement, scalability, real-world constraints
**Activities**:
- Improve baseline implementations
- Scale to larger datasets and models
- Optimize for production constraints
- Iterate on system design
**Example modules**: MLOps, Kernels, Compression, Distributed
---
## 📋 Pattern Selection Guide
### **For Module Developers: Choosing the Right Pattern**
When starting a new module, ask:
1. **What type of learning is most important?**
- **Conceptual understanding** → Use **Reflect**
- **Technical mastery** → Use **Analyze**
- **Systems optimization** → Use **Optimize**
2. **Where does this fit in the progression?**
- **Early modules (1-4)** → Usually **Reflect**
- **Middle modules (5-8)** → Usually **Analyze**
- **Advanced modules (9-12)** → Usually **Optimize**
3. **What skills do students need most?**
- **Design thinking** → **Reflect**
- **Debugging/profiling** → **Analyze**
- **Performance engineering** → **Optimize**
### **Pattern Documentation Template**
When creating a module, document your choice:
```markdown
## 🎯 Learning Pattern: Build → Use → [Pattern]
**Pattern Choice**: [Reflect/Analyze/Optimize]
**Rationale**: [Why this pattern fits the learning objectives]
**Key Activities**:
- [Specific activity 1]
- [Specific activity 2]
- [Specific activity 3]
```
---
@@ -27,10 +105,10 @@ This approach ensures students get **immediate positive feedback** and **concret
ML systems are compositions of simple building blocks. Students learn best when they:
- Build each block independently
- See it work immediately
- Understand its role in the larger system
- Engage with it through the appropriate cognitive process
- Compose it with other blocks
### **Progressive Complexity**
### **Progressive Complexity with Varied Engagement**
```
Tensors → Layers → Networks → Training → Production
↓ ↓ ↓ ↓ ↓
@@ -38,11 +116,113 @@ Tensors → Layers → Networks → Training → Production
↓ ↓ ↓ ↓ ↓
Use Use Use Use Use
↓ ↓ ↓ ↓ ↓
Understand Understand Understand Understand Understand
Reflect Reflect Analyze Analyze Optimize
```
---
## 🎯 Module Pattern Assignments
### **🤔 Reflection Modules** (Design & Systems Thinking)
#### **Module 0: Setup**
**Pattern**: Build → Use → Reflect
**Key Questions**:
- Why do we need virtual environments?
- How does our development setup compare to production?
- What are the trade-offs in our toolchain choices?
#### **Module 1: Tensors**
**Pattern**: Build → Use → Reflect
**Key Questions**:
- Why did we choose this memory layout?
- How does our design compare to NumPy/PyTorch?
- What are the implications for performance?
#### **Module 2: Layers**
**Pattern**: Build → Use → Reflect
**Key Questions**:
- Why separate Dense and Activation layers?
- How does our API design affect usability?
- What abstraction trade-offs did we make?
#### **Module 3: Data**
**Pattern**: Build → Use → Reflect
**Key Questions**:
- Why batch data processing?
- How does our pipeline design affect memory usage?
- What are the trade-offs in caching strategies?
### **🔍 Analysis Modules** (Technical Depth)
#### **Module 4: Training**
**Pattern**: Build → Use → Analyze
**Key Activities**:
- Profile memory usage during training
- Analyze gradient flow and convergence
- Debug training instabilities
- Measure training performance bottlenecks
#### **Module 5: Autograd**
**Pattern**: Build → Use → Analyze
**Key Activities**:
- Inspect computational graph construction
- Analyze memory usage in backpropagation
- Debug gradient computation issues
- Profile forward vs. backward pass performance
#### **Module 6: Profiling**
**Pattern**: Build → Use → Analyze
**Key Activities**:
- Measure system resource usage
- Identify computational bottlenecks
- Analyze memory allocation patterns
- Compare performance across implementations
#### **Module 7: Benchmarking**
**Pattern**: Build → Use → Analyze
**Key Activities**:
- Analyze performance across different scenarios
- Compare implementation trade-offs
- Identify scaling characteristics
- Debug performance regressions
### **⚡ Optimization Modules** (Systems Iteration)
#### **Module 8: MLOps**
**Pattern**: Build → Use → Optimize
**Key Activities**:
- Optimize monitoring system performance
- Scale logging and metrics collection
- Improve deployment pipeline efficiency
- Iterate on production system design
#### **Module 9: Kernels**
**Pattern**: Build → Use → Optimize
**Key Activities**:
- Optimize compute kernel performance
- Improve memory access patterns
- Scale to larger problem sizes
- Iterate on vectorization strategies
#### **Module 10: Compression**
**Pattern**: Build → Use → Optimize
**Key Activities**:
- Optimize compression algorithms
- Improve accuracy vs. size trade-offs
- Scale compression to larger models
- Iterate on compression strategies
#### **Module 11: Distributed**
**Pattern**: Build → Use → Optimize
**Key Activities**:
- Optimize communication patterns
- Scale to multiple devices/nodes
- Improve fault tolerance
- Iterate on distributed system design
---
## 📚 Module Design Principles
### 1. **Immediate Gratification**
@@ -129,73 +309,88 @@ Understand Understand Understand Understand Understand
### **For Module Developers**
1. **Start with the simplest possible example**
- What's the minimal code that demonstrates the concept?
- Can students see results in < 5 lines of code?
1. **Choose the right engagement pattern first**
- Consider the learning objectives and student needs
- Use the Pattern Selection Guide above
- Document your choice and rationale
2. **Build up complexity gradually**
- Add one feature at a time
- Each addition should be motivated by a clear problem
- Students should understand why each piece is needed
2. **Design activities that match the pattern**
- **Reflect**: Design thinking questions, trade-off analysis
- **Analyze**: Profiling tools, debugging exercises, measurement tasks
- **Optimize**: Performance challenges, scaling exercises, iteration tasks
3. **Provide immediate visual feedback**
- Print results, show plots, demonstrate changes
- Students should see their code working
- Abstract concepts need concrete demonstrations
3. **Provide pattern-specific scaffolding**
- **Reflect**: Guiding questions, comparison frameworks
- **Analyze**: Profiling tools, debugging guides, measurement templates
- **Optimize**: Performance baselines, optimization targets, iteration guidelines
4. **Connect to bigger picture**
- How does this building block fit into ML systems?
- What problems does it solve?
- How does it relate to industry frameworks?
4. **Connect to the bigger picture**
- How does this pattern prepare students for real-world work?
- What professional skills are they developing?
- How does this connect to industry practices?
### **For Students**
1. **Don't skip the "Use" step**
- Always run the code and see results
- Experiment with different inputs
- Break things to understand boundaries
1. **Engage actively with the chosen pattern**
- **Reflect**: Think deeply about design decisions and trade-offs
- **Analyze**: Use tools to understand system behavior
- **Optimize**: Iterate to improve performance and scalability
2. **Build intuition before memorizing**
- Understand what the code does before how it works
- Play with examples before reading theory
- Ask "what if" questions
2. **Build pattern-specific skills**
- **Reflect**: Systems thinking, design evaluation, trade-off analysis
- **Analyze**: Profiling, debugging, measurement, technical investigation
- **Optimize**: Performance engineering, scalability, iteration
3. **Connect to previous modules**
- How does this build on what you learned before?
- What problems does this solve that previous modules couldn't?
- How do the pieces fit together?
3. **Connect patterns to professional practice**
- How do senior engineers use these thinking patterns?
- When would you use each pattern in real projects?
- How do these patterns complement each other?
---
## 🎯 Success Metrics
### **Student Success Indicators**
- Can explain what each building block does in simple terms
- Can modify code to solve related problems
- Can debug issues by understanding the underlying system
- Can connect module concepts to real-world ML systems
### **Pattern-Specific Success Indicators**
### **Module Success Indicators**
- Students complete modules with high engagement
- Tests pass consistently across different student backgrounds
- Students can apply concepts to new problems
- Smooth progression to next module
#### **Reflection Success**
- Can articulate design decisions and trade-offs
- Can compare their implementation to alternatives
- Can connect technical choices to systems implications
- Can evaluate designs from multiple perspectives
#### **Analysis Success**
- Can use profiling tools to understand system behavior
- Can identify and debug performance bottlenecks
- Can measure and interpret system characteristics
- Can investigate technical problems systematically
#### **Optimization Success**
- Can improve baseline implementations measurably
- Can scale systems to larger problem sizes
- Can iterate effectively on system design
- Can balance multiple optimization objectives
### **Overall Module Success**
- Students engage actively with the chosen pattern
- Pattern activities deepen understanding of core concepts
- Students develop transferable professional skills
- Smooth progression to modules using different patterns
---
## 🔄 Continuous Improvement
### **Feedback Loops**
- Monitor where students get stuck
- Identify concepts that need more scaffolding
- Adjust complexity based on student success rates
- Iterate on examples and explanations
### **Pattern Effectiveness Evaluation**
- Monitor student engagement with pattern activities
- Assess learning outcomes for each pattern type
- Adjust pattern assignments based on student success
- Evolve pattern activities based on industry changes
### **Evolution Principles**
- Maintain the "Build → Use → Understand" cycle
- Keep immediate feedback and concrete results
- Preserve the building block progression
- Adapt examples to current ML landscape
### **Cross-Pattern Integration**
- Help students see connections between patterns
- Design capstone projects that use multiple patterns
- Create reflection opportunities across pattern types
- Build metacognitive awareness of when to use each pattern
---
@@ -206,7 +401,9 @@ This pedagogical approach draws from:
- **Experiential Learning**: Learning through direct experience and reflection
- **Scaffolded Learning**: Temporary support structures that build independence
- **Systems Thinking**: Understanding how components interact in complex systems
- **Metacognitive Theory**: Awareness and understanding of one's own thought processes
- **Deliberate Practice**: Focused, goal-oriented practice for skill development
---
*This document should guide all TinyTorch module development decisions. When in doubt, return to the core principle: **Build → Use → Understand → Repeat***
*This document should guide all TinyTorch module development decisions. When starting a new module, first choose the appropriate engagement pattern, then design all activities to support that pattern's learning objectives.*