mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-03-11 23:03:33 -05:00
753ae52ae0
This commit implements the pedagogically optimal "inevitable discovery" module progression based on expert validation and educational design principles. ## Module Reordering Summary **Previous Order (Problems)**: - 05_losses → 06_autograd → 07_dataloader → 08_optimizers → 09_spatial → 10_training - Issues: Autograd before optimizers, DataLoader before training, scattered dependencies **New Order (Beautiful Progression)**: - 05_losses → 06_optimizers → 07_autograd → 08_training → 09_spatial → 10_dataloader - Benefits: Each module creates inevitable need for the next ## Pedagogical Flow Achieved **05_losses** → "Need systematic weight updates" → **06_optimizers** **06_optimizers** → "Need automatic gradients" → **07_autograd** **07_autograd** → "Need systematic training" → **08_training** **08_training** → "MLPs hit limits on images" → **09_spatial** **09_spatial** → "Training is too slow" → **10_dataloader** ## Technical Changes ### Module Directory Renaming - `06_autograd` → `07_autograd` - `07_dataloader` → `10_dataloader` - `08_optimizers` → `06_optimizers` - `10_training` → `08_training` - `09_spatial` → `09_spatial` (no change) ### System Integration Updates - **MODULE_TO_CHECKPOINT mapping**: Updated in tito/commands/export.py - **Test directories**: Renamed module_XX directories to match new numbers - **Documentation**: Updated all references in MD files and agent configurations - **CLI integration**: Updated next-steps suggestions for proper flow ### Agent Configuration Updates - **Quality Assurance**: Updated module audit status with new numbers - **Module Developer**: Updated work tracking with new sequence - **Documentation**: Updated MASTER_PLAN_OF_RECORD.md with beautiful progression ## Educational Benefits 1. **Inevitable Discovery**: Each module naturally leads to the next 2. **Cognitive Load**: Concepts introduced exactly when needed 3. **Motivation**: Students understand WHY each tool is necessary 4. **Synthesis**: Everything flows toward complete ML systems understanding 5. **Professional Alignment**: Matches real ML engineering workflows ## Quality Assurance - ✅ All CLI commands still function - ✅ Checkpoint system mappings updated - ✅ Documentation consistency maintained - ✅ Test directory structure aligned - ✅ Agent configurations synchronized **Impact**: This reordering transforms TinyTorch from a collection of modules into a coherent educational journey where each step naturally motivates the next, creating optimal conditions for deep learning systems understanding.
80 lines
2.4 KiB
Python
80 lines
2.4 KiB
Python
#!/usr/bin/env python3
|
|
"""
|
|
Example: How to Modify Existing Layers to Use Backend System
|
|
|
|
This shows the minimal changes needed to existing tinytorch.core.layers
|
|
to support the backend dispatch system for competition optimization.
|
|
"""
|
|
|
|
# This is how you would modify the existing matmul function in layers_dev.py:
|
|
|
|
# BEFORE (Original Implementation):
|
|
def matmul_original(a, b):
|
|
"""Original matrix multiplication implementation"""
|
|
return a.data @ b.data # Simple NumPy operation
|
|
|
|
# AFTER (Backend-Aware Implementation):
|
|
def matmul_backend_aware(a, b):
|
|
"""Matrix multiplication with backend dispatch"""
|
|
from kernels_dev import get_backend # Import the backend system
|
|
|
|
backend = get_backend()
|
|
result_data = backend.matmul(a.data, b.data)
|
|
|
|
from tensor_dev import Tensor
|
|
return Tensor(result_data)
|
|
|
|
# The Dense layer automatically inherits the optimization!
|
|
# NO CHANGES needed to Dense.forward() method
|
|
|
|
print("""
|
|
🔧 MODIFICATION STRATEGY:
|
|
|
|
1. MINIMAL CHANGES: Only modify the low-level operation functions
|
|
- matmul() gets backend dispatch
|
|
- conv2d() gets backend dispatch
|
|
- Other layers inherit optimizations automatically
|
|
|
|
2. PRESERVE EXISTING APIs: No changes to:
|
|
- Dense layer implementation
|
|
- Module base class
|
|
- Training loops
|
|
- Student-facing code
|
|
|
|
3. ADDITIVE OPTIMIZATIONS:
|
|
- Add backend system alongside existing code
|
|
- Default to naive backend (safe for learning)
|
|
- Students opt-in to optimized backend for competition
|
|
|
|
4. EXPORT COMPATIBILITY:
|
|
- `tito module complete` still works
|
|
- NBGrader integration preserved
|
|
- Learning progression unchanged
|
|
|
|
RESULT: Students can run EXACTLY THE SAME CODE with 10-100x speedup
|
|
just by calling set_backend('optimized') before their training loop!
|
|
""")
|
|
|
|
# Example usage in student code:
|
|
example_student_code = '''
|
|
# Student writes this code normally (learning mode):
|
|
import tinytorch
|
|
model = MyNetwork()
|
|
optimizer = Adam(model.parameters())
|
|
|
|
# Train normally with naive backend (default)
|
|
for epoch in range(10):
|
|
loss = train_epoch(model, data, optimizer)
|
|
print(f"Epoch {epoch}: {loss:.4f}")
|
|
|
|
# NOW COMPETITION MODE - same code, much faster!
|
|
tinytorch.set_backend("optimized") # Only line that changes!
|
|
|
|
# Re-run the EXACT SAME training code - 10x faster!
|
|
for epoch in range(10):
|
|
loss = train_epoch(model, data, optimizer) # Same function!
|
|
print(f"Fast Epoch {epoch}: {loss:.4f}")
|
|
'''
|
|
|
|
print("💡 STUDENT EXPERIENCE:")
|
|
print(example_student_code) |