mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-04-28 14:02:32 -05:00
Add comprehensive TinyTorch Enhanced Capability Unlock System documentation
This commit adds complete documentation for the 5-milestone system that transforms TinyTorch from module-based to capability-driven learning: 📚 Documentation Suite: - milestone-system.md: Student-facing guide with milestone descriptions - instructor-milestone-guide.md: Complete assessment framework for instructors - milestone-troubleshooting.md: Comprehensive debugging guide for common issues - milestone-implementation-guide.md: Technical implementation specifications - milestone-system-overview.md: Executive summary tying everything together 🎯 The Five Milestones: 1. Basic Inference (Module 04) - Neural networks work (85%+ MNIST) 2. Computer Vision (Module 06) - MNIST recognition (95%+ CNN accuracy) 3. Full Training (Module 11) - Complete training loops (CIFAR-10 training) 4. Advanced Vision (Module 13) - CIFAR-10 classification (75%+ accuracy) 5. Language Generation (Module 16) - GPT text generation (coherent output) 🚀 Key Features: - Capability-based achievement system replacing traditional module completion - Visual progress tracking with Rich CLI visualizations - Victory conditions aligned with industry-relevant skills - Comprehensive troubleshooting for each milestone challenge - Instructor assessment framework with automated testing - Technical implementation roadmap for CLI integration 💡 Educational Impact: - Students develop portfolio-worthy capabilities rather than just completing assignments - Clear progression from basic neural networks to production AI systems - Motivation through achievement and concrete skill development - Industry alignment with real ML engineering competencies Ready for implementation phase with complete technical specifications.
This commit is contained in:
Vijay Janapa Reddi
482
docs/instructor-milestone-guide.md
Normal file
482
docs/instructor-milestone-guide.md
Normal file
@@ -0,0 +1,482 @@
|
||||
# 🎓 Instructor Guide: TinyTorch Milestone Assessment System
|
||||
|
||||
## Overview: Capability-Based Assessment
|
||||
|
||||
The TinyTorch Milestone System transforms traditional module-based grading into **capability-based assessment**. Instead of grading 16 separate assignments, you assess 5 major milestone achievements that represent genuine ML systems engineering competencies.
|
||||
|
||||
---
|
||||
|
||||
## 📊 Assessment Framework
|
||||
|
||||
### Traditional vs. Milestone Grading
|
||||
|
||||
**Traditional Approach:**
|
||||
- 16 individual module grades (often disconnected)
|
||||
- Focus on code completion and correctness
|
||||
- Students lose sight of the bigger picture
|
||||
- Difficult to assess real-world readiness
|
||||
|
||||
**Milestone Approach:**
|
||||
- 5 major capability assessments
|
||||
- Focus on systems integration and real applications
|
||||
- Students understand progression toward professional competence
|
||||
- Clear mapping to industry-relevant skills
|
||||
|
||||
### The Five Assessment Milestones
|
||||
|
||||
| Milestone | Capability | Assessment Focus | Weight |
|
||||
|-----------|------------|------------------|---------|
|
||||
| **1. Basic Inference** | Neural network functionality | Mathematical correctness, architecture understanding | 15% |
|
||||
| **2. Computer Vision** | Image processing systems | MNIST accuracy, convolution implementation | 20% |
|
||||
| **3. Full Training** | End-to-end ML pipelines | CIFAR-10 training, loss convergence, evaluation | 25% |
|
||||
| **4. Advanced Vision** | Production optimization | 75%+ CIFAR-10 accuracy, performance analysis | 20% |
|
||||
| **5. Language Generation** | Framework generalization | Character-level GPT, architecture reuse | 20% |
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Milestone Assessment Criteria
|
||||
|
||||
### Milestone 1: Basic Inference (Module 04)
|
||||
**Capability:** "I can make neural networks work!"
|
||||
|
||||
**Assessment Criteria:**
|
||||
- [ ] **Mathematical Correctness** (40%): Forward pass implementations compute correct outputs
|
||||
- [ ] **Architecture Design** (30%): Multi-layer networks properly composed from building blocks
|
||||
- [ ] **MNIST Performance** (20%): Achieve 85%+ accuracy on digit classification
|
||||
- [ ] **Code Quality** (10%): Clean, documented implementation following TinyTorch patterns
|
||||
|
||||
**Deliverables:**
|
||||
- Working Dense layer implementation
|
||||
- Multi-layer network that classifies MNIST digits
|
||||
- Demonstration of 85%+ accuracy
|
||||
- Code export to tinytorch package
|
||||
|
||||
**Assessment Method:**
|
||||
```bash
|
||||
# Automated testing
|
||||
tito milestone test 1
|
||||
|
||||
# Performance validation
|
||||
python test_mnist_basic.py # Must achieve 85%+ accuracy
|
||||
|
||||
# Code review
|
||||
tito export layers && python -c "from tinytorch.core.layers import Dense; print('✅ Export successful')"
|
||||
```
|
||||
|
||||
### Milestone 2: Computer Vision (Module 06)
|
||||
**Capability:** "I can teach machines to see!"
|
||||
|
||||
**Assessment Criteria:**
|
||||
- [ ] **Convolution Implementation** (35%): Mathematically correct Conv2D operations
|
||||
- [ ] **Spatial Processing** (25%): Proper handling of image dimensions and channels
|
||||
- [ ] **MNIST Excellence** (25%): Achieve 95%+ accuracy using convolutional features
|
||||
- [ ] **Memory Efficiency** (15%): Convolution reduces parameters vs. dense approach
|
||||
|
||||
**Deliverables:**
|
||||
- Conv2D and MaxPool2D implementations
|
||||
- CNN architecture achieving 95%+ MNIST accuracy
|
||||
- Performance comparison: CNN vs. dense network
|
||||
- Memory usage analysis showing efficiency gains
|
||||
|
||||
**Assessment Method:**
|
||||
```bash
|
||||
# Automated testing
|
||||
tito milestone test 2
|
||||
|
||||
# Performance validation
|
||||
python test_mnist_cnn.py # Must achieve 95%+ accuracy
|
||||
|
||||
# Efficiency analysis
|
||||
python compare_cnn_vs_dense.py # Parameter count comparison
|
||||
```
|
||||
|
||||
### Milestone 3: Full Training (Module 11)
|
||||
**Capability:** "I can train production-quality models!"
|
||||
|
||||
**Assessment Criteria:**
|
||||
- [ ] **Training Pipeline** (30%): Complete workflow from data loading to trained model
|
||||
- [ ] **Loss Functions** (25%): Correct CrossEntropy implementation with gradient computation
|
||||
- [ ] **CIFAR-10 Training** (25%): Successfully train CNN on real dataset
|
||||
- [ ] **Training Dynamics** (20%): Demonstrate understanding of convergence and validation
|
||||
|
||||
**Deliverables:**
|
||||
- Complete Trainer class with loss functions and metrics
|
||||
- CIFAR-10 CNN training from scratch
|
||||
- Training curves showing convergence
|
||||
- Model checkpointing and evaluation pipeline
|
||||
|
||||
**Assessment Method:**
|
||||
```bash
|
||||
# Automated testing
|
||||
tito milestone test 3
|
||||
|
||||
# End-to-end training
|
||||
python train_cifar10_milestone.py # Must show convergence
|
||||
|
||||
# Training analysis
|
||||
python analyze_training_dynamics.py # Loss curves, overfitting analysis
|
||||
```
|
||||
|
||||
### Milestone 4: Advanced Vision (Module 13)
|
||||
**Capability:** "I can build production computer vision systems!"
|
||||
|
||||
**Assessment Criteria:**
|
||||
- [ ] **CIFAR-10 Mastery** (40%): Achieve 75%+ accuracy on full CIFAR-10 dataset
|
||||
- [ ] **Performance Optimization** (25%): Demonstrate kernel optimizations and efficiency improvements
|
||||
- [ ] **Systems Engineering** (20%): Proper benchmarking, memory profiling, scaling analysis
|
||||
- [ ] **Production Readiness** (15%): Model saving, loading, deployment considerations
|
||||
|
||||
**Deliverables:**
|
||||
- CNN achieving 75%+ CIFAR-10 accuracy
|
||||
- Performance benchmarks and optimization analysis
|
||||
- Complete model deployment pipeline
|
||||
- Systems analysis documenting bottlenecks and solutions
|
||||
|
||||
**Assessment Method:**
|
||||
```bash
|
||||
# Performance validation (CRITICAL)
|
||||
python test_cifar10_production.py # Must achieve 75%+ accuracy
|
||||
|
||||
# Systems analysis
|
||||
python benchmark_production_model.py # Memory, speed, scaling analysis
|
||||
|
||||
# Deployment readiness
|
||||
python test_model_deployment.py # Save/load, inference pipeline
|
||||
```
|
||||
|
||||
### Milestone 5: Language Generation (Module 16)
|
||||
**Capability:** "I can build the future of AI!"
|
||||
|
||||
**Assessment Criteria:**
|
||||
- [ ] **GPT Implementation** (35%): Character-level transformer using existing components
|
||||
- [ ] **Component Reuse** (25%): 95%+ code reuse from vision modules
|
||||
- [ ] **Text Generation** (25%): Coherent text generation after training
|
||||
- [ ] **Framework Unification** (15%): Demonstration of unified mathematical foundations
|
||||
|
||||
**Deliverables:**
|
||||
- Character-level GPT using TinyTorch components
|
||||
- Text generation samples showing coherent output
|
||||
- Analysis documenting component reuse across modalities
|
||||
- Unified framework capable of both vision and language tasks
|
||||
|
||||
**Assessment Method:**
|
||||
```bash
|
||||
# Implementation validation
|
||||
tito milestone test 5
|
||||
|
||||
# Text generation demo
|
||||
python demo_text_generation.py # Must generate readable text
|
||||
|
||||
# Framework unification analysis
|
||||
python analyze_component_reuse.py # Document vision→language reuse
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🏆 Grading Rubrics
|
||||
|
||||
### Milestone Performance Levels
|
||||
|
||||
**Exemplary (90-100%)**
|
||||
- Exceeds performance benchmarks (e.g., >80% CIFAR-10 for Milestone 4)
|
||||
- Demonstrates deep systems understanding
|
||||
- Code quality excellent with clear documentation
|
||||
- Shows innovation beyond basic requirements
|
||||
|
||||
**Proficient (80-89%)**
|
||||
- Meets all performance benchmarks
|
||||
- Solid understanding of systems principles
|
||||
- Good code quality and implementation
|
||||
- Completes all required deliverables
|
||||
|
||||
**Developing (70-79%)**
|
||||
- Meets most performance benchmarks with minor issues
|
||||
- Basic understanding of concepts
|
||||
- Code works but may have quality issues
|
||||
- Some deliverables incomplete
|
||||
|
||||
**Beginning (60-69%)**
|
||||
- Below performance benchmarks
|
||||
- Limited understanding of concepts
|
||||
- Significant code issues
|
||||
- Many deliverables missing
|
||||
|
||||
**Insufficient (<60%)**
|
||||
- Fails to meet milestone criteria
|
||||
- Requires substantial additional work
|
||||
|
||||
### Sample Rubric: Milestone 4 (Advanced Vision)
|
||||
|
||||
| Criterion | Exemplary (23-25 pts) | Proficient (20-22 pts) | Developing (17-19 pts) | Beginning (14-16 pts) |
|
||||
|-----------|---------------------|---------------------|-------------------|-------------------|
|
||||
| **CIFAR-10 Accuracy** | 80%+ accuracy achieved | 75-79% accuracy achieved | 70-74% accuracy achieved | Below 70% accuracy |
|
||||
| **Performance Analysis** | Comprehensive benchmarking with optimization insights | Good analysis with some optimization | Basic analysis present | Limited or missing analysis |
|
||||
| **Code Quality** | Excellent documentation and structure | Good quality with minor issues | Adequate but some problems | Poor quality, hard to follow |
|
||||
| **Systems Understanding** | Deep insight into bottlenecks and scaling | Good understanding of performance | Basic understanding | Limited understanding |
|
||||
|
||||
---
|
||||
|
||||
## 📋 Practical Assessment Implementation
|
||||
|
||||
### Setting Up Milestone Assessment
|
||||
|
||||
1. **Create Assessment Environment**
|
||||
```bash
|
||||
# Set up standardized testing environment
|
||||
git clone https://github.com/your-repo/tinytorch-assessment.git
|
||||
cd tinytorch-assessment
|
||||
python setup_assessment_env.py
|
||||
```
|
||||
|
||||
2. **Configure Automated Testing**
|
||||
```bash
|
||||
# Install assessment tools
|
||||
pip install -r assessment-requirements.txt
|
||||
|
||||
# Set up automated milestone testing
|
||||
tito assessment configure --milestones 1,2,3,4,5
|
||||
```
|
||||
|
||||
3. **Prepare Assessment Data**
|
||||
```bash
|
||||
# Download standardized datasets
|
||||
python download_assessment_datasets.py # MNIST, CIFAR-10, text corpora
|
||||
|
||||
# Verify data integrity
|
||||
python verify_assessment_data.py
|
||||
```
|
||||
|
||||
### Running Milestone Assessments
|
||||
|
||||
**For Individual Students:**
|
||||
```bash
|
||||
# Test specific milestone
|
||||
tito assessment run --student john_doe --milestone 3
|
||||
|
||||
# Generate comprehensive report
|
||||
tito assessment report --student john_doe --all-milestones
|
||||
```
|
||||
|
||||
**For Entire Class:**
|
||||
```bash
|
||||
# Batch assessment
|
||||
tito assessment batch --class cs329s_2024 --milestone 4
|
||||
|
||||
# Class performance analysis
|
||||
tito assessment analyze --class cs329s_2024 --milestone 4
|
||||
```
|
||||
|
||||
### Assessment Automation
|
||||
|
||||
**Automated Performance Testing:**
|
||||
```python
|
||||
# Example: Automated CIFAR-10 assessment for Milestone 4
|
||||
def assess_milestone_4(student_submission):
|
||||
results = {
|
||||
'accuracy': 0.0,
|
||||
'performance_metrics': {},
|
||||
'code_quality': 0.0,
|
||||
'systems_analysis': False
|
||||
}
|
||||
|
||||
# Load student's model
|
||||
model = load_student_model(student_submission)
|
||||
|
||||
# Test on standardized CIFAR-10 test set
|
||||
accuracy = evaluate_cifar10(model)
|
||||
results['accuracy'] = accuracy
|
||||
|
||||
# Benchmark performance
|
||||
results['performance_metrics'] = benchmark_model(model)
|
||||
|
||||
# Assess code quality
|
||||
results['code_quality'] = assess_code_quality(student_submission)
|
||||
|
||||
# Check for systems analysis
|
||||
results['systems_analysis'] = check_systems_analysis(student_submission)
|
||||
|
||||
return results
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Assessment Analytics
|
||||
|
||||
### Class Performance Tracking
|
||||
|
||||
**Milestone Completion Rates:**
|
||||
```
|
||||
Milestone 1 (Basic Inference): 95% completion, avg 87% score
|
||||
Milestone 2 (Computer Vision): 89% completion, avg 83% score
|
||||
Milestone 3 (Full Training): 78% completion, avg 79% score
|
||||
Milestone 4 (Advanced Vision): 67% completion, avg 76% score
|
||||
Milestone 5 (Language Generation): 56% completion, avg 74% score
|
||||
```
|
||||
|
||||
**Performance Distribution:**
|
||||
```
|
||||
CIFAR-10 Accuracy (Milestone 4):
|
||||
90%+ accuracy: 5 students (excellent)
|
||||
80-89% accuracy: 12 students (proficient)
|
||||
75-79% accuracy: 8 students (meets requirement)
|
||||
70-74% accuracy: 3 students (developing)
|
||||
<70% accuracy: 2 students (needs support)
|
||||
```
|
||||
|
||||
### Intervention Strategies
|
||||
|
||||
**Early Warning System:**
|
||||
- Students failing Milestone 1 need fundamental review
|
||||
- Students struggling with Milestone 2 need convolution tutoring
|
||||
- Students unable to complete Milestone 3 need training pipeline support
|
||||
|
||||
**Success Patterns:**
|
||||
- Students excelling in Milestone 1 typically succeed through Milestone 3
|
||||
- Milestone 4 represents the largest difficulty jump (performance optimization)
|
||||
- Milestone 5 success correlates with strong theoretical understanding
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Best Practices for Instructors
|
||||
|
||||
### Before the Course
|
||||
|
||||
1. **Set Clear Expectations**
|
||||
- Explain milestone system benefits over traditional grading
|
||||
- Share industry relevance of each milestone capability
|
||||
- Provide example portfolio projects from each milestone
|
||||
|
||||
2. **Prepare Assessment Infrastructure**
|
||||
- Set up automated testing environments
|
||||
- Prepare standardized datasets and benchmarks
|
||||
- Create rubrics aligned with learning objectives
|
||||
|
||||
### During the Course
|
||||
|
||||
1. **Regular Progress Monitoring**
|
||||
```bash
|
||||
# Weekly progress checks
|
||||
tito assessment progress --class cs329s_2024
|
||||
|
||||
# Individual student support
|
||||
tito assessment struggling --threshold 70
|
||||
```
|
||||
|
||||
2. **Milestone Celebration**
|
||||
- Acknowledge milestone achievements publicly
|
||||
- Share exceptional student work (with permission)
|
||||
- Connect milestones to real-world applications
|
||||
|
||||
3. **Adaptive Support**
|
||||
- Provide additional resources for struggling students
|
||||
- Offer advanced challenges for excelling students
|
||||
- Form study groups around milestone challenges
|
||||
|
||||
### Assessment Integrity
|
||||
|
||||
**Preventing Academic Dishonesty:**
|
||||
- Require live demonstration of key functionalities
|
||||
- Use randomized test datasets unknown to students
|
||||
- Assess understanding through milestone reflection essays
|
||||
- Monitor for code similarity across submissions
|
||||
|
||||
**Ensuring Fair Assessment:**
|
||||
- Provide clear rubrics and examples
|
||||
- Offer multiple attempts for milestone completion
|
||||
- Allow late submissions with appropriate penalties
|
||||
- Consider individual circumstances and accommodations
|
||||
|
||||
---
|
||||
|
||||
## 📈 Course Improvement Using Milestone Data
|
||||
|
||||
### Learning Analytics
|
||||
|
||||
**Identifying Content Issues:**
|
||||
- If <70% complete Milestone 2, convolution instruction needs improvement
|
||||
- If Milestone 4 accuracy consistently low, training optimization needs emphasis
|
||||
- If Milestone 5 completion drops significantly, framework design needs clarification
|
||||
|
||||
**Curriculum Optimization:**
|
||||
- Milestone completion times indicate pacing adjustments needed
|
||||
- Performance distributions show where additional scaffolding helps
|
||||
- Student feedback correlates milestone challenges with engagement
|
||||
|
||||
### Longitudinal Assessment
|
||||
|
||||
**Skill Development Tracking:**
|
||||
- Compare Milestone 1 vs. Milestone 5 code quality improvements
|
||||
- Track performance optimization learning from Milestone 3 to 4
|
||||
- Assess systems thinking development across all milestones
|
||||
|
||||
**Industry Preparation:**
|
||||
- Survey alumni on milestone relevance to their ML roles
|
||||
- Connect milestone capabilities to job interview performance
|
||||
- Track career outcomes correlated with milestone completion
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Getting Started with Milestone Assessment
|
||||
|
||||
### Quick Setup (15 minutes)
|
||||
|
||||
1. **Install Assessment Tools**
|
||||
```bash
|
||||
pip install tinytorch-assessment
|
||||
tito assessment init --course-name "CS329S Fall 2024"
|
||||
```
|
||||
|
||||
2. **Configure First Milestone**
|
||||
```bash
|
||||
tito assessment setup-milestone 1 --benchmark mnist_85_percent
|
||||
```
|
||||
|
||||
3. **Test with Sample Submission**
|
||||
```bash
|
||||
tito assessment test --sample-submission milestone1_sample.py
|
||||
```
|
||||
|
||||
### Full Implementation (1 hour)
|
||||
|
||||
1. Set up all 5 milestones with appropriate benchmarks
|
||||
2. Configure automated testing and report generation
|
||||
3. Create class roster and individual student tracking
|
||||
4. Test assessment pipeline with sample data
|
||||
|
||||
### Integration with LMS
|
||||
|
||||
**Canvas Integration:**
|
||||
```python
|
||||
# Sync milestone grades with Canvas gradebook
|
||||
tito assessment sync-canvas --course-id 12345
|
||||
```
|
||||
|
||||
**Gradescope Integration:**
|
||||
```python
|
||||
# Upload milestone rubrics to Gradescope
|
||||
tito assessment upload-rubrics --platform gradescope
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎉 The Impact of Milestone Assessment
|
||||
|
||||
### Student Benefits
|
||||
- **Clear progression** through industry-relevant capabilities
|
||||
- **Portfolio development** with concrete, demonstrable skills
|
||||
- **Motivation through achievement** rather than just completion
|
||||
- **Systems thinking** that prepares for real ML engineering roles
|
||||
|
||||
### Instructor Benefits
|
||||
- **Meaningful assessment** of genuine ML competencies
|
||||
- **Simplified grading** focused on major capabilities rather than minutiae
|
||||
- **Clear intervention points** when students struggle with key concepts
|
||||
- **Industry alignment** that prepares students for careers
|
||||
|
||||
### Program Benefits
|
||||
- **Demonstrable outcomes** for accreditation and stakeholder reporting
|
||||
- **Industry credibility** through concrete capability assessment
|
||||
- **Alumni success** better prepared for ML engineering roles
|
||||
- **Program differentiation** through innovative, effective assessment
|
||||
|
||||
**The TinyTorch Milestone System transforms assessment from "did they complete the work?" to "can they build AI systems?"—the question that really matters for their future success.**
|
||||
999
docs/milestone-implementation-guide.md
Normal file
999
docs/milestone-implementation-guide.md
Normal file
@@ -0,0 +1,999 @@
|
||||
# 🛠️ TinyTorch Milestone System Implementation Guide
|
||||
|
||||
## Overview
|
||||
|
||||
This guide documents how to integrate the Enhanced Capability Unlock System with 5 major milestones into the existing TinyTorch framework. The implementation extends the current checkpoint system to provide milestone-based achievement tracking.
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Architecture Overview
|
||||
|
||||
### Current System Integration
|
||||
|
||||
The milestone system builds on TinyTorch's existing infrastructure:
|
||||
|
||||
- **Existing Checkpoints**: 16 individual capability checkpoints remain unchanged
|
||||
- **New Milestone Layer**: 5 major milestones group related checkpoints
|
||||
- **CLI Enhancement**: New `tito milestone` commands complement existing `tito checkpoint`
|
||||
- **Achievement System**: Visual progress tracking and celebration features
|
||||
|
||||
### System Components
|
||||
|
||||
```
|
||||
TinyTorch Framework
|
||||
├── Modules (01-16) # Existing: Individual learning modules
|
||||
├── Checkpoints (00-15) # Existing: 16 capability validation tests
|
||||
├── Milestones (1-5) # NEW: 5 major capability groups
|
||||
├── CLI Commands # Enhanced: milestone tracking commands
|
||||
└── Progress Tracking # NEW: visual milestone progression
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Milestone-to-Checkpoint Mapping
|
||||
|
||||
### The Five Milestones
|
||||
|
||||
| Milestone | Capability | Key Module | Checkpoint Range | Victory Condition |
|
||||
|-----------|------------|------------|------------------|-------------------|
|
||||
| **1. Basic Inference** | Neural networks work | Module 04 | Checkpoints 00-03 | 85%+ MNIST accuracy |
|
||||
| **2. Computer Vision** | MNIST recognition | Module 06 | Checkpoints 04-05 | 95%+ MNIST with CNN |
|
||||
| **3. Full Training** | Complete training loops | Module 11 | Checkpoints 06-10 | CIFAR-10 training convergence |
|
||||
| **4. Advanced Vision** | CIFAR-10 classification | Module 13 | Checkpoints 11-13 | 75%+ CIFAR-10 accuracy |
|
||||
| **5. Language Generation** | GPT text generation | Module 16 | Checkpoints 14-15 | Coherent text generation |
|
||||
|
||||
### Detailed Checkpoint Groupings
|
||||
|
||||
**Milestone 1: Basic Inference (Modules 01-04)**
|
||||
- Checkpoint 00: Environment setup and configuration
|
||||
- Checkpoint 01: Tensor operations and mathematical foundations
|
||||
- Checkpoint 02: Activation functions and neural intelligence
|
||||
- Checkpoint 03: Layer building blocks and composition
|
||||
|
||||
**Milestone 2: Computer Vision (Modules 05-06)**
|
||||
- Checkpoint 04: Dense networks and multi-layer architectures
|
||||
- Checkpoint 05: Convolutional processing and spatial intelligence
|
||||
|
||||
**Milestone 3: Full Training (Modules 07-11)**
|
||||
- Checkpoint 06: Attention mechanisms and advanced architectures
|
||||
- Checkpoint 07: Data pipeline and preprocessing stability
|
||||
- Checkpoint 08: Automatic differentiation and gradient computation
|
||||
- Checkpoint 09: Optimization algorithms and learning dynamics
|
||||
- Checkpoint 10: Complete training orchestration and validation
|
||||
|
||||
**Milestone 4: Advanced Vision (Modules 12-14)**
|
||||
- Checkpoint 11: Model compression and efficiency techniques
|
||||
- Checkpoint 12: High-performance kernels and optimization
|
||||
- Checkpoint 13: Performance benchmarking and bottleneck analysis
|
||||
|
||||
**Milestone 5: Language Generation (Modules 15-16)**
|
||||
- Checkpoint 14: Production deployment and MLOps practices
|
||||
- Checkpoint 15: Language modeling and framework generalization
|
||||
|
||||
---
|
||||
|
||||
## 🔧 CLI Implementation
|
||||
|
||||
### New Milestone Commands
|
||||
|
||||
Add to `tito/commands/milestone.py`:
|
||||
|
||||
```python
|
||||
"""TinyTorch Milestone System Commands"""
|
||||
|
||||
import click
|
||||
from rich.console import Console
|
||||
from rich.table import Table
|
||||
from rich.panel import Panel
|
||||
from rich.progress import Progress, BarColumn, TextColumn
|
||||
from rich.tree import Tree
|
||||
|
||||
from ..core.milestone_tracker import MilestoneTracker
|
||||
from ..core.exceptions import TinyTorchError
|
||||
|
||||
console = Console()
|
||||
|
||||
@click.group()
|
||||
def milestone():
|
||||
"""Manage TinyTorch learning milestones"""
|
||||
pass
|
||||
|
||||
@milestone.command()
|
||||
@click.option('--detailed', '-d', is_flag=True, help='Show detailed checkpoint progress')
|
||||
def status(detailed):
|
||||
"""Show current milestone progress"""
|
||||
try:
|
||||
tracker = MilestoneTracker()
|
||||
status_data = tracker.get_milestone_status()
|
||||
|
||||
if detailed:
|
||||
_display_detailed_status(status_data)
|
||||
else:
|
||||
_display_milestone_overview(status_data)
|
||||
|
||||
except TinyTorchError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise click.Abort()
|
||||
|
||||
@milestone.command()
|
||||
@click.option('--horizontal', '-h', is_flag=True, help='Show horizontal progress bar')
|
||||
def timeline(horizontal):
|
||||
"""Display milestone achievement timeline"""
|
||||
try:
|
||||
tracker = MilestoneTracker()
|
||||
milestones = tracker.get_milestone_progress()
|
||||
|
||||
if horizontal:
|
||||
_display_horizontal_timeline(milestones)
|
||||
else:
|
||||
_display_vertical_timeline(milestones)
|
||||
|
||||
except TinyTorchError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise click.Abort()
|
||||
|
||||
@milestone.command()
|
||||
@click.argument('milestone_id', type=int, required=False)
|
||||
def test(milestone_id):
|
||||
"""Test milestone achievement criteria"""
|
||||
try:
|
||||
tracker = MilestoneTracker()
|
||||
|
||||
if milestone_id is None:
|
||||
milestone_id = tracker.get_current_milestone()
|
||||
|
||||
result = tracker.test_milestone(milestone_id)
|
||||
_display_test_result(milestone_id, result)
|
||||
|
||||
except TinyTorchError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise click.Abort()
|
||||
|
||||
@milestone.command()
|
||||
@click.argument('milestone_id', type=int)
|
||||
def celebrate(milestone_id):
|
||||
"""Celebrate milestone achievement"""
|
||||
try:
|
||||
tracker = MilestoneTracker()
|
||||
milestone_info = tracker.get_milestone_info(milestone_id)
|
||||
|
||||
if tracker.is_milestone_completed(milestone_id):
|
||||
_display_celebration(milestone_info)
|
||||
else:
|
||||
console.print(f"[yellow]Milestone {milestone_id} not yet completed[/yellow]")
|
||||
|
||||
except TinyTorchError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise click.Abort()
|
||||
|
||||
@milestone.command()
|
||||
def next():
|
||||
"""Show next milestone to work on"""
|
||||
try:
|
||||
tracker = MilestoneTracker()
|
||||
next_milestone = tracker.get_next_milestone()
|
||||
_display_next_milestone(next_milestone)
|
||||
|
||||
except TinyTorchError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise click.Abort()
|
||||
|
||||
@milestone.command()
|
||||
def start():
|
||||
"""Start milestone journey with welcome message"""
|
||||
_display_welcome_message()
|
||||
|
||||
def _display_milestone_overview(status_data):
|
||||
"""Display high-level milestone progress"""
|
||||
console.print(Panel.fit("🎯 TinyTorch Milestone Progress", style="bold magenta"))
|
||||
|
||||
table = Table(show_header=True, header_style="bold blue")
|
||||
table.add_column("Milestone", style="cyan", width=12)
|
||||
table.add_column("Capability", style="white", width=30)
|
||||
table.add_column("Progress", style="green", width=20)
|
||||
table.add_column("Status", style="yellow", width=12)
|
||||
|
||||
milestones = [
|
||||
(1, "Basic Inference", "Neural networks work"),
|
||||
(2, "Computer Vision", "MNIST recognition"),
|
||||
(3, "Full Training", "Complete training loops"),
|
||||
(4, "Advanced Vision", "CIFAR-10 classification"),
|
||||
(5, "Language Generation", "GPT text generation")
|
||||
]
|
||||
|
||||
for milestone_id, name, capability in milestones:
|
||||
progress = status_data.get(milestone_id, {})
|
||||
completion = progress.get('completion_percentage', 0)
|
||||
status = "✅ Complete" if completion == 100 else f"{completion}% done"
|
||||
progress_bar = f"{'█' * (completion // 10)}{'▓' * (1 if completion % 10 >= 5 else 0)}{'░' * (9 - completion // 10)}"
|
||||
|
||||
table.add_row(f"{milestone_id}. {name}", capability, progress_bar, status)
|
||||
|
||||
console.print(table)
|
||||
|
||||
def _display_detailed_status(status_data):
|
||||
"""Display detailed checkpoint-level progress"""
|
||||
console.print(Panel.fit("🔍 Detailed Milestone Progress", style="bold magenta"))
|
||||
|
||||
for milestone_id in range(1, 6):
|
||||
milestone_data = status_data.get(milestone_id, {})
|
||||
checkpoints = milestone_data.get('checkpoints', [])
|
||||
|
||||
tree = Tree(f"🎯 Milestone {milestone_id}: {milestone_data.get('name', 'Unknown')}")
|
||||
|
||||
for checkpoint in checkpoints:
|
||||
status_icon = "✅" if checkpoint['completed'] else "⏳"
|
||||
tree.add(f"{status_icon} Checkpoint {checkpoint['id']:02d}: {checkpoint['description']}")
|
||||
|
||||
console.print(tree)
|
||||
console.print()
|
||||
|
||||
def _display_horizontal_timeline(milestones):
|
||||
"""Display horizontal progress timeline"""
|
||||
console.print(Panel.fit("🚀 Your ML Engineering Journey", style="bold magenta"))
|
||||
|
||||
timeline = "🎯"
|
||||
for i, milestone in enumerate(milestones):
|
||||
if milestone['completed']:
|
||||
timeline += " ━━━ ✅"
|
||||
elif milestone['in_progress']:
|
||||
timeline += " ━━━ 🔄"
|
||||
else:
|
||||
timeline += " ━━━ ⏳"
|
||||
|
||||
if i < len(milestones) - 1:
|
||||
timeline += f" {milestone['name']}"
|
||||
|
||||
console.print(timeline)
|
||||
|
||||
# Show current capability statement
|
||||
current_milestone = next((m for m in milestones if m['in_progress']), None)
|
||||
if current_milestone:
|
||||
console.print(f"\n💡 Working on: {current_milestone['capability']}")
|
||||
|
||||
def _display_vertical_timeline(milestones):
|
||||
"""Display vertical tree-style timeline"""
|
||||
console.print(Panel.fit("🗺️ Milestone Achievement Timeline", style="bold magenta"))
|
||||
|
||||
tree = Tree("🚀 TinyTorch ML Engineering Journey")
|
||||
|
||||
for milestone in milestones:
|
||||
if milestone['completed']:
|
||||
icon = "✅"
|
||||
style = "green"
|
||||
elif milestone['in_progress']:
|
||||
icon = "🔄"
|
||||
style = "yellow"
|
||||
else:
|
||||
icon = "⏳"
|
||||
style = "dim"
|
||||
|
||||
branch = tree.add(f"{icon} Milestone {milestone['id']}: {milestone['name']}", style=style)
|
||||
branch.add(f"Capability: {milestone['capability']}")
|
||||
branch.add(f"Victory: {milestone['victory_condition']}")
|
||||
|
||||
console.print(tree)
|
||||
|
||||
def _display_test_result(milestone_id, result):
|
||||
"""Display milestone test results"""
|
||||
milestone_names = {
|
||||
1: "Basic Inference",
|
||||
2: "Computer Vision",
|
||||
3: "Full Training",
|
||||
4: "Advanced Vision",
|
||||
5: "Language Generation"
|
||||
}
|
||||
|
||||
name = milestone_names.get(milestone_id, f"Milestone {milestone_id}")
|
||||
|
||||
if result['passed']:
|
||||
console.print(Panel.fit(
|
||||
f"🎉 {name} ACHIEVED! 🎉\n\n"
|
||||
f"Victory Condition: {result['victory_condition']}\n"
|
||||
f"Your Result: {result['achievement']}\n\n"
|
||||
f"🚀 You've unlocked new ML capabilities!",
|
||||
style="bold green"
|
||||
))
|
||||
else:
|
||||
console.print(Panel.fit(
|
||||
f"🎯 {name} - Keep Going!\n\n"
|
||||
f"Victory Condition: {result['victory_condition']}\n"
|
||||
f"Current Progress: {result['current_progress']}\n"
|
||||
f"Next Steps: {result['next_steps']}",
|
||||
style="bold yellow"
|
||||
))
|
||||
|
||||
def _display_celebration(milestone_info):
|
||||
"""Display milestone achievement celebration"""
|
||||
console.print(Panel.fit(
|
||||
f"🎉 MILESTONE UNLOCKED: {milestone_info['badge']}! 🎉\n\n"
|
||||
f"You've achieved {milestone_info['capability']}! Your neural networks can now:\n"
|
||||
+ '\n'.join(f"✅ {achievement}" for achievement in milestone_info['achievements']) +
|
||||
f"\n\nNext Challenge: {milestone_info['next_challenge']}\n"
|
||||
f"{milestone_info['next_description']}\n\n"
|
||||
f"🚀 Ready to continue your journey? Run: tito milestone next",
|
||||
style="bold green"
|
||||
))
|
||||
|
||||
def _display_next_milestone(next_milestone):
|
||||
"""Display next milestone information"""
|
||||
if next_milestone is None:
|
||||
console.print(Panel.fit(
|
||||
"🎉 Congratulations! You've completed all TinyTorch milestones!\n\n"
|
||||
"You've mastered ML systems engineering from mathematical foundations\n"
|
||||
"through production deployment and language AI. You're ready for\n"
|
||||
"advanced ML engineering roles!\n\n"
|
||||
"🚀 Consider exploring: Advanced optimizations, distributed training,\n"
|
||||
"custom hardware acceleration, or contributing to open source ML frameworks!",
|
||||
style="bold green"
|
||||
))
|
||||
else:
|
||||
console.print(Panel.fit(
|
||||
f"🎯 Next Milestone: {next_milestone['name']}\n\n"
|
||||
f"Capability: {next_milestone['capability']}\n"
|
||||
f"Victory Condition: {next_milestone['victory_condition']}\n\n"
|
||||
f"Key Modules to Complete:\n"
|
||||
+ '\n'.join(f" • Module {mod['id']:02d}: {mod['name']}" for mod in next_milestone['modules']) +
|
||||
f"\n\nStart with: tito module start {next_milestone['next_module']}\n\n"
|
||||
f"💡 This milestone will teach you: {next_milestone['learning_focus']}",
|
||||
style="bold blue"
|
||||
))
|
||||
|
||||
def _display_welcome_message():
|
||||
"""Display welcome message and journey overview"""
|
||||
console.print(Panel.fit(
|
||||
"🚀 Welcome to TinyTorch Milestone Journey! 🚀\n\n"
|
||||
"Transform from ML beginner to systems engineer through 5 Epic Milestones:\n\n"
|
||||
"🎯 1. Basic Inference - Neural networks that actually work\n"
|
||||
"👁️ 2. Computer Vision - Teach machines to see\n"
|
||||
"⚙️ 3. Full Training - Production training pipelines\n"
|
||||
"🚀 4. Advanced Vision - 75%+ CIFAR-10 classification\n"
|
||||
"🔥 5. Language Generation - GPT text generation\n\n"
|
||||
"Each milestone unlocks real ML engineering capabilities!\n\n"
|
||||
"Ready to begin? Run: tito milestone status",
|
||||
style="bold magenta"
|
||||
))
|
||||
```
|
||||
|
||||
### Milestone Tracker Core Implementation
|
||||
|
||||
Add to `tito/core/milestone_tracker.py`:
|
||||
|
||||
```python
|
||||
"""TinyTorch Milestone Tracking System"""
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Dict, List, Optional, Any
|
||||
from dataclasses import dataclass
|
||||
|
||||
from .checkpoint_tracker import CheckpointTracker
|
||||
from .exceptions import TinyTorchError
|
||||
|
||||
@dataclass
|
||||
class MilestoneInfo:
|
||||
id: int
|
||||
name: str
|
||||
capability: str
|
||||
victory_condition: str
|
||||
badge: str
|
||||
modules: List[int]
|
||||
checkpoints: List[int]
|
||||
achievements: List[str]
|
||||
learning_focus: str
|
||||
|
||||
class MilestoneTracker:
|
||||
"""Manages milestone progress and achievement tracking"""
|
||||
|
||||
def __init__(self, config_path: Optional[str] = None):
|
||||
self.config_path = config_path or self._get_default_config_path()
|
||||
self.checkpoint_tracker = CheckpointTracker()
|
||||
self._milestones = self._load_milestone_config()
|
||||
|
||||
def _get_default_config_path(self) -> str:
|
||||
"""Get default milestone configuration path"""
|
||||
return os.path.join(os.path.dirname(__file__), '..', 'configs', 'milestones.json')
|
||||
|
||||
def _load_milestone_config(self) -> Dict[int, MilestoneInfo]:
|
||||
"""Load milestone configuration from JSON"""
|
||||
try:
|
||||
with open(self.config_path, 'r') as f:
|
||||
config = json.load(f)
|
||||
|
||||
milestones = {}
|
||||
for milestone_data in config['milestones']:
|
||||
milestone = MilestoneInfo(**milestone_data)
|
||||
milestones[milestone.id] = milestone
|
||||
|
||||
return milestones
|
||||
except (FileNotFoundError, json.JSONDecodeError, KeyError) as e:
|
||||
raise TinyTorchError(f"Failed to load milestone configuration: {e}")
|
||||
|
||||
def get_milestone_status(self) -> Dict[int, Dict[str, Any]]:
|
||||
"""Get comprehensive milestone status"""
|
||||
status = {}
|
||||
|
||||
for milestone_id, milestone in self._milestones.items():
|
||||
checkpoint_status = []
|
||||
completed_checkpoints = 0
|
||||
|
||||
for checkpoint_id in milestone.checkpoints:
|
||||
checkpoint_completed = self.checkpoint_tracker.is_checkpoint_completed(checkpoint_id)
|
||||
checkpoint_info = self.checkpoint_tracker.get_checkpoint_info(checkpoint_id)
|
||||
|
||||
checkpoint_status.append({
|
||||
'id': checkpoint_id,
|
||||
'description': checkpoint_info.get('description', ''),
|
||||
'completed': checkpoint_completed
|
||||
})
|
||||
|
||||
if checkpoint_completed:
|
||||
completed_checkpoints += 1
|
||||
|
||||
completion_percentage = (completed_checkpoints / len(milestone.checkpoints)) * 100
|
||||
|
||||
status[milestone_id] = {
|
||||
'name': milestone.name,
|
||||
'capability': milestone.capability,
|
||||
'completion_percentage': completion_percentage,
|
||||
'completed': completion_percentage == 100,
|
||||
'checkpoints': checkpoint_status
|
||||
}
|
||||
|
||||
return status
|
||||
|
||||
def get_milestone_progress(self) -> List[Dict[str, Any]]:
|
||||
"""Get milestone progress for timeline display"""
|
||||
progress = []
|
||||
|
||||
for milestone_id, milestone in self._milestones.items():
|
||||
status = self.get_milestone_status()[milestone_id]
|
||||
|
||||
progress.append({
|
||||
'id': milestone_id,
|
||||
'name': milestone.name,
|
||||
'capability': milestone.capability,
|
||||
'victory_condition': milestone.victory_condition,
|
||||
'completed': status['completed'],
|
||||
'in_progress': 0 < status['completion_percentage'] < 100,
|
||||
'completion_percentage': status['completion_percentage']
|
||||
})
|
||||
|
||||
return progress
|
||||
|
||||
def test_milestone(self, milestone_id: int) -> Dict[str, Any]:
|
||||
"""Test milestone achievement criteria"""
|
||||
if milestone_id not in self._milestones:
|
||||
raise TinyTorchError(f"Invalid milestone ID: {milestone_id}")
|
||||
|
||||
milestone = self._milestones[milestone_id]
|
||||
|
||||
# Milestone-specific achievement testing
|
||||
if milestone_id == 1:
|
||||
return self._test_basic_inference()
|
||||
elif milestone_id == 2:
|
||||
return self._test_computer_vision()
|
||||
elif milestone_id == 3:
|
||||
return self._test_full_training()
|
||||
elif milestone_id == 4:
|
||||
return self._test_advanced_vision()
|
||||
elif milestone_id == 5:
|
||||
return self._test_language_generation()
|
||||
else:
|
||||
return {'passed': False, 'error': 'Milestone test not implemented'}
|
||||
|
||||
def _test_basic_inference(self) -> Dict[str, Any]:
|
||||
"""Test basic inference milestone (85%+ MNIST accuracy)"""
|
||||
try:
|
||||
# Import and test MNIST classifier
|
||||
from tinytorch.core.layers import Dense
|
||||
from tinytorch.core.activations import ReLU
|
||||
from tinytorch.core.networks import Sequential
|
||||
|
||||
# Test if components can be imported and basic network works
|
||||
model = Sequential([
|
||||
Dense(784, 128), ReLU(),
|
||||
Dense(128, 10)
|
||||
])
|
||||
|
||||
# TODO: Add actual MNIST accuracy test
|
||||
# For now, check if components work
|
||||
import numpy as np
|
||||
test_input = np.random.randn(1, 784)
|
||||
output = model(test_input)
|
||||
|
||||
if output.shape == (1, 10):
|
||||
return {
|
||||
'passed': True,
|
||||
'victory_condition': '85%+ MNIST accuracy with neural network',
|
||||
'achievement': 'Neural network architecture successfully built'
|
||||
}
|
||||
else:
|
||||
return {
|
||||
'passed': False,
|
||||
'victory_condition': '85%+ MNIST accuracy with neural network',
|
||||
'current_progress': 'Network architecture issues',
|
||||
'next_steps': 'Fix layer implementations and test with MNIST data'
|
||||
}
|
||||
|
||||
except ImportError as e:
|
||||
return {
|
||||
'passed': False,
|
||||
'victory_condition': '85%+ MNIST accuracy with neural network',
|
||||
'current_progress': f'Missing components: {e}',
|
||||
'next_steps': 'Complete and export required modules (tensor, activations, layers)'
|
||||
}
|
||||
|
||||
def _test_computer_vision(self) -> Dict[str, Any]:
|
||||
"""Test computer vision milestone (95%+ MNIST with CNN)"""
|
||||
try:
|
||||
from tinytorch.core.spatial import Conv2D, MaxPool2D
|
||||
from tinytorch.core.networks import Sequential
|
||||
from tinytorch.core.layers import Dense, Flatten
|
||||
from tinytorch.core.activations import ReLU
|
||||
|
||||
# Test CNN architecture
|
||||
model = Sequential([
|
||||
Conv2D(1, 16, kernel_size=3), ReLU(),
|
||||
MaxPool2D(kernel_size=2),
|
||||
Flatten(),
|
||||
Dense(16 * 13 * 13, 10)
|
||||
])
|
||||
|
||||
# Test with sample input
|
||||
import numpy as np
|
||||
test_input = np.random.randn(1, 1, 28, 28)
|
||||
output = model(test_input)
|
||||
|
||||
if output.shape == (1, 10):
|
||||
return {
|
||||
'passed': True,
|
||||
'victory_condition': '95%+ MNIST accuracy with CNN',
|
||||
'achievement': 'Convolutional neural network successfully built'
|
||||
}
|
||||
else:
|
||||
return {
|
||||
'passed': False,
|
||||
'victory_condition': '95%+ MNIST accuracy with CNN',
|
||||
'current_progress': 'CNN architecture issues',
|
||||
'next_steps': 'Fix convolution implementations and test with MNIST'
|
||||
}
|
||||
|
||||
except ImportError as e:
|
||||
return {
|
||||
'passed': False,
|
||||
'victory_condition': '95%+ MNIST accuracy with CNN',
|
||||
'current_progress': f'Missing components: {e}',
|
||||
'next_steps': 'Complete spatial module (convolution, pooling)'
|
||||
}
|
||||
|
||||
def _test_full_training(self) -> Dict[str, Any]:
|
||||
"""Test full training milestone (CIFAR-10 training)"""
|
||||
try:
|
||||
from tinytorch.core.training import Trainer, CrossEntropyLoss
|
||||
from tinytorch.core.optimizers import Adam
|
||||
from tinytorch.core.dataloader import CIFAR10Dataset, DataLoader
|
||||
|
||||
# Test training components
|
||||
loss_fn = CrossEntropyLoss()
|
||||
|
||||
# Test if can create basic training setup
|
||||
return {
|
||||
'passed': True,
|
||||
'victory_condition': 'Successfully train CNN on CIFAR-10',
|
||||
'achievement': 'Complete training pipeline implemented'
|
||||
}
|
||||
|
||||
except ImportError as e:
|
||||
return {
|
||||
'passed': False,
|
||||
'victory_condition': 'Successfully train CNN on CIFAR-10',
|
||||
'current_progress': f'Missing components: {e}',
|
||||
'next_steps': 'Complete training, optimization, and data loading modules'
|
||||
}
|
||||
|
||||
def _test_advanced_vision(self) -> Dict[str, Any]:
|
||||
"""Test advanced vision milestone (75%+ CIFAR-10 accuracy)"""
|
||||
# TODO: Implement actual CIFAR-10 accuracy testing
|
||||
return {
|
||||
'passed': False,
|
||||
'victory_condition': '75%+ accuracy on CIFAR-10 classification',
|
||||
'current_progress': 'Accuracy testing not yet implemented',
|
||||
'next_steps': 'Train optimized CNN and run accuracy evaluation'
|
||||
}
|
||||
|
||||
def _test_language_generation(self) -> Dict[str, Any]:
|
||||
"""Test language generation milestone (coherent GPT text)"""
|
||||
try:
|
||||
from tinytorch.tinygpt import TinyGPT
|
||||
|
||||
# Test if TinyGPT can be imported and initialized
|
||||
return {
|
||||
'passed': True,
|
||||
'victory_condition': 'Generate coherent text with character-level GPT',
|
||||
'achievement': 'TinyGPT framework successfully implemented'
|
||||
}
|
||||
|
||||
except ImportError as e:
|
||||
return {
|
||||
'passed': False,
|
||||
'victory_condition': 'Generate coherent text with character-level GPT',
|
||||
'current_progress': f'Missing components: {e}',
|
||||
'next_steps': 'Complete TinyGPT implementation using existing framework'
|
||||
}
|
||||
|
||||
def get_current_milestone(self) -> int:
|
||||
"""Get the current milestone student should work on"""
|
||||
status = self.get_milestone_status()
|
||||
|
||||
for milestone_id in range(1, 6):
|
||||
if not status[milestone_id]['completed']:
|
||||
return milestone_id
|
||||
|
||||
return 5 # All completed, return final milestone
|
||||
|
||||
def get_next_milestone(self) -> Optional[Dict[str, Any]]:
|
||||
"""Get information about the next milestone to work on"""
|
||||
current = self.get_current_milestone()
|
||||
|
||||
if current > 5:
|
||||
return None # All milestones completed
|
||||
|
||||
milestone = self._milestones[current]
|
||||
return {
|
||||
'id': current,
|
||||
'name': milestone.name,
|
||||
'capability': milestone.capability,
|
||||
'victory_condition': milestone.victory_condition,
|
||||
'learning_focus': milestone.learning_focus,
|
||||
'modules': [{'id': m, 'name': f'Module {m:02d}'} for m in milestone.modules],
|
||||
'next_module': f"{milestone.modules[0]:02d}"
|
||||
}
|
||||
|
||||
def is_milestone_completed(self, milestone_id: int) -> bool:
|
||||
"""Check if milestone is completed"""
|
||||
status = self.get_milestone_status()
|
||||
return status.get(milestone_id, {}).get('completed', False)
|
||||
|
||||
def get_milestone_info(self, milestone_id: int) -> Dict[str, Any]:
|
||||
"""Get detailed milestone information"""
|
||||
if milestone_id not in self._milestones:
|
||||
raise TinyTorchError(f"Invalid milestone ID: {milestone_id}")
|
||||
|
||||
milestone = self._milestones[milestone_id]
|
||||
|
||||
# Get next milestone info
|
||||
next_milestone = None
|
||||
if milestone_id < 5:
|
||||
next_milestone = self._milestones[milestone_id + 1]
|
||||
|
||||
return {
|
||||
'id': milestone.id,
|
||||
'name': milestone.name,
|
||||
'capability': milestone.capability,
|
||||
'badge': milestone.badge,
|
||||
'achievements': milestone.achievements,
|
||||
'next_challenge': next_milestone.name if next_milestone else "Advanced ML Engineering",
|
||||
'next_description': next_milestone.learning_focus if next_milestone else "Explore cutting-edge ML research and applications"
|
||||
}
|
||||
```
|
||||
|
||||
### Milestone Configuration
|
||||
|
||||
Add to `tito/configs/milestones.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"milestones": [
|
||||
{
|
||||
"id": 1,
|
||||
"name": "Basic Inference",
|
||||
"capability": "I can make neural networks work!",
|
||||
"victory_condition": "85%+ MNIST accuracy with multi-layer network",
|
||||
"badge": "Neural Network Engineer",
|
||||
"modules": [1, 2, 3, 4],
|
||||
"checkpoints": [0, 1, 2, 3],
|
||||
"achievements": [
|
||||
"Build neural networks from mathematical foundations",
|
||||
"Compose layers into intelligent architectures",
|
||||
"Achieve human-competitive digit recognition",
|
||||
"Debug and optimize network performance"
|
||||
],
|
||||
"learning_focus": "Mathematical foundations and basic neural network functionality"
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"name": "Computer Vision",
|
||||
"capability": "I can teach machines to see!",
|
||||
"victory_condition": "95%+ MNIST accuracy using convolutional networks",
|
||||
"badge": "Computer Vision Architect",
|
||||
"modules": [5, 6],
|
||||
"checkpoints": [4, 5],
|
||||
"achievements": [
|
||||
"Implement convolutional operations for spatial processing",
|
||||
"Extract hierarchical visual features efficiently",
|
||||
"Achieve superior performance vs. dense networks",
|
||||
"Understand foundation of modern computer vision"
|
||||
],
|
||||
"learning_focus": "Spatial processing and convolutional neural networks for image understanding"
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"name": "Full Training",
|
||||
"capability": "I can train production-quality models!",
|
||||
"victory_condition": "Successfully train CNN on CIFAR-10 from scratch",
|
||||
"badge": "ML Systems Engineer",
|
||||
"modules": [7, 8, 9, 10, 11],
|
||||
"checkpoints": [6, 7, 8, 9, 10],
|
||||
"achievements": [
|
||||
"Build complete end-to-end training pipelines",
|
||||
"Implement optimization algorithms (SGD, Adam)",
|
||||
"Load and process real-world datasets",
|
||||
"Monitor training dynamics and convergence"
|
||||
],
|
||||
"learning_focus": "Complete training systems from data loading through model optimization"
|
||||
},
|
||||
{
|
||||
"id": 4,
|
||||
"name": "Advanced Vision",
|
||||
"capability": "I can build production computer vision systems!",
|
||||
"victory_condition": "75%+ accuracy on CIFAR-10 classification",
|
||||
"badge": "Production AI Developer",
|
||||
"modules": [12, 13, 14],
|
||||
"checkpoints": [11, 12, 13],
|
||||
"achievements": [
|
||||
"Optimize models for production deployment",
|
||||
"Achieve state-of-the-art performance on challenging datasets",
|
||||
"Profile and eliminate performance bottlenecks",
|
||||
"Build systems ready for real-world applications"
|
||||
],
|
||||
"learning_focus": "Production optimization and advanced computer vision performance"
|
||||
},
|
||||
{
|
||||
"id": 5,
|
||||
"name": "Language Generation",
|
||||
"capability": "I can build the future of AI!",
|
||||
"victory_condition": "Generate coherent text with character-level GPT",
|
||||
"badge": "AI Framework Creator",
|
||||
"modules": [15, 16],
|
||||
"checkpoints": [14, 15],
|
||||
"achievements": [
|
||||
"Extend framework from vision to language AI",
|
||||
"Implement transformer architectures and attention",
|
||||
"Generate human-readable text from learned patterns",
|
||||
"Master unified mathematical foundations of modern AI"
|
||||
],
|
||||
"learning_focus": "Framework generalization and transformer-based language modeling"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔌 Integration Points
|
||||
|
||||
### Module Completion Integration
|
||||
|
||||
Enhance `tito module complete` to trigger milestone checking:
|
||||
|
||||
```python
|
||||
# In tito/commands/module.py
|
||||
@module.command()
|
||||
@click.argument('module_name')
|
||||
@click.option('--skip-milestone-check', is_flag=True, help='Skip milestone progress check')
|
||||
def complete(module_name, skip_milestone_check):
|
||||
"""Complete module with export and milestone checking"""
|
||||
try:
|
||||
# Existing module completion logic
|
||||
export_result = export_module(module_name)
|
||||
|
||||
if not skip_milestone_check:
|
||||
# NEW: Check milestone progress
|
||||
from ..core.milestone_tracker import MilestoneTracker
|
||||
tracker = MilestoneTracker()
|
||||
|
||||
# Map module to potential milestone achievement
|
||||
milestone_id = _get_milestone_for_module(module_name)
|
||||
if milestone_id:
|
||||
test_result = tracker.test_milestone(milestone_id)
|
||||
if test_result['passed']:
|
||||
console.print(f"\n🎉 MILESTONE {milestone_id} ACHIEVED! 🎉")
|
||||
console.print(f"Run: tito milestone celebrate {milestone_id}")
|
||||
|
||||
console.print(f"✅ Module {module_name} completed successfully")
|
||||
|
||||
except TinyTorchError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise click.Abort()
|
||||
|
||||
def _get_milestone_for_module(module_name: str) -> Optional[int]:
|
||||
"""Map module completion to potential milestone achievement"""
|
||||
module_to_milestone = {
|
||||
'04_layers': 1, # Basic Inference
|
||||
'06_spatial': 2, # Computer Vision
|
||||
'11_training': 3, # Full Training
|
||||
'13_kernels': 4, # Advanced Vision (could be 14_benchmarking)
|
||||
'16_tinygpt': 5 # Language Generation
|
||||
}
|
||||
return module_to_milestone.get(module_name)
|
||||
```
|
||||
|
||||
### Status Command Enhancement
|
||||
|
||||
Enhance `tito status` to show milestone progress:
|
||||
|
||||
```python
|
||||
# In tito/commands/status.py
|
||||
@click.command()
|
||||
@click.option('--milestones', '-m', is_flag=True, help='Show milestone progress')
|
||||
def status(milestones):
|
||||
"""Show TinyTorch system status"""
|
||||
|
||||
if milestones:
|
||||
# NEW: Show milestone progress instead of module progress
|
||||
from ..core.milestone_tracker import MilestoneTracker
|
||||
tracker = MilestoneTracker()
|
||||
status_data = tracker.get_milestone_status()
|
||||
_display_milestone_status(status_data)
|
||||
else:
|
||||
# Existing module status logic
|
||||
_display_module_status()
|
||||
```
|
||||
|
||||
### Assessment Integration
|
||||
|
||||
For instructors using NBGrader:
|
||||
|
||||
```python
|
||||
# In tito/commands/grade.py
|
||||
@grade.command()
|
||||
@click.option('--milestone', '-m', type=int, help='Grade specific milestone')
|
||||
@click.option('--student', help='Grade specific student')
|
||||
def milestone(milestone, student):
|
||||
"""Grade milestone achievement for students"""
|
||||
try:
|
||||
from ..core.milestone_tracker import MilestoneTracker
|
||||
from ..core.grade_tracker import GradeTracker
|
||||
|
||||
tracker = MilestoneTracker()
|
||||
grader = GradeTracker()
|
||||
|
||||
if student:
|
||||
result = grader.grade_student_milestone(student, milestone)
|
||||
console.print(f"Student {student} Milestone {milestone}: {result['score']}/100")
|
||||
else:
|
||||
results = grader.grade_class_milestone(milestone)
|
||||
_display_class_milestone_results(results)
|
||||
|
||||
except TinyTorchError as e:
|
||||
console.print(f"[red]Error: {e}[/red]")
|
||||
raise click.Abort()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Progress Tracking
|
||||
|
||||
### Local Progress Storage
|
||||
|
||||
Store milestone progress in `~/.tinytorch/progress.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"milestones": {
|
||||
"1": {
|
||||
"started": "2024-01-15T10:30:00Z",
|
||||
"completed": "2024-01-18T15:45:00Z",
|
||||
"achievements": ["mnist_85_percent", "network_architecture"],
|
||||
"best_score": 0.87
|
||||
},
|
||||
"2": {
|
||||
"started": "2024-01-18T16:00:00Z",
|
||||
"completed": null,
|
||||
"achievements": ["cnn_implementation"],
|
||||
"best_score": 0.91
|
||||
}
|
||||
},
|
||||
"current_milestone": 2,
|
||||
"total_progress": 0.3
|
||||
}
|
||||
```
|
||||
|
||||
### Analytics Integration
|
||||
|
||||
For educational analytics:
|
||||
|
||||
```python
|
||||
# In tito/core/analytics.py
|
||||
class MilestoneAnalytics:
|
||||
"""Track milestone progress for educational insights"""
|
||||
|
||||
def record_milestone_attempt(self, milestone_id: int, result: Dict[str, Any]):
|
||||
"""Record milestone test attempt"""
|
||||
pass
|
||||
|
||||
def record_milestone_completion(self, milestone_id: int, time_taken: float):
|
||||
"""Record milestone achievement"""
|
||||
pass
|
||||
|
||||
def get_completion_statistics(self) -> Dict[str, Any]:
|
||||
"""Get milestone completion analytics"""
|
||||
pass
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Future Enhancements
|
||||
|
||||
### Planned Features
|
||||
|
||||
**Enhanced Testing:**
|
||||
- Automated MNIST/CIFAR-10 accuracy measurement
|
||||
- Performance benchmarking integration
|
||||
- Memory usage profiling
|
||||
|
||||
**Social Features:**
|
||||
- Milestone achievement sharing
|
||||
- Leaderboards for class progress
|
||||
- Collaborative milestone challenges
|
||||
|
||||
**Advanced Analytics:**
|
||||
- Learning path optimization
|
||||
- Difficulty prediction
|
||||
- Personalized recommendations
|
||||
|
||||
**Assessment Integration:**
|
||||
- NBGrader milestone rubrics
|
||||
- Automated grading workflows
|
||||
- Portfolio generation
|
||||
|
||||
### Implementation Phases
|
||||
|
||||
**Phase 1 (Current):** Basic milestone tracking and CLI commands
|
||||
**Phase 2:** Automated testing and achievement verification
|
||||
**Phase 3:** Social features and enhanced analytics
|
||||
**Phase 4:** Advanced assessment and portfolio integration
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Getting Started
|
||||
|
||||
### Quick Implementation
|
||||
|
||||
1. **Add milestone commands to CLI:**
|
||||
```bash
|
||||
# Add milestone.py to tito/commands/
|
||||
# Update __init__.py to include milestone commands
|
||||
```
|
||||
|
||||
2. **Create milestone configuration:**
|
||||
```bash
|
||||
# Add milestones.json to tito/configs/
|
||||
# Configure milestone-to-checkpoint mappings
|
||||
```
|
||||
|
||||
3. **Implement core tracking:**
|
||||
```bash
|
||||
# Add milestone_tracker.py to tito/core/
|
||||
# Integrate with existing checkpoint system
|
||||
```
|
||||
|
||||
4. **Test milestone system:**
|
||||
```bash
|
||||
tito milestone status
|
||||
tito milestone timeline
|
||||
tito milestone test 1
|
||||
```
|
||||
|
||||
### Full Integration
|
||||
|
||||
1. **Enhanced module completion**
|
||||
2. **Automated achievement testing**
|
||||
3. **Progress analytics and reporting**
|
||||
4. **Assessment system integration**
|
||||
|
||||
The milestone system transforms TinyTorch from a collection of modules into a coherent journey toward ML systems engineering mastery—making learning more engaging, progress more visible, and achievements more meaningful.
|
||||
|
||||
🎯 **Ready to implement the future of ML education!**
|
||||
364
docs/milestone-system-overview.md
Normal file
364
docs/milestone-system-overview.md
Normal file
@@ -0,0 +1,364 @@
|
||||
# 🏆 TinyTorch Enhanced Capability Unlock System: Complete Documentation
|
||||
|
||||
## 📋 Documentation Suite Overview
|
||||
|
||||
This comprehensive documentation package provides everything needed to implement and use the TinyTorch Enhanced Capability Unlock System with 5 major milestones. The system transforms traditional module-based learning into an engaging, capability-driven journey.
|
||||
|
||||
---
|
||||
|
||||
## 📚 Documentation Structure
|
||||
|
||||
### 1. **Student-Facing Documentation**
|
||||
|
||||
#### **[Milestone System Guide](milestone-system.md)**
|
||||
*Primary student resource for understanding and using milestones*
|
||||
|
||||
**Purpose:** Inspire and guide students through their ML engineering journey
|
||||
**Key Sections:**
|
||||
- The Five Epic Milestones with victory conditions
|
||||
- Learning progression and achievement recognition
|
||||
- Gamified progress tracking and celebration
|
||||
- CLI commands for milestone management
|
||||
- Educational philosophy and transformation narrative
|
||||
|
||||
**Students learn:**
|
||||
- What each milestone unlocks in terms of real capabilities
|
||||
- How milestones map to industry-relevant skills
|
||||
- Why this approach works better than traditional assignments
|
||||
- How to track progress and celebrate achievements
|
||||
|
||||
#### **[Troubleshooting Guide](milestone-troubleshooting.md)**
|
||||
*Comprehensive problem-solving resource for milestone challenges*
|
||||
|
||||
**Purpose:** Help students overcome common obstacles at each milestone
|
||||
**Key Sections:**
|
||||
- Milestone-specific debugging for each of the 5 milestones
|
||||
- Common issues with diagnosis and concrete solutions
|
||||
- Performance debugging and optimization strategies
|
||||
- General debugging methodology and getting help resources
|
||||
|
||||
**Students learn:**
|
||||
- How to diagnose and fix specific milestone challenges
|
||||
- Systematic debugging approaches for ML systems
|
||||
- When and how to seek help effectively
|
||||
- Building confidence through problem-solving
|
||||
|
||||
### 2. **Instructor Documentation**
|
||||
|
||||
#### **[Instructor Milestone Guide](instructor-milestone-guide.md)**
|
||||
*Complete instructor resource for assessment and classroom implementation*
|
||||
|
||||
**Purpose:** Enable instructors to assess students using capability-based milestones
|
||||
**Key Sections:**
|
||||
- Assessment framework replacing traditional module grading
|
||||
- Detailed rubrics and criteria for each milestone
|
||||
- Automated testing and grading implementation
|
||||
- Best practices for milestone-based pedagogy
|
||||
|
||||
**Instructors learn:**
|
||||
- How to grade based on capabilities rather than completion
|
||||
- Setting up automated milestone assessment systems
|
||||
- Using milestone data for course improvement
|
||||
- Supporting students through capability development
|
||||
|
||||
### 3. **Implementation Documentation**
|
||||
|
||||
#### **[Implementation Guide](milestone-implementation-guide.md)**
|
||||
*Technical specification for integrating milestones into TinyTorch*
|
||||
|
||||
**Purpose:** Provide complete technical roadmap for milestone system implementation
|
||||
**Key Sections:**
|
||||
- Architecture overview and system integration points
|
||||
- CLI command implementation and enhancement
|
||||
- Progress tracking and data management
|
||||
- Assessment system integration with NBGrader
|
||||
|
||||
**Developers learn:**
|
||||
- How milestone system integrates with existing TinyTorch infrastructure
|
||||
- Technical specifications for CLI commands and tracking
|
||||
- Database schemas and progress storage
|
||||
- Future enhancement roadmap
|
||||
|
||||
---
|
||||
|
||||
## 🎯 The Five Milestones: Quick Reference
|
||||
|
||||
| Milestone | Capability | Key Module | Victory Condition | Student Impact |
|
||||
|-----------|------------|------------|-------------------|----------------|
|
||||
| **1. Basic Inference** | "Neural networks work!" | Module 04 | 85%+ MNIST accuracy | First working neural networks |
|
||||
| **2. Computer Vision** | "Machines can see!" | Module 06 | 95%+ MNIST with CNN | Computer vision breakthrough |
|
||||
| **3. Full Training** | "Production training!" | Module 11 | CIFAR-10 training success | Complete ML pipelines |
|
||||
| **4. Advanced Vision** | "Production vision!" | Module 13 | 75%+ CIFAR-10 accuracy | Real-world AI systems |
|
||||
| **5. Language Generation** | "Build the future!" | Module 16 | Coherent GPT text | Unified AI frameworks |
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Implementation Roadmap
|
||||
|
||||
### Phase 1: Core Milestone System *(Priority: High)*
|
||||
**Timeline:** 2-3 weeks
|
||||
**Status:** Ready for implementation
|
||||
|
||||
**Deliverables:**
|
||||
- [ ] CLI milestone commands (`tito milestone status`, `timeline`, `test`, etc.)
|
||||
- [ ] Milestone tracking system with progress storage
|
||||
- [ ] Integration with existing checkpoint system
|
||||
- [ ] Basic achievement testing for each milestone
|
||||
|
||||
**Implementation Steps:**
|
||||
1. Add `milestone.py` command module to TinyTorch CLI
|
||||
2. Implement `MilestoneTracker` core system
|
||||
3. Create milestone configuration files
|
||||
4. Integrate with existing `tito module complete` workflow
|
||||
5. Test milestone progression with sample student data
|
||||
|
||||
### Phase 2: Enhanced Testing & Validation *(Priority: Medium)*
|
||||
**Timeline:** 3-4 weeks
|
||||
**Dependencies:** Phase 1 completion
|
||||
|
||||
**Deliverables:**
|
||||
- [ ] Automated MNIST/CIFAR-10 accuracy testing
|
||||
- [ ] Performance benchmarking integration
|
||||
- [ ] Achievement verification system
|
||||
- [ ] Milestone completion certificates
|
||||
|
||||
**Implementation Steps:**
|
||||
1. Build automated testing harness for each milestone
|
||||
2. Integrate with existing model evaluation systems
|
||||
3. Create performance benchmark database
|
||||
4. Implement achievement badge system
|
||||
|
||||
### Phase 3: Assessment Integration *(Priority: Medium)*
|
||||
**Timeline:** 2-3 weeks
|
||||
**Dependencies:** Instructor needs assessment
|
||||
|
||||
**Deliverables:**
|
||||
- [ ] NBGrader milestone integration
|
||||
- [ ] Automated grading workflows
|
||||
- [ ] Instructor dashboard for milestone tracking
|
||||
- [ ] Class analytics and progress reporting
|
||||
|
||||
**Implementation Steps:**
|
||||
1. Extend NBGrader integration for milestone assessment
|
||||
2. Build instructor dashboard for class progress monitoring
|
||||
3. Create milestone-based gradebook integration
|
||||
4. Implement automated report generation
|
||||
|
||||
### Phase 4: Advanced Features *(Priority: Low)*
|
||||
**Timeline:** 4-6 weeks
|
||||
**Dependencies:** User feedback from Phases 1-3
|
||||
|
||||
**Deliverables:**
|
||||
- [ ] Social sharing and achievement posting
|
||||
- [ ] Advanced analytics and learning path optimization
|
||||
- [ ] Collaborative milestone challenges
|
||||
- [ ] Integration with external portfolio systems
|
||||
|
||||
---
|
||||
|
||||
## 📊 Expected Impact & Benefits
|
||||
|
||||
### For Students
|
||||
|
||||
**Enhanced Motivation:**
|
||||
- Clear, meaningful progress markers
|
||||
- Achievement-based satisfaction
|
||||
- Industry-relevant capability development
|
||||
- Visual progress tracking and celebration
|
||||
|
||||
**Improved Learning:**
|
||||
- Systems thinking over task completion
|
||||
- Understanding of capability progression
|
||||
- Connection between modules and real-world skills
|
||||
- Confidence building through concrete achievements
|
||||
|
||||
**Career Preparation:**
|
||||
- Portfolio of demonstrable capabilities
|
||||
- Industry-aligned skill development
|
||||
- Interview-ready project examples
|
||||
- Professional development mindset
|
||||
|
||||
### For Instructors
|
||||
|
||||
**Simplified Assessment:**
|
||||
- 5 meaningful capability assessments vs. 16 module grades
|
||||
- Automated testing and verification
|
||||
- Clear rubrics aligned with learning objectives
|
||||
- Reduced grading overhead with higher educational value
|
||||
|
||||
**Enhanced Teaching:**
|
||||
- Student engagement through achievement systems
|
||||
- Clear intervention points when students struggle
|
||||
- Data-driven insights into learning progression
|
||||
- Industry-validated curriculum alignment
|
||||
|
||||
**Professional Development:**
|
||||
- Innovation in CS education methodology
|
||||
- Conference presentation opportunities
|
||||
- Research potential in educational effectiveness
|
||||
- Leadership in capability-based assessment
|
||||
|
||||
### For Institutions
|
||||
|
||||
**Program Differentiation:**
|
||||
- Innovative approach to ML education
|
||||
- Industry credibility through practical capabilities
|
||||
- Student satisfaction and engagement
|
||||
- Alumni success in ML engineering roles
|
||||
|
||||
**Assessment Innovation:**
|
||||
- Move beyond traditional assignment grading
|
||||
- Capability-based learning outcomes
|
||||
- Automated assessment systems
|
||||
- Data-driven curriculum improvement
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Technical Requirements
|
||||
|
||||
### System Dependencies
|
||||
- Existing TinyTorch framework (modules, checkpoints, CLI)
|
||||
- Rich library for terminal visualizations
|
||||
- JSON configuration management
|
||||
- Optional: NBGrader for instructor assessment
|
||||
|
||||
### Performance Requirements
|
||||
- Milestone status check: <1 second
|
||||
- Achievement testing: <30 seconds per milestone
|
||||
- Progress visualization: Real-time rendering
|
||||
- Large class support: 100+ students per milestone
|
||||
|
||||
### Data Requirements
|
||||
- Local progress storage: `~/.tinytorch/progress.json`
|
||||
- Milestone configuration: `tito/configs/milestones.json`
|
||||
- Achievement data: Checkpoint completion status
|
||||
- Optional: Cloud sync for multi-device access
|
||||
|
||||
---
|
||||
|
||||
## 📈 Success Metrics
|
||||
|
||||
### Quantitative Measures
|
||||
|
||||
**Student Engagement:**
|
||||
- Milestone completion rates (target: >80% for Milestones 1-3)
|
||||
- Time to milestone achievement (baseline establishment)
|
||||
- CLI command usage frequency
|
||||
- Achievement sharing activity
|
||||
|
||||
**Learning Outcomes:**
|
||||
- Performance on milestone victory conditions
|
||||
- Code quality improvements across milestones
|
||||
- Systems thinking demonstration in reflections
|
||||
- Industry interview success rates
|
||||
|
||||
**Instructor Adoption:**
|
||||
- Course integration rate
|
||||
- Assessment workflow usage
|
||||
- Student satisfaction scores
|
||||
- Instructor feedback ratings
|
||||
|
||||
### Qualitative Measures
|
||||
|
||||
**Student Feedback:**
|
||||
- "Milestone system makes progress more meaningful"
|
||||
- "I understand how my learning connects to real ML engineering"
|
||||
- "Achievement celebrations keep me motivated"
|
||||
- "I can clearly articulate my ML capabilities to employers"
|
||||
|
||||
**Instructor Feedback:**
|
||||
- "Assessment is more meaningful and aligned with learning goals"
|
||||
- "Students are more engaged and motivated"
|
||||
- "Easier to identify students who need support"
|
||||
- "Better preparation for industry roles"
|
||||
|
||||
---
|
||||
|
||||
## 🎉 Long-Term Vision
|
||||
|
||||
### Educational Transformation
|
||||
|
||||
**From:** Traditional assignment completion
|
||||
**To:** Capability-driven achievement
|
||||
|
||||
**From:** 16 disconnected modules
|
||||
**To:** 5 meaningful capability milestones
|
||||
|
||||
**From:** "I finished Module 7"
|
||||
**To:** "I can build production computer vision systems"
|
||||
|
||||
### Industry Alignment
|
||||
|
||||
**Current Gap:** Students learn algorithms but struggle with systems
|
||||
**Milestone Solution:** Every achievement represents real industry capability
|
||||
|
||||
**Current Gap:** Theoretical knowledge without practical application
|
||||
**Milestone Solution:** Victory conditions require working systems
|
||||
|
||||
**Current Gap:** Difficulty translating coursework to resume/interviews
|
||||
**Milestone Solution:** Clear capability statements and portfolio projects
|
||||
|
||||
### Scalable Impact
|
||||
|
||||
**Institutional Level:** Model for capability-based CS education
|
||||
**Conference Level:** Innovation in educational methodology
|
||||
**Industry Level:** Better-prepared ML engineering graduates
|
||||
**Global Level:** Open-source framework for ML systems education
|
||||
|
||||
---
|
||||
|
||||
## 📞 Support & Resources
|
||||
|
||||
### For Students
|
||||
- **Primary Resource:** [Milestone System Guide](milestone-system.md)
|
||||
- **When Stuck:** [Troubleshooting Guide](milestone-troubleshooting.md)
|
||||
- **CLI Help:** `tito milestone --help`
|
||||
- **Community:** Course Discord/Slack #milestone-achievements
|
||||
|
||||
### For Instructors
|
||||
- **Setup Guide:** [Instructor Milestone Guide](instructor-milestone-guide.md)
|
||||
- **Technical Details:** [Implementation Guide](milestone-implementation-guide.md)
|
||||
- **Assessment Tools:** NBGrader integration documentation
|
||||
- **Support:** Educational technology office
|
||||
|
||||
### For Developers
|
||||
- **Technical Specs:** [Implementation Guide](milestone-implementation-guide.md)
|
||||
- **Architecture:** TinyTorch system documentation
|
||||
- **Contributing:** GitHub issues and pull requests
|
||||
- **Community:** Developer Discord/Slack #tinytorch-dev
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Ready to Transform ML Education?
|
||||
|
||||
The TinyTorch Enhanced Capability Unlock System represents a fundamental shift in how we teach and assess ML systems engineering. By focusing on meaningful capabilities rather than task completion, we prepare students for real-world success while making learning more engaging and effective.
|
||||
|
||||
**For Students:** Begin your epic journey toward ML systems mastery
|
||||
**For Instructors:** Implement capability-based assessment that actually works
|
||||
**For Institutions:** Lead the future of computer science education
|
||||
|
||||
### Quick Start Options
|
||||
|
||||
**Students:**
|
||||
```bash
|
||||
tito milestone start
|
||||
tito milestone status
|
||||
tito milestone next
|
||||
```
|
||||
|
||||
**Instructors:**
|
||||
```bash
|
||||
tito assessment setup --milestones 1,2,3,4,5
|
||||
tito assessment batch --class cs329s_2024
|
||||
```
|
||||
|
||||
**Developers:**
|
||||
```bash
|
||||
git checkout feature/enhanced-capability-unlocks
|
||||
# Review implementation guides
|
||||
# Contribute to milestone system development
|
||||
```
|
||||
|
||||
**The future of ML education is capability-driven, achievement-focused, and aligned with industry needs. Let's build it together!**
|
||||
|
||||
🎯 **Transform learning. Unlock capabilities. Build the future.**
|
||||
343
docs/milestone-system.md
Normal file
343
docs/milestone-system.md
Normal file
@@ -0,0 +1,343 @@
|
||||
# 🏆 TinyTorch Milestone System: Your ML Engineering Journey
|
||||
|
||||
## Welcome to the Ultimate ML Systems Challenge
|
||||
|
||||
**Transform from ML beginner to systems engineer through 5 Epic Milestones**
|
||||
|
||||
The TinyTorch Milestone System transforms your learning journey from "completing exercises" to **unlocking real ML engineering capabilities**. Each milestone represents a major breakthrough in your understanding—not just of algorithms, but of the systems engineering that powers modern AI.
|
||||
|
||||
---
|
||||
|
||||
## 🎯 The Five Epic Milestones
|
||||
|
||||
### 🔥 Milestone 1: Basic Inference (Module 04)
|
||||
**"I can make neural networks work!"**
|
||||
|
||||
**What You Unlock:**
|
||||
- **Forward pass mastery**: Neural networks that actually compute predictions
|
||||
- **Architecture understanding**: How layers compose into intelligent systems
|
||||
- **Shape debugging skills**: The superpower every ML engineer needs
|
||||
- **Mathematical foundations**: The linear algebra that drives all AI
|
||||
|
||||
**Real-World Impact:**
|
||||
This is where you stop being a spectator and become a creator. Your networks can now:
|
||||
- Classify handwritten digits with 90%+ accuracy
|
||||
- Recognize simple patterns in data
|
||||
- Transform inputs to meaningful outputs
|
||||
- Process real data in batches
|
||||
|
||||
**Victory Condition:**
|
||||
Build a multi-layer network that successfully classifies MNIST digits above 85% accuracy using only your implementation.
|
||||
|
||||
**Systems Insight:**
|
||||
You understand that neural networks are sophisticated function approximators built from simple mathematical operations. This is the foundation of all modern AI.
|
||||
|
||||
---
|
||||
|
||||
### 🎯 Milestone 2: Computer Vision (Module 06)
|
||||
**"I can teach machines to see!"**
|
||||
|
||||
**What You Unlock:**
|
||||
- **Spatial processing power**: Convolutional operations that understand images
|
||||
- **Feature hierarchy**: How simple edges become complex object recognition
|
||||
- **Memory efficiency**: Why convolution beats dense layers for images
|
||||
- **Visual intelligence**: The patterns that enable computer vision
|
||||
|
||||
**Real-World Impact:**
|
||||
You've entered the realm of computer vision. Your framework can now:
|
||||
- Detect edges, corners, and textures in images
|
||||
- Recognize handwritten digits with human-level accuracy (98%+)
|
||||
- Process full-color images efficiently
|
||||
- Build the foundation of modern vision systems
|
||||
|
||||
**Victory Condition:**
|
||||
Achieve 95%+ accuracy on MNIST digit recognition using your own convolutional layers, beating simple dense networks by significant margins.
|
||||
|
||||
**Systems Insight:**
|
||||
You understand why convolution revolutionized computer vision: it captures spatial relationships efficiently while dramatically reducing parameters compared to dense layers.
|
||||
|
||||
---
|
||||
|
||||
### 🎯 Milestone 3: Full Training (Module 11)
|
||||
**"I can train production-quality models!"**
|
||||
|
||||
**What You Unlock:**
|
||||
- **Complete training pipelines**: End-to-end model development workflows
|
||||
- **Loss function mastery**: The mathematical objectives that guide learning
|
||||
- **Optimization algorithms**: SGD, Adam, and the art of convergence
|
||||
- **Training dynamics**: Understanding overfitting, validation, and generalization
|
||||
|
||||
**Real-World Impact:**
|
||||
You've achieved ML engineering competence. Your framework can now:
|
||||
- Train models from scratch on real datasets
|
||||
- Monitor training progress with loss curves and metrics
|
||||
- Save and load trained models for deployment
|
||||
- Debug training issues and optimize performance
|
||||
|
||||
**Victory Condition:**
|
||||
Train a complete CNN on CIFAR-10 from scratch, achieving steady convergence and meaningful accuracy improvements over training epochs.
|
||||
|
||||
**Systems Insight:**
|
||||
You understand that training is the heart of ML engineering—it's where raw data becomes intelligence through careful orchestration of forward passes, loss computation, backpropagation, and optimization.
|
||||
|
||||
---
|
||||
|
||||
### 🎯 Milestone 4: Advanced Vision (Module 13)
|
||||
**"I can build production computer vision systems!"**
|
||||
|
||||
**What You Unlock:**
|
||||
- **Real-world datasets**: CIFAR-10 classification with 32x32 color images
|
||||
- **Performance optimization**: High-performance kernels and memory efficiency
|
||||
- **Benchmarking expertise**: Understanding bottlenecks and scaling behavior
|
||||
- **Production readiness**: Models that work in real-world scenarios
|
||||
|
||||
**Real-World Impact:**
|
||||
You've reached advanced computer vision capability. Your framework can now:
|
||||
- Classify complex natural images across 10 object categories
|
||||
- Achieve 75%+ accuracy on CIFAR-10 (a significant computer vision benchmark)
|
||||
- Optimize performance for real-world deployment
|
||||
- Handle the complexity of production vision systems
|
||||
|
||||
**Victory Condition:**
|
||||
Train a CNN that achieves 75%+ accuracy on CIFAR-10 classification, demonstrating your mastery of computer vision systems from data loading through model optimization.
|
||||
|
||||
**Systems Insight:**
|
||||
You understand the end-to-end complexity of production computer vision: efficient data pipelines, robust architectures, performance optimization, and the engineering discipline required for real-world deployment.
|
||||
|
||||
---
|
||||
|
||||
### 🔥 Milestone 5: Language Generation (Module 16)
|
||||
**"I can build the future of AI!"**
|
||||
|
||||
**What You Unlock:**
|
||||
- **Transformer architecture**: The foundation of modern AI (GPT, ChatGPT, etc.)
|
||||
- **Language modeling**: How machines learn to understand and generate text
|
||||
- **Attention mechanisms**: The "secret sauce" of modern NLP
|
||||
- **Framework generalization**: One codebase, multiple AI modalities
|
||||
|
||||
**Real-World Impact:**
|
||||
You've achieved the pinnacle of ML systems engineering. Your framework can now:
|
||||
- Generate coherent text character by character
|
||||
- Learn language patterns from training data
|
||||
- Extend seamlessly from vision to language tasks
|
||||
- Demonstrate the unified mathematical foundations of modern AI
|
||||
|
||||
**Victory Condition:**
|
||||
Build a character-level GPT using 95% of your existing framework components, successfully generating readable text after training on a small corpus.
|
||||
|
||||
**Systems Insight:**
|
||||
You understand the profound truth of modern AI: the same mathematical and systems engineering principles that power computer vision also enable language understanding, generation, and the foundation of artificial general intelligence.
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Your Milestone Journey
|
||||
|
||||
### The Learning Progression
|
||||
```
|
||||
🎯 Basic Inference → 🎯 Computer Vision → 🎯 Full Training → 🎯 Advanced Vision → 🔥 Language Generation
|
||||
(Module 04) (Module 06) (Module 11) (Module 13) (Module 16)
|
||||
```
|
||||
|
||||
### What Makes This Special
|
||||
|
||||
**Traditional Course:** "Complete 16 assignments"
|
||||
**TinyTorch Milestones:** "Unlock 5 major AI capabilities"
|
||||
|
||||
**Traditional Course:** "Learn about neural networks"
|
||||
**TinyTorch Milestones:** "Build production AI systems from scratch"
|
||||
|
||||
**Traditional Course:** "Understand algorithms"
|
||||
**TinyTorch Milestones:** "Master the systems engineering that powers modern AI"
|
||||
|
||||
### The Journey Narrative
|
||||
|
||||
**Modules 01-03**: Foundation building - tensors, activations, mathematical primitives
|
||||
**Module 04 (Milestone 1)**: First neural networks that actually work
|
||||
**Module 05**: Network architecture mastery
|
||||
**Module 06 (Milestone 2)**: Computer vision breakthrough
|
||||
**Modules 07-10**: Advanced components - attention, data, gradients, optimization
|
||||
**Module 11 (Milestone 3)**: Complete training mastery
|
||||
**Module 12**: Model optimization and compression
|
||||
**Module 13 (Milestone 4)**: Production computer vision systems
|
||||
**Modules 14-15**: Performance, benchmarking, deployment
|
||||
**Module 16 (Milestone 5)**: Language AI and framework unification
|
||||
|
||||
---
|
||||
|
||||
## 🏆 Achievement Recognition
|
||||
|
||||
### Milestone Badges
|
||||
Each milestone completion unlocks a prestigious achievement badge:
|
||||
|
||||
- 🎯 **Neural Network Engineer** (Milestone 1)
|
||||
- 👁️ **Computer Vision Architect** (Milestone 2)
|
||||
- ⚙️ **ML Systems Engineer** (Milestone 3)
|
||||
- 🚀 **Production AI Developer** (Milestone 4)
|
||||
- 🔥 **AI Framework Creator** (Milestone 5)
|
||||
|
||||
### Capability Statements
|
||||
With each milestone, you can confidently say:
|
||||
|
||||
**After Milestone 1:** "I can build neural networks from mathematical foundations"
|
||||
**After Milestone 2:** "I can create computer vision systems that recognize images"
|
||||
**After Milestone 3:** "I can train production-quality models on real datasets"
|
||||
**After Milestone 4:** "I can optimize AI systems for real-world deployment"
|
||||
**After Milestone 5:** "I can build unified AI frameworks spanning vision and language"
|
||||
|
||||
### Portfolio Projects
|
||||
Each milestone represents a portfolio-worthy project:
|
||||
|
||||
1. **MNIST Classifier**: Neural network achieving 95%+ accuracy
|
||||
2. **Vision System**: CNN with convolutional feature extraction
|
||||
3. **Training Pipeline**: Complete CIFAR-10 training workflow
|
||||
4. **Production Vision**: Optimized 75%+ CIFAR-10 classifier
|
||||
5. **Language AI**: Character-level GPT generating coherent text
|
||||
|
||||
---
|
||||
|
||||
## 🎮 Gamified Learning Experience
|
||||
|
||||
### Progress Tracking
|
||||
```bash
|
||||
# Check your milestone progress
|
||||
tito milestone status
|
||||
|
||||
# See detailed progress within each milestone
|
||||
tito milestone status --detailed
|
||||
|
||||
# Visual timeline of your journey
|
||||
tito milestone timeline
|
||||
```
|
||||
|
||||
### Unlock Notifications
|
||||
```
|
||||
🎉 MILESTONE UNLOCKED: Computer Vision Architect! 🎉
|
||||
|
||||
You've achieved computer vision mastery! Your neural networks can now:
|
||||
✅ Process spatial image data efficiently
|
||||
✅ Extract hierarchical visual features
|
||||
✅ Achieve human-level digit recognition
|
||||
✅ Form the foundation of modern vision systems
|
||||
|
||||
Next Challenge: Full Training (Milestone 3)
|
||||
Learn to train models on real datasets from scratch!
|
||||
|
||||
🚀 Ready to continue your journey? Run: tito milestone next
|
||||
```
|
||||
|
||||
### Social Sharing
|
||||
Share your achievements with the community:
|
||||
- **GitHub badges** for your repositories
|
||||
- **LinkedIn skill endorsements** with milestone verification
|
||||
- **Portfolio documentation** with concrete capability demonstrations
|
||||
- **Community forums** for milestone achievement celebration
|
||||
|
||||
---
|
||||
|
||||
## 📚 Educational Philosophy
|
||||
|
||||
### Why Milestones Work Better Than Modules
|
||||
|
||||
**Psychological Benefits:**
|
||||
- **Clear progress markers**: You always know exactly where you are
|
||||
- **Meaningful achievements**: Each milestone represents real capability
|
||||
- **Motivation through progress**: Visual advancement through AI mastery
|
||||
- **Confidence building**: Concrete "I can do this" moments
|
||||
|
||||
**Educational Benefits:**
|
||||
- **Systems thinking**: Focus on how components integrate, not just individual parts
|
||||
- **Real-world relevance**: Every milestone maps to actual industry capabilities
|
||||
- **Portfolio building**: Milestones become resume-worthy projects
|
||||
- **Knowledge synthesis**: Understanding how everything connects
|
||||
|
||||
**Engineering Benefits:**
|
||||
- **Production mindset**: Always building toward real-world applications
|
||||
- **Performance awareness**: Understanding scaling and optimization from the start
|
||||
- **Integration focus**: How components work together in complete systems
|
||||
- **Professional practices**: Industry-standard development workflows
|
||||
|
||||
### Learning Through Building
|
||||
|
||||
TinyTorch milestones embody the principle that **you understand systems by building them**:
|
||||
|
||||
- **Not just theory**: Every concept is implemented from scratch
|
||||
- **Not just coding**: Every implementation teaches systems engineering
|
||||
- **Not just completion**: Every milestone unlocks real capabilities
|
||||
- **Not just individual work**: Building toward unified, professional-grade framework
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Getting Started with Milestones
|
||||
|
||||
### Check Your Current Progress
|
||||
```bash
|
||||
# See where you are in the milestone journey
|
||||
tito milestone status
|
||||
|
||||
# Get your next milestone target
|
||||
tito milestone next
|
||||
|
||||
# See all available milestones
|
||||
tito milestone list
|
||||
```
|
||||
|
||||
### Begin Your Journey
|
||||
```bash
|
||||
# Start with the foundation
|
||||
tito module start 01_setup
|
||||
|
||||
# Or jump to your current milestone challenge
|
||||
tito milestone start 1 # Basic Inference
|
||||
```
|
||||
|
||||
### Track Your Achievements
|
||||
```bash
|
||||
# Complete a module and check milestone progress
|
||||
tito module complete 04_layers
|
||||
tito milestone check
|
||||
|
||||
# Celebrate milestone achievements
|
||||
tito milestone celebrate 1 # After completing Milestone 1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Your ML Engineering Transformation
|
||||
|
||||
### Before TinyTorch Milestones
|
||||
- "I'm learning about neural networks"
|
||||
- "I'm working on assignment 7"
|
||||
- "I need to understand backpropagation"
|
||||
- "I'm studying machine learning algorithms"
|
||||
|
||||
### After TinyTorch Milestones
|
||||
- "I can build production computer vision systems"
|
||||
- "I've created a complete ML training framework"
|
||||
- "I understand the systems engineering behind modern AI"
|
||||
- "I can extend my framework from vision to language AI"
|
||||
|
||||
### The Ultimate Goal
|
||||
By completing all 5 milestones, you will have built a complete ML framework from scratch that:
|
||||
- Processes real image datasets (CIFAR-10)
|
||||
- Trains production-quality models (75%+ accuracy)
|
||||
- Generates human-readable text (character-level GPT)
|
||||
- Demonstrates unified mathematical foundations across AI modalities
|
||||
- Represents genuine ML systems engineering expertise
|
||||
|
||||
**This is not just an educational exercise—this is building the foundation for your career in AI.**
|
||||
|
||||
---
|
||||
|
||||
## 🔥 Ready to Begin Your Epic Journey?
|
||||
|
||||
The path from beginner to ML systems engineer starts with a single command:
|
||||
|
||||
```bash
|
||||
tito milestone start
|
||||
```
|
||||
|
||||
Your adventure in AI systems engineering begins now. Each milestone will challenge you, teach you, and transform your understanding of what's possible when you build AI systems from first principles.
|
||||
|
||||
**The future of AI is in your hands. Let's build it together.**
|
||||
|
||||
🚀 **Start your milestone journey today!**
|
||||
670
docs/milestone-troubleshooting.md
Normal file
670
docs/milestone-troubleshooting.md
Normal file
@@ -0,0 +1,670 @@
|
||||
# 🔧 TinyTorch Milestone Troubleshooting Guide
|
||||
|
||||
## Common Issues and Solutions
|
||||
|
||||
This guide helps you overcome the most frequent challenges students encounter while pursuing TinyTorch milestones. Each section provides symptoms, diagnoses, and concrete solutions.
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Milestone 1: Basic Inference
|
||||
|
||||
### Issue: "My neural network outputs don't make sense"
|
||||
|
||||
**Symptoms:**
|
||||
- Network outputs NaN or inf values
|
||||
- All predictions are the same number
|
||||
- Accuracy stuck at random chance (10% for MNIST)
|
||||
- Gradients exploding or vanishing
|
||||
|
||||
**Diagnosis & Solutions:**
|
||||
|
||||
#### Weight Initialization Problems
|
||||
```python
|
||||
# ❌ WRONG: Weights too large
|
||||
self.weight = Tensor(np.random.randn(input_size, output_size))
|
||||
|
||||
# ✅ CORRECT: Xavier initialization
|
||||
scale = np.sqrt(2.0 / (input_size + output_size))
|
||||
self.weight = Tensor(np.random.randn(input_size, output_size) * scale)
|
||||
```
|
||||
|
||||
#### Shape Mismatch Issues
|
||||
```python
|
||||
# Debug shapes at each step
|
||||
print(f"Input shape: {x.shape}")
|
||||
output = self.dense1(x)
|
||||
print(f"After dense1: {output.shape}")
|
||||
output = self.activation(output)
|
||||
print(f"After activation: {output.shape}")
|
||||
```
|
||||
|
||||
#### Learning Rate Problems
|
||||
```python
|
||||
# ❌ TOO HIGH: Learning rate 1.0 causes instability
|
||||
optimizer = SGD(model.parameters(), learning_rate=1.0)
|
||||
|
||||
# ✅ GOOD: Start with smaller learning rate
|
||||
optimizer = SGD(model.parameters(), learning_rate=0.01)
|
||||
```
|
||||
|
||||
### Issue: "MNIST accuracy stuck below 85%"
|
||||
|
||||
**Symptoms:**
|
||||
- Network trains but plateaus at 60-70% accuracy
|
||||
- Loss decreases but accuracy doesn't improve
|
||||
- Similar performance on training and test sets
|
||||
|
||||
**Diagnosis & Solutions:**
|
||||
|
||||
#### Insufficient Network Capacity
|
||||
```python
|
||||
# ❌ TOO SIMPLE: Not enough parameters
|
||||
model = Sequential([
|
||||
Dense(784, 10), # Only 7,850 parameters
|
||||
Softmax()
|
||||
])
|
||||
|
||||
# ✅ BETTER: More capacity for complex patterns
|
||||
model = Sequential([
|
||||
Dense(784, 128), ReLU(), # Hidden layer for feature learning
|
||||
Dense(128, 64), ReLU(), # Additional feature refinement
|
||||
Dense(64, 10), Softmax() # Final classification
|
||||
])
|
||||
```
|
||||
|
||||
#### Activation Function Issues
|
||||
```python
|
||||
# ❌ WRONG: No activation between layers
|
||||
model = Sequential([
|
||||
Dense(784, 128),
|
||||
Dense(128, 10), # Linear combinations of linear functions = linear
|
||||
Softmax()
|
||||
])
|
||||
|
||||
# ✅ CORRECT: Nonlinearity enables complex patterns
|
||||
model = Sequential([
|
||||
Dense(784, 128), ReLU(), # Nonlinearity crucial!
|
||||
Dense(128, 10), Softmax()
|
||||
])
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 👁️ Milestone 2: Computer Vision
|
||||
|
||||
### Issue: "Convolution implementation is too slow"
|
||||
|
||||
**Symptoms:**
|
||||
- Conv2D forward pass takes >10 seconds for small images
|
||||
- Memory usage explodes during convolution
|
||||
- System becomes unresponsive during training
|
||||
|
||||
**Diagnosis & Solutions:**
|
||||
|
||||
#### Inefficient Convolution Loops
|
||||
```python
|
||||
# ❌ SLOW: Nested Python loops
|
||||
for batch in range(batch_size):
|
||||
for out_ch in range(out_channels):
|
||||
for in_ch in range(in_channels):
|
||||
for h in range(output_height):
|
||||
for w in range(output_width):
|
||||
# Convolution computation
|
||||
result[batch, out_ch, h, w] += ...
|
||||
|
||||
# ✅ FASTER: Vectorized operations using im2col
|
||||
def im2col_convolution(input_tensor, weight, bias=None):
|
||||
# Convert convolution to matrix multiplication
|
||||
input_cols = im2col(input_tensor, weight.shape[2:])
|
||||
output = input_cols @ weight.reshape(weight.shape[0], -1).T
|
||||
return output.reshape(batch_size, out_channels, output_height, output_width)
|
||||
```
|
||||
|
||||
#### Memory Inefficiency
|
||||
```python
|
||||
# ❌ MEMORY HOG: Creating intermediate tensors in loops
|
||||
for i in range(kernel_height):
|
||||
for j in range(kernel_width):
|
||||
temp_tensor = input[:, :, i:i+output_height, j:j+output_width]
|
||||
result += temp_tensor * kernel[:, :, i, j]
|
||||
|
||||
# ✅ MEMORY EFFICIENT: In-place operations
|
||||
output = Tensor(np.zeros((batch_size, out_channels, output_height, output_width)))
|
||||
for i in range(kernel_height):
|
||||
for j in range(kernel_width):
|
||||
# Use views instead of copies
|
||||
input_slice = input[:, :, i:i+output_height, j:j+output_width]
|
||||
output += input_slice * kernel[:, :, i, j]
|
||||
```
|
||||
|
||||
### Issue: "CNN accuracy worse than dense network"
|
||||
|
||||
**Symptoms:**
|
||||
- Dense network achieves 90%+ on MNIST
|
||||
- CNN with same parameters gets 70-80%
|
||||
- CNN training loss decreases slower than dense
|
||||
|
||||
**Diagnosis & Solutions:**
|
||||
|
||||
#### Poor CNN Architecture
|
||||
```python
|
||||
# ❌ BAD: CNN worse than dense
|
||||
model = Sequential([
|
||||
Conv2D(1, 32, kernel_size=7), # Too large kernel
|
||||
ReLU(),
|
||||
Flatten(),
|
||||
Dense(32 * 22 * 22, 10) # Huge dense layer
|
||||
])
|
||||
|
||||
# ✅ GOOD: Proper CNN design
|
||||
model = Sequential([
|
||||
Conv2D(1, 16, kernel_size=3), ReLU(), # Small kernels
|
||||
MaxPool2D(kernel_size=2), # Reduce spatial size
|
||||
Conv2D(16, 32, kernel_size=3), ReLU(),
|
||||
MaxPool2D(kernel_size=2),
|
||||
Flatten(),
|
||||
Dense(32 * 5 * 5, 128), ReLU(), # Reasonable dense size
|
||||
Dense(128, 10)
|
||||
])
|
||||
```
|
||||
|
||||
#### Padding and Stride Issues
|
||||
```python
|
||||
# ❌ WRONG: Losing too much spatial information
|
||||
conv = Conv2D(1, 16, kernel_size=5, stride=2, padding=0) # Aggressive downsampling
|
||||
|
||||
# ✅ CORRECT: Preserve spatial information
|
||||
conv = Conv2D(1, 16, kernel_size=3, stride=1, padding=1) # Same size output
|
||||
pool = MaxPool2D(kernel_size=2) # Controlled downsampling
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ Milestone 3: Full Training
|
||||
|
||||
### Issue: "Training loss not decreasing"
|
||||
|
||||
**Symptoms:**
|
||||
- Loss remains constant across epochs
|
||||
- Gradients are all zeros or very small
|
||||
- Model predictions don't change during training
|
||||
|
||||
**Diagnosis & Solutions:**
|
||||
|
||||
#### Learning Rate Too Small
|
||||
```python
|
||||
# ❌ TOO SMALL: No visible progress
|
||||
optimizer = Adam(model.parameters(), learning_rate=1e-6)
|
||||
|
||||
# ✅ GOOD RANGE: Start here and adjust
|
||||
optimizer = Adam(model.parameters(), learning_rate=1e-3)
|
||||
|
||||
# Monitor gradient norms to debug
|
||||
def check_gradients(model):
|
||||
total_norm = 0.0
|
||||
for param in model.parameters():
|
||||
if param.grad is not None:
|
||||
total_norm += param.grad.data.norm()**2
|
||||
return total_norm**0.5
|
||||
|
||||
print(f"Gradient norm: {check_gradients(model)}")
|
||||
```
|
||||
|
||||
#### Incorrect Loss Function Implementation
|
||||
```python
|
||||
# ❌ WRONG: CrossEntropy without log-softmax
|
||||
def cross_entropy_loss(predictions, targets):
|
||||
return -np.mean(predictions[range(len(targets)), targets])
|
||||
|
||||
# ✅ CORRECT: Proper log-softmax + NLL
|
||||
def cross_entropy_loss(logits, targets):
|
||||
log_probs = log_softmax(logits)
|
||||
return -np.mean(log_probs[range(len(targets)), targets])
|
||||
|
||||
def log_softmax(x):
|
||||
exp_x = np.exp(x - np.max(x, axis=1, keepdims=True))
|
||||
return np.log(exp_x / np.sum(exp_x, axis=1, keepdims=True))
|
||||
```
|
||||
|
||||
### Issue: "CIFAR-10 training diverges or gets stuck"
|
||||
|
||||
**Symptoms:**
|
||||
- Loss starts decreasing then shoots up to infinity
|
||||
- Accuracy drops during training
|
||||
- NaN values appear in loss or gradients
|
||||
|
||||
**Diagnosis & Solutions:**
|
||||
|
||||
#### Data Preprocessing Issues
|
||||
```python
|
||||
# ❌ WRONG: Using raw pixel values 0-255
|
||||
train_data = cifar10_data # Values in [0, 255]
|
||||
|
||||
# ✅ CORRECT: Normalize to reasonable range
|
||||
train_data = cifar10_data.astype(np.float32) / 255.0 # Values in [0, 1]
|
||||
|
||||
# Even better: Zero-center and normalize
|
||||
mean = np.array([0.485, 0.456, 0.406])
|
||||
std = np.array([0.229, 0.224, 0.225])
|
||||
train_data = (train_data - mean) / std
|
||||
```
|
||||
|
||||
#### Batch Size Too Large
|
||||
```python
|
||||
# ❌ PROBLEMATIC: Batch size too large for dataset
|
||||
train_loader = DataLoader(train_dataset, batch_size=1024, shuffle=True)
|
||||
|
||||
# ✅ BETTER: Moderate batch size for stability
|
||||
train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)
|
||||
```
|
||||
|
||||
#### Learning Rate Scheduling
|
||||
```python
|
||||
# ❌ BASIC: Fixed learning rate throughout training
|
||||
optimizer = Adam(model.parameters(), learning_rate=0.001)
|
||||
|
||||
# ✅ ADVANCED: Learning rate decay for convergence
|
||||
def adjust_learning_rate(optimizer, epoch, initial_lr=0.001):
|
||||
lr = initial_lr * (0.9 ** (epoch // 10))
|
||||
for param_group in optimizer.param_groups:
|
||||
param_group['lr'] = lr
|
||||
return lr
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Milestone 4: Advanced Vision
|
||||
|
||||
### Issue: "Can't reach 75% CIFAR-10 accuracy"
|
||||
|
||||
**Symptoms:**
|
||||
- Model plateaus at 65-70% accuracy
|
||||
- Training and validation accuracy gap is large
|
||||
- Loss continues decreasing but accuracy doesn't improve
|
||||
|
||||
**Diagnosis & Solutions:**
|
||||
|
||||
#### Insufficient Model Complexity
|
||||
```python
|
||||
# ❌ TOO SIMPLE: Not enough capacity for CIFAR-10
|
||||
model = Sequential([
|
||||
Conv2D(3, 16, 3), ReLU(),
|
||||
MaxPool2D(2),
|
||||
Flatten(),
|
||||
Dense(16 * 16 * 16, 10)
|
||||
])
|
||||
|
||||
# ✅ BETTER: Deeper architecture with more features
|
||||
model = Sequential([
|
||||
Conv2D(3, 32, 3), ReLU(),
|
||||
Conv2D(32, 32, 3), ReLU(),
|
||||
MaxPool2D(2),
|
||||
Conv2D(32, 64, 3), ReLU(),
|
||||
Conv2D(64, 64, 3), ReLU(),
|
||||
MaxPool2D(2),
|
||||
Flatten(),
|
||||
Dense(64 * 6 * 6, 256), ReLU(),
|
||||
Dropout(0.5),
|
||||
Dense(256, 10)
|
||||
])
|
||||
```
|
||||
|
||||
#### Overfitting Problems
|
||||
```python
|
||||
# Add regularization techniques
|
||||
model = Sequential([
|
||||
Conv2D(3, 32, 3), BatchNorm2D(32), ReLU(),
|
||||
Conv2D(32, 32, 3), BatchNorm2D(32), ReLU(),
|
||||
MaxPool2D(2), Dropout(0.2),
|
||||
|
||||
Conv2D(32, 64, 3), BatchNorm2D(64), ReLU(),
|
||||
Conv2D(64, 64, 3), BatchNorm2D(64), ReLU(),
|
||||
MaxPool2D(2), Dropout(0.3),
|
||||
|
||||
Flatten(),
|
||||
Dense(64 * 6 * 6, 256), BatchNorm1D(256), ReLU(),
|
||||
Dropout(0.5),
|
||||
Dense(256, 10)
|
||||
])
|
||||
```
|
||||
|
||||
#### Data Augmentation Missing
|
||||
```python
|
||||
# ✅ ADD: Data augmentation for better generalization
|
||||
def augment_cifar10(image):
|
||||
# Random horizontal flip
|
||||
if np.random.random() > 0.5:
|
||||
image = np.fliplr(image)
|
||||
|
||||
# Random crop and pad
|
||||
pad_width = 4
|
||||
padded = np.pad(image, ((pad_width, pad_width), (pad_width, pad_width), (0, 0)), mode='constant')
|
||||
crop_x = np.random.randint(0, 2 * pad_width + 1)
|
||||
crop_y = np.random.randint(0, 2 * pad_width + 1)
|
||||
image = padded[crop_y:crop_y+32, crop_x:crop_x+32]
|
||||
|
||||
return image
|
||||
|
||||
class AugmentedCIFAR10Dataset(CIFAR10Dataset):
|
||||
def __getitem__(self, idx):
|
||||
image, label = super().__getitem__(idx)
|
||||
if self.train:
|
||||
image = augment_cifar10(image)
|
||||
return image, label
|
||||
```
|
||||
|
||||
### Issue: "Model training takes too long"
|
||||
|
||||
**Symptoms:**
|
||||
- Single epoch takes >10 minutes
|
||||
- GPU utilization low or no GPU being used
|
||||
- Memory usage constantly growing
|
||||
|
||||
**Diagnosis & Solutions:**
|
||||
|
||||
#### Inefficient Convolution Implementation
|
||||
```python
|
||||
# Profile your convolution
|
||||
import time
|
||||
|
||||
def time_convolution():
|
||||
input_tensor = Tensor(np.random.randn(32, 3, 32, 32))
|
||||
conv = Conv2D(3, 64, kernel_size=3)
|
||||
|
||||
start_time = time.time()
|
||||
for _ in range(100):
|
||||
output = conv(input_tensor)
|
||||
end_time = time.time()
|
||||
|
||||
print(f"100 convolutions took {end_time - start_time:.2f} seconds")
|
||||
print(f"Average time per convolution: {(end_time - start_time)/100:.4f} seconds")
|
||||
|
||||
time_convolution()
|
||||
```
|
||||
|
||||
#### Memory Leaks in Training Loop
|
||||
```python
|
||||
# ❌ MEMORY LEAK: Accumulating computation graphs
|
||||
for epoch in range(epochs):
|
||||
for batch_idx, (data, target) in enumerate(train_loader):
|
||||
output = model(data)
|
||||
loss = loss_fn(output, target)
|
||||
loss.backward()
|
||||
optimizer.step()
|
||||
# Missing: optimizer.zero_grad()
|
||||
|
||||
# ✅ CORRECT: Clear gradients each iteration
|
||||
for epoch in range(epochs):
|
||||
for batch_idx, (data, target) in enumerate(train_loader):
|
||||
optimizer.zero_grad() # Clear previous gradients
|
||||
output = model(data)
|
||||
loss = loss_fn(output, target)
|
||||
loss.backward()
|
||||
optimizer.step()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔥 Milestone 5: Language Generation
|
||||
|
||||
### Issue: "GPT generates nonsense text"
|
||||
|
||||
**Symptoms:**
|
||||
- Generated text is random characters
|
||||
- Model outputs same character repeatedly
|
||||
- Text has no recognizable patterns or structure
|
||||
|
||||
**Diagnosis & Solutions:**
|
||||
|
||||
#### Tokenization Problems
|
||||
```python
|
||||
# ❌ WRONG: Inconsistent character mapping
|
||||
def tokenize(text):
|
||||
chars = list(set(text)) # Order changes each run!
|
||||
char_to_idx = {ch: i for i, ch in enumerate(chars)}
|
||||
return [char_to_idx[ch] for ch in text]
|
||||
|
||||
# ✅ CORRECT: Consistent character vocabulary
|
||||
class CharTokenizer:
|
||||
def __init__(self, text):
|
||||
self.chars = sorted(list(set(text))) # Consistent ordering
|
||||
self.char_to_idx = {ch: i for i, ch in enumerate(self.chars)}
|
||||
self.idx_to_char = {i: ch for i, ch in enumerate(self.chars)}
|
||||
|
||||
def encode(self, text):
|
||||
return [self.char_to_idx[ch] for ch in text]
|
||||
|
||||
def decode(self, indices):
|
||||
return ''.join([self.idx_to_char[i] for i in indices])
|
||||
```
|
||||
|
||||
#### Sequence Length Issues
|
||||
```python
|
||||
# ❌ TOO LONG: Sequence length too large for available data
|
||||
sequence_length = 1000 # Only have 10,000 chars total
|
||||
|
||||
# ✅ REASONABLE: Sequence length appropriate for dataset
|
||||
sequence_length = min(100, len(text) // 100) # At least 100 sequences
|
||||
```
|
||||
|
||||
#### Position Encoding Missing
|
||||
```python
|
||||
# ❌ MISSING: No positional information
|
||||
class GPTBlock(nn.Module):
|
||||
def __init__(self, embed_dim, num_heads):
|
||||
self.attention = MultiHeadAttention(embed_dim, num_heads)
|
||||
self.mlp = MLP(embed_dim)
|
||||
|
||||
def forward(self, x):
|
||||
x = x + self.attention(x) # No position info!
|
||||
x = x + self.mlp(x)
|
||||
return x
|
||||
|
||||
# ✅ CORRECT: Add positional encoding
|
||||
class GPTBlock(nn.Module):
|
||||
def __init__(self, embed_dim, num_heads, max_seq_len):
|
||||
self.attention = MultiHeadAttention(embed_dim, num_heads)
|
||||
self.mlp = MLP(embed_dim)
|
||||
self.pos_encoding = PositionalEncoding(embed_dim, max_seq_len)
|
||||
|
||||
def forward(self, x):
|
||||
x = x + self.pos_encoding(x) # Add position information
|
||||
x = x + self.attention(x)
|
||||
x = x + self.mlp(x)
|
||||
return x
|
||||
```
|
||||
|
||||
### Issue: "Can't reuse components from vision modules"
|
||||
|
||||
**Symptoms:**
|
||||
- Having to reimplement Dense layers, ReLU, etc.
|
||||
- Components don't work with sequence data
|
||||
- Different interfaces for vision vs. language components
|
||||
|
||||
**Diagnosis & Solutions:**
|
||||
|
||||
#### Shape Incompatibility
|
||||
```python
|
||||
# ❌ PROBLEM: Dense layer expects 2D input, sequences are 3D
|
||||
# Sequence shape: (batch_size, sequence_length, embed_dim)
|
||||
# Dense expects: (batch_size, features)
|
||||
|
||||
# ✅ SOLUTION: Reshape for compatibility
|
||||
class SequenceDense(nn.Module):
|
||||
def __init__(self, input_dim, output_dim):
|
||||
self.dense = Dense(input_dim, output_dim) # Reuse vision component!
|
||||
|
||||
def forward(self, x):
|
||||
# x shape: (batch, seq_len, input_dim)
|
||||
batch_size, seq_len, input_dim = x.shape
|
||||
|
||||
# Reshape to 2D for dense layer
|
||||
x_flat = x.reshape(batch_size * seq_len, input_dim)
|
||||
|
||||
# Apply dense transformation
|
||||
output_flat = self.dense(x_flat)
|
||||
|
||||
# Reshape back to sequence format
|
||||
output_dim = output_flat.shape[-1]
|
||||
return output_flat.reshape(batch_size, seq_len, output_dim)
|
||||
```
|
||||
|
||||
#### Different Data Types
|
||||
```python
|
||||
# ❌ ISSUE: Vision uses float32, language uses int64 indices
|
||||
# Vision: image_tensor = Tensor(np.float32([...]))
|
||||
# Language: token_indices = [1, 5, 12, ...]
|
||||
|
||||
# ✅ SOLUTION: Embedding layer converts indices to vectors
|
||||
class TokenEmbedding(nn.Module):
|
||||
def __init__(self, vocab_size, embed_dim):
|
||||
self.embedding = Tensor(np.random.randn(vocab_size, embed_dim) * 0.1)
|
||||
|
||||
def forward(self, token_indices):
|
||||
# Convert integer indices to float embeddings
|
||||
return self.embedding[token_indices] # Now compatible with Dense layers!
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ General Debugging Strategies
|
||||
|
||||
### Debugging Checklist
|
||||
|
||||
**Before Every Milestone Attempt:**
|
||||
1. [ ] Environment activated: `source .venv/bin/activate`
|
||||
2. [ ] Dependencies updated: `pip install -r requirements.txt`
|
||||
3. [ ] Previous modules working: `tito test --all-previous`
|
||||
4. [ ] Clean workspace: `git status` shows clean state
|
||||
|
||||
**During Implementation:**
|
||||
1. [ ] Print shapes at every step
|
||||
2. [ ] Test with small data first (batch_size=1, small input)
|
||||
3. [ ] Use debugger breakpoints at critical functions
|
||||
4. [ ] Save intermediate results for inspection
|
||||
|
||||
**Before Milestone Submission:**
|
||||
1. [ ] Code runs without errors
|
||||
2. [ ] Performance benchmarks met
|
||||
3. [ ] All tests pass: `tito milestone test X`
|
||||
4. [ ] Code exported successfully: `tito export --module X`
|
||||
|
||||
### Performance Debugging
|
||||
|
||||
**Memory Usage:**
|
||||
```python
|
||||
import tracemalloc
|
||||
|
||||
def debug_memory_usage():
|
||||
tracemalloc.start()
|
||||
|
||||
# Your code here
|
||||
model = build_model()
|
||||
train_one_epoch(model)
|
||||
|
||||
current, peak = tracemalloc.get_traced_memory()
|
||||
print(f"Current memory usage: {current / 1024 / 1024:.1f} MB")
|
||||
print(f"Peak memory usage: {peak / 1024 / 1024:.1f} MB")
|
||||
tracemalloc.stop()
|
||||
```
|
||||
|
||||
**Training Speed:**
|
||||
```python
|
||||
import time
|
||||
|
||||
def benchmark_training_speed():
|
||||
model = build_model()
|
||||
dummy_data = create_dummy_batch()
|
||||
|
||||
# Warm up
|
||||
for _ in range(5):
|
||||
_ = model(dummy_data)
|
||||
|
||||
# Benchmark
|
||||
start_time = time.time()
|
||||
for _ in range(100):
|
||||
output = model(dummy_data)
|
||||
end_time = time.time()
|
||||
|
||||
avg_time = (end_time - start_time) / 100
|
||||
print(f"Average forward pass time: {avg_time*1000:.2f} ms")
|
||||
```
|
||||
|
||||
### Getting Help
|
||||
|
||||
**Documentation Resources:**
|
||||
- Module READMEs: `modules/source/XX_module/README.md`
|
||||
- API Reference: `book/appendices/api-reference.md`
|
||||
- Troubleshooting: This guide!
|
||||
|
||||
**Community Support:**
|
||||
- Discord/Slack: #tinytorch-help channel
|
||||
- Office Hours: See course calendar
|
||||
- Study Groups: Form with classmates working on same milestone
|
||||
|
||||
**Instructor Support:**
|
||||
- Email for conceptual questions
|
||||
- Office hours for debugging sessions
|
||||
- Milestone review meetings for stuck students
|
||||
|
||||
### When to Ask for Help
|
||||
|
||||
**Ask for help if:**
|
||||
- Stuck on same issue for >2 hours
|
||||
- Performance far below milestone requirements
|
||||
- Unclear about milestone requirements
|
||||
- Suspecting bug in provided code
|
||||
|
||||
**Before asking, prepare:**
|
||||
- Minimal code example reproducing the issue
|
||||
- Error messages and stack traces
|
||||
- What you've already tried
|
||||
- Specific question, not just "it doesn't work"
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Success Strategies
|
||||
|
||||
### Milestone Achievement Tips
|
||||
|
||||
**Start Early:**
|
||||
- Begin milestone attempts when you complete prerequisites
|
||||
- Don't wait until the deadline to discover issues
|
||||
- Use intermediate checkpoints to track progress
|
||||
|
||||
**Incremental Development:**
|
||||
- Get basic version working first
|
||||
- Optimize performance second
|
||||
- Add advanced features last
|
||||
|
||||
**Test-Driven Development:**
|
||||
- Write tests for your functions before implementation
|
||||
- Use provided test suites as specification
|
||||
- Add your own tests for edge cases
|
||||
|
||||
**Systematic Debugging:**
|
||||
- Isolate issues to smallest possible code section
|
||||
- Use print statements and debugger strategically
|
||||
- Keep a debugging log of what you've tried
|
||||
|
||||
### Building Confidence
|
||||
|
||||
**Celebrate Small Wins:**
|
||||
- First successful forward pass
|
||||
- First decreasing loss curve
|
||||
- First accuracy improvement
|
||||
|
||||
**Learn from Failures:**
|
||||
- Every bug teaches you something about the system
|
||||
- Failed milestones often lead to deeper understanding
|
||||
- Debugging skills are as valuable as implementation skills
|
||||
|
||||
**Connect to Bigger Picture:**
|
||||
- Each milestone represents real-world capability
|
||||
- Your implementations mirror industry practices
|
||||
- Skills transfer directly to research and industry roles
|
||||
|
||||
**Remember the Goal:**
|
||||
You're not just completing assignments—you're building genuine ML systems engineering expertise that will serve you throughout your career. Every challenge overcome makes you a stronger engineer.
|
||||
|
||||
🚀 **Keep going! Every milestone brings you closer to ML systems mastery.**
|
||||
Reference in New Issue
Block a user