mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-06-02 08:32:31 -05:00
Cleaning up the repo
This commit is contained in:
Vijay Janapa Reddi
255
COURSE_GUIDE.md
255
COURSE_GUIDE.md
@@ -1,255 +0,0 @@
|
||||
# 🔥 TinyTorch Course Guide
|
||||
|
||||
**Building Machine Learning Systems from Scratch**
|
||||
|
||||
This guide helps you navigate through the complete TinyTorch course. Each project builds progressively toward a complete ML system.
|
||||
|
||||
## 🎯 Current Status
|
||||
|
||||
Track your progress through the course:
|
||||
|
||||
- [ ] **Project 0: Setup** - Environment & CLI setup
|
||||
- [ ] **Project 1: Tensor** - Core tensor operations
|
||||
- [ ] **Project 2: MLP** - Multi-layer perceptron
|
||||
- [ ] **Project 3: CNN** - Convolutional networks
|
||||
- [ ] **Project 4: Autograd** - Automatic differentiation
|
||||
- [ ] **Project 5: Data** - Data loading pipeline
|
||||
- [ ] **Project 6: Training** - Training loop & optimization
|
||||
- [ ] **Project 7: Config** - Configuration system
|
||||
- [ ] **Project 8: Profiling** - Performance profiling
|
||||
- [ ] **Project 9: Compression** - Model compression
|
||||
- [ ] **Project 10: Kernels** - Custom compute kernels
|
||||
- [ ] **Project 11: Benchmarking** - Performance benchmarking
|
||||
- [ ] **Project 12: MLOps** - Production monitoring
|
||||
|
||||
## 🚀 Getting Started
|
||||
|
||||
### First Time Setup
|
||||
1. **Clone the repository**
|
||||
2. **Go to**: [`projects/setup/README.md`](projects/setup/README.md)
|
||||
3. **Follow all setup instructions**
|
||||
4. **Verify with**: `python3 projects/setup/check_setup.py`
|
||||
|
||||
### Daily Workflow
|
||||
```bash
|
||||
cd TinyTorch
|
||||
source .venv/bin/activate # Always activate first!
|
||||
python3 bin/tito.py info # Check system status
|
||||
```
|
||||
|
||||
## 📋 Project Details
|
||||
|
||||
### 🔧 Project 0: Setup
|
||||
**Goal**: Get your development environment ready
|
||||
**Time**: 30 minutes
|
||||
**Instructions**: [`projects/setup/README.md`](projects/setup/README.md)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Create virtual environment
|
||||
- [ ] Install dependencies
|
||||
- [ ] Implement `hello_tinytorch()` function
|
||||
- [ ] Pass all setup tests
|
||||
- [ ] Learn the `tito` CLI
|
||||
|
||||
**Verification**:
|
||||
```bash
|
||||
python3 projects/setup/check_setup.py # Should show all ✅
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🔢 Project 1: Tensor
|
||||
**Goal**: Build the core tensor system
|
||||
**Prerequisites**: Project 0 complete
|
||||
**Instructions**: [`projects/tensor/README.md`](projects/tensor/README.md)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Implement `Tensor` class
|
||||
- [ ] Basic operations (add, mul, reshape)
|
||||
- [ ] Memory management
|
||||
- [ ] Shape validation
|
||||
- [ ] Broadcasting support
|
||||
|
||||
**Verification**:
|
||||
```bash
|
||||
python3 -m pytest projects/tensor/test_tensor.py -v
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🧠 Project 2: MLP
|
||||
**Goal**: Build multi-layer perceptron
|
||||
**Prerequisites**: Project 1 complete
|
||||
**Instructions**: [`projects/mlp/README.md`](projects/mlp/README.md)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Implement `Linear` layer
|
||||
- [ ] Activation functions (ReLU, Sigmoid)
|
||||
- [ ] Forward pass
|
||||
- [ ] Basic backward pass
|
||||
- [ ] Train on MNIST
|
||||
|
||||
**Target**: >95% accuracy on MNIST
|
||||
|
||||
---
|
||||
|
||||
### 🖼️ Project 3: CNN
|
||||
**Goal**: Build convolutional neural networks
|
||||
**Prerequisites**: Project 2 complete
|
||||
**Instructions**: [`projects/cnn/README.md`](projects/cnn/README.md)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Implement `Conv2d` layer
|
||||
- [ ] `MaxPool2d` layer
|
||||
- [ ] Padding and stride support
|
||||
- [ ] Train CNN on CIFAR-10
|
||||
|
||||
**Target**: >80% accuracy on CIFAR-10
|
||||
|
||||
---
|
||||
|
||||
### ⚡ Project 4: Autograd
|
||||
**Goal**: Automatic differentiation engine
|
||||
**Prerequisites**: Project 3 complete
|
||||
**Instructions**: [`projects/autograd/README.md`](projects/autograd/README.md)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Computational graph construction
|
||||
- [ ] Backward pass automation
|
||||
- [ ] Gradient checking
|
||||
- [ ] Memory efficient gradients
|
||||
|
||||
**Verification**: All gradient checks pass
|
||||
|
||||
---
|
||||
|
||||
### 📊 Project 5: Data
|
||||
**Goal**: Efficient data loading
|
||||
**Prerequisites**: Project 4 complete
|
||||
**Instructions**: [`projects/data/README.md`](projects/data/README.md)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Custom `DataLoader` implementation
|
||||
- [ ] Batch processing
|
||||
- [ ] Data transformations
|
||||
- [ ] Multi-threaded loading
|
||||
|
||||
---
|
||||
|
||||
### 🎯 Project 6: Training
|
||||
**Goal**: Complete training system
|
||||
**Prerequisites**: Project 5 complete
|
||||
**Instructions**: [`projects/training/README.md`](projects/training/README.md)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Training loop implementation
|
||||
- [ ] SGD optimizer
|
||||
- [ ] Adam optimizer
|
||||
- [ ] Learning rate scheduling
|
||||
- [ ] Metric tracking
|
||||
|
||||
---
|
||||
|
||||
### ⚙️ Project 7: Config
|
||||
**Goal**: Configuration management
|
||||
**Prerequisites**: Project 6 complete
|
||||
**Instructions**: [`projects/config/README.md`](projects/config/README.md)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] YAML configuration system
|
||||
- [ ] Experiment logging
|
||||
- [ ] Reproducible training
|
||||
- [ ] Hyperparameter management
|
||||
|
||||
---
|
||||
|
||||
### 📊 Project 8: Profiling
|
||||
**Goal**: Performance measurement
|
||||
**Prerequisites**: Project 7 complete
|
||||
**Instructions**: [`projects/profiling/README.md`](projects/profiling/README.md)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Memory profiler
|
||||
- [ ] Compute profiler
|
||||
- [ ] Bottleneck identification
|
||||
- [ ] Performance visualizations
|
||||
|
||||
---
|
||||
|
||||
### 🗜️ Project 9: Compression
|
||||
**Goal**: Model compression techniques
|
||||
**Prerequisites**: Project 8 complete
|
||||
**Instructions**: [`projects/compression/README.md`](projects/compression/README.md)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Pruning implementation
|
||||
- [ ] Quantization
|
||||
- [ ] Knowledge distillation
|
||||
- [ ] Compression benchmarks
|
||||
|
||||
---
|
||||
|
||||
### 🔥 Project 10: Kernels
|
||||
**Goal**: Custom compute kernels
|
||||
**Prerequisites**: Project 9 complete
|
||||
**Instructions**: [`projects/kernels/README.md`](projects/kernels/README.md)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Optimized matrix multiplication
|
||||
- [ ] Vectorized operations
|
||||
- [ ] CPU optimization
|
||||
- [ ] Performance comparisons
|
||||
|
||||
---
|
||||
|
||||
### 📈 Project 11: Benchmarking
|
||||
**Goal**: Performance evaluation
|
||||
**Prerequisites**: Project 10 complete
|
||||
**Instructions**: [`projects/benchmarking/README.md`](projects/benchmarking/README.md)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Comprehensive benchmarking suite
|
||||
- [ ] Performance regression testing
|
||||
- [ ] Comparison with other frameworks
|
||||
- [ ] Performance reporting
|
||||
|
||||
---
|
||||
|
||||
### 🚀 Project 12: MLOps
|
||||
**Goal**: Production monitoring
|
||||
**Prerequisites**: Project 11 complete
|
||||
**Instructions**: [`projects/mlops/README.md`](projects/mlops/README.md)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Data drift detection
|
||||
- [ ] Model performance monitoring
|
||||
- [ ] Auto-retraining system
|
||||
- [ ] Production deployment
|
||||
|
||||
## 🏆 Final Achievement
|
||||
|
||||
By the end of the course, you'll have:
|
||||
- ✅ Complete ML system built from scratch
|
||||
- ✅ CNN trained on CIFAR-10 achieving >85% accuracy
|
||||
- ✅ Production-ready MLOps pipeline
|
||||
- ✅ Deep understanding of ML system internals
|
||||
|
||||
## 🆘 Getting Help
|
||||
|
||||
**Stuck on a project?**
|
||||
1. Read the project's README thoroughly
|
||||
2. Run the verification commands
|
||||
3. Check the troubleshooting section
|
||||
4. Ask in office hours
|
||||
5. Review previous projects for patterns
|
||||
|
||||
**Common Commands**:
|
||||
```bash
|
||||
python3 bin/tito.py info # System status
|
||||
python3 bin/tito.py test --project setup # Test specific project
|
||||
python3 projects/setup/check_setup.py # Comprehensive verification
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Ready to start?** Go to [`projects/setup/README.md`](projects/setup/README.md) 🚀
|
||||
@@ -1,261 +0,0 @@
|
||||
# 🔥 TinyTorch Module Template
|
||||
|
||||
This document defines the standard structure that all TinyTorch modules should follow. This ensures consistency across the course and makes it easier for students to navigate and understand the codebase.
|
||||
|
||||
## 📁 Standard Module Structure
|
||||
|
||||
Each module should have the following structure:
|
||||
|
||||
```
|
||||
modules/[module-name]/
|
||||
├── README.md # 📖 Module overview and instructions
|
||||
├── notebook/ # 📓 Interactive development
|
||||
│ └── [module]_dev.ipynb # Main development notebook
|
||||
├── tutorials/ # 🎓 Step-by-step learning guides
|
||||
│ ├── 01_[topic]_basics.ipynb # Fundamental concepts
|
||||
│ ├── 02_[topic]_advanced.ipynb # (optional) Advanced topics
|
||||
│ └── ... # Additional tutorials as needed
|
||||
├── test_[module].py # 🧪 Automated tests (pytest)
|
||||
├── check_[module].py # ✅ Manual verification script
|
||||
├── solutions/ # 🔑 Reference solutions (instructors)
|
||||
│ └── solution_[module].py # Complete reference implementation
|
||||
└── [optional files] # Module-specific additional files
|
||||
```
|
||||
|
||||
## 📖 File Descriptions
|
||||
|
||||
### `README.md` - Module Overview
|
||||
**Purpose**: Primary entry point for students
|
||||
**Content**:
|
||||
- Learning objectives (what students will achieve)
|
||||
- Module structure overview
|
||||
- Step-by-step getting started guide
|
||||
- What students will implement
|
||||
- Testing instructions
|
||||
- Success criteria
|
||||
- Learning resources
|
||||
- Implementation tips
|
||||
- Next steps
|
||||
|
||||
**Template Sections**:
|
||||
1. **Module Title**: `# 🔥 Module XX: [Name]`
|
||||
2. **Learning Objectives**: Clear, measurable goals
|
||||
3. **Module Structure**: Visual directory tree
|
||||
4. **Getting Started**: Step-by-step workflow
|
||||
5. **What You'll Implement**: Clear task description
|
||||
6. **Testing Your Implementation**: All testing methods
|
||||
7. **Success Criteria**: When they're done
|
||||
8. **Learning Resources**: Tutorials and concepts
|
||||
9. **Implementation Tips**: Helpful guidance
|
||||
10. **Next Steps**: What comes after
|
||||
|
||||
### `notebook/[module]_dev.ipynb` - Development Notebook
|
||||
**Purpose**: Interactive development environment
|
||||
**Content**:
|
||||
- Environment setup verification
|
||||
- Import testing
|
||||
- Step-by-step implementation guide
|
||||
- Interactive testing
|
||||
- Progress tracking
|
||||
- Next steps
|
||||
|
||||
### `tutorials/` - Learning Resources
|
||||
**Purpose**: Educational content before implementation
|
||||
**Content**:
|
||||
- **01_[topic]_basics.ipynb**: Fundamental concepts
|
||||
- **02_[topic]_advanced.ipynb**: Advanced topics (optional)
|
||||
- Background theory
|
||||
- Examples and explanations
|
||||
- Best practices
|
||||
|
||||
### `test_[module].py` - Automated Tests
|
||||
**Purpose**: Comprehensive automated validation
|
||||
**Content**:
|
||||
- Pytest-compatible test functions
|
||||
- Import testing
|
||||
- Functionality testing
|
||||
- Edge case testing
|
||||
- Type checking
|
||||
- Documentation verification
|
||||
|
||||
**Naming Convention**: `test_[function_name]_[aspect]()`
|
||||
|
||||
### `check_[module].py` - Manual Verification
|
||||
**Purpose**: Human-readable progress tracking
|
||||
**Content**:
|
||||
- Detailed verification steps
|
||||
- Clear success/failure messages
|
||||
- Helpful error explanations
|
||||
- Progress visualization
|
||||
- Next steps guidance
|
||||
|
||||
### `solutions/solution_[module].py` - Reference Implementation
|
||||
**Purpose**: Complete working solution for instructors
|
||||
**Content**:
|
||||
- Full implementation
|
||||
- Comprehensive documentation
|
||||
- Example usage
|
||||
- Testing examples
|
||||
|
||||
## 🎯 Design Principles
|
||||
|
||||
### 1. **Progressive Complexity**
|
||||
- Start with simple concepts
|
||||
- Build complexity gradually
|
||||
- Each step prepares for the next
|
||||
|
||||
### 2. **Clear Learning Path**
|
||||
```
|
||||
README.md → tutorials/ → notebook/ → test → check → submit
|
||||
```
|
||||
|
||||
### 3. **Multiple Learning Styles**
|
||||
- **Visual learners**: Rich documentation with diagrams
|
||||
- **Hands-on learners**: Interactive notebooks
|
||||
- **Theory-first learners**: Tutorial notebooks
|
||||
- **Test-driven learners**: Clear test specifications
|
||||
|
||||
### 4. **Consistent Workflow**
|
||||
Every module follows the same pattern:
|
||||
1. Read overview (README.md)
|
||||
2. Study concepts (tutorials/)
|
||||
3. Implement interactively (notebook/)
|
||||
4. Test implementation (test + check)
|
||||
5. Submit and move forward
|
||||
|
||||
### 5. **Comprehensive Testing**
|
||||
- **Automated tests**: Fast, comprehensive validation
|
||||
- **Manual verification**: Human-readable feedback
|
||||
- **Integration testing**: Works with rest of system
|
||||
|
||||
## 🛠️ Implementation Guidelines
|
||||
|
||||
### Module README Template
|
||||
|
||||
```markdown
|
||||
# 🔥 Module XX: [Module Name]
|
||||
|
||||
[Brief welcome and overview]
|
||||
|
||||
## 🎯 Learning Objectives
|
||||
|
||||
By the end of this module, you will:
|
||||
- ✅ [Objective 1]
|
||||
- ✅ [Objective 2]
|
||||
- ✅ [Objective 3]
|
||||
|
||||
## 📋 Module Structure
|
||||
|
||||
[Directory tree with descriptions]
|
||||
|
||||
## 🚀 Getting Started
|
||||
|
||||
### Step 1: [Environment/Setup]
|
||||
### Step 2: [Learning]
|
||||
### Step 3: [Implementation]
|
||||
### Step 4: [Testing]
|
||||
### Step 5: [Submission]
|
||||
|
||||
## 📚 What You'll Implement
|
||||
|
||||
[Clear description and code examples]
|
||||
|
||||
## 🧪 Testing Your Implementation
|
||||
|
||||
### Automated Tests
|
||||
### Manual Verification
|
||||
|
||||
## 🎯 Success Criteria
|
||||
|
||||
[Clear completion criteria]
|
||||
|
||||
## 📖 Learning Resources
|
||||
## 🔧 Implementation Tips
|
||||
## 🚀 Next Steps
|
||||
## 💡 Need Help?
|
||||
```
|
||||
|
||||
### Development Notebook Template
|
||||
|
||||
```json
|
||||
{
|
||||
"cells": [
|
||||
{"cell_type": "markdown", "source": ["# Module Title", "## Learning Objectives"]},
|
||||
{"cell_type": "markdown", "source": ["## Step 1: Environment Check"]},
|
||||
{"cell_type": "code", "source": ["# Environment verification code"]},
|
||||
{"cell_type": "markdown", "source": ["## Step 2: Implementation"]},
|
||||
{"cell_type": "code", "source": ["# Implementation guidance"]},
|
||||
{"cell_type": "markdown", "source": ["## Step 3: Testing"]},
|
||||
{"cell_type": "code", "source": ["# Testing code"]},
|
||||
{"cell_type": "markdown", "source": ["## Next Steps"]}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## ✅ Quality Checklist
|
||||
|
||||
Before releasing a module, verify:
|
||||
|
||||
### Structure
|
||||
- [ ] All required files present
|
||||
- [ ] Consistent naming conventions
|
||||
- [ ] Proper directory structure
|
||||
|
||||
### Documentation
|
||||
- [ ] README.md follows template
|
||||
- [ ] Clear learning objectives
|
||||
- [ ] Step-by-step instructions
|
||||
- [ ] Complete success criteria
|
||||
|
||||
### Educational Content
|
||||
- [ ] Tutorial notebooks explain concepts
|
||||
- [ ] Development notebook guides implementation
|
||||
- [ ] Multiple learning approaches supported
|
||||
|
||||
### Testing
|
||||
- [ ] Comprehensive automated tests
|
||||
- [ ] Clear manual verification
|
||||
- [ ] Helpful error messages
|
||||
- [ ] Integration testing
|
||||
|
||||
### Student Experience
|
||||
- [ ] Clear progression path
|
||||
- [ ] Appropriate difficulty level
|
||||
- [ ] Builds on previous modules
|
||||
- [ ] Prepares for next modules
|
||||
|
||||
## 🔄 Module Dependencies
|
||||
|
||||
Modules should be designed with clear dependencies:
|
||||
|
||||
```
|
||||
setup → tensor → mlp → autograd → optimizer → training
|
||||
↓ ↓ ↓ ↓ ↓
|
||||
data → cnn → compression → benchmarking → profiling → config → mlops
|
||||
```
|
||||
|
||||
Each module should:
|
||||
- Build upon previous modules
|
||||
- Be self-contained where possible
|
||||
- Prepare foundation for future modules
|
||||
- Have clear prerequisite knowledge
|
||||
|
||||
## 📝 Maintenance Guidelines
|
||||
|
||||
### Regular Updates
|
||||
- Keep examples current
|
||||
- Update dependencies
|
||||
- Refresh learning resources
|
||||
- Improve error messages
|
||||
|
||||
### Student Feedback Integration
|
||||
- Track common issues
|
||||
- Update documentation based on confusion
|
||||
- Add clarifications for frequent questions
|
||||
- Improve progression flow
|
||||
|
||||
### Version Control
|
||||
- Tag stable versions
|
||||
- Document changes
|
||||
- Maintain backward compatibility
|
||||
- Test across Python versions
|
||||
6
Makefile
6
Makefile
@@ -1,4 +1,4 @@
|
||||
# TinyTorch - Notebook-first ML Systems Course
|
||||
# TinyTorch - Module-first ML Systems Course
|
||||
# Makefile for convenience commands
|
||||
|
||||
.PHONY: help install sync test clean docs jupyter lab setup
|
||||
@@ -21,8 +21,8 @@ help:
|
||||
@echo " doctor Run environment diagnosis"
|
||||
@echo ""
|
||||
@echo "Development workflow:"
|
||||
@echo " 1. make jupyter # Work in notebooks/"
|
||||
@echo " 2. make sync # Export to Python"
|
||||
@echo " 1. make jupyter # Work in modules/[name]/[name].ipynb"
|
||||
@echo " 2. make sync # Export to tinytorch package"
|
||||
@echo " 3. make test # Run tests"
|
||||
@echo " 4. make docs # Build docs (optional)"
|
||||
|
||||
|
||||
276
PROJECT_GUIDE.md
Normal file
276
PROJECT_GUIDE.md
Normal file
@@ -0,0 +1,276 @@
|
||||
# 🔥 TinyTorch Project Guide
|
||||
|
||||
**Building Machine Learning Systems from Scratch**
|
||||
|
||||
This guide helps you navigate through the complete TinyTorch course. Each module builds progressively toward a complete ML system using a notebook-first development approach with nbdev.
|
||||
|
||||
## 🎯 Module Progress Tracker
|
||||
|
||||
Track your progress through the course:
|
||||
|
||||
- [ ] **Module 0: Setup** - Environment & CLI setup
|
||||
- [ ] **Module 1: Tensor** - Core tensor operations
|
||||
- [ ] **Module 2: MLP** - Multi-layer perceptron
|
||||
- [ ] **Module 3: CNN** - Convolutional networks
|
||||
- [ ] **Module 4: Autograd** - Automatic differentiation
|
||||
- [ ] **Module 5: Data** - Data loading pipeline
|
||||
- [ ] **Module 6: Training** - Training loop & optimization
|
||||
- [ ] **Module 7: Config** - Configuration system
|
||||
- [ ] **Module 8: Profiling** - Performance profiling
|
||||
- [ ] **Module 9: Compression** - Model compression
|
||||
- [ ] **Module 10: Kernels** - Custom compute kernels
|
||||
- [ ] **Module 11: Benchmarking** - Performance benchmarking
|
||||
- [ ] **Module 12: MLOps** - Production monitoring
|
||||
|
||||
## 🚀 Getting Started
|
||||
|
||||
### First Time Setup
|
||||
1. **Clone the repository**
|
||||
2. **Go to**: [`modules/setup/README.md`](modules/setup/README.md)
|
||||
3. **Follow all setup instructions**
|
||||
4. **Verify with**: `python modules/setup/check_setup.py`
|
||||
|
||||
### Daily Workflow
|
||||
```bash
|
||||
cd TinyTorch
|
||||
source .venv/bin/activate # Always activate first!
|
||||
python bin/tito.py info # Check system status
|
||||
```
|
||||
|
||||
## 📋 Module Development Workflow
|
||||
|
||||
Each module follows this pattern:
|
||||
1. **Read overview**: `modules/[name]/README.md`
|
||||
2. **Work in notebook**: `modules/[name]/[name].ipynb`
|
||||
3. **Export code**: `python bin/tito.py sync`
|
||||
4. **Run tests**: `python bin/tito.py test --module [name]`
|
||||
5. **Move to next module when tests pass**
|
||||
|
||||
## 📚 Module Details
|
||||
|
||||
### 🔧 Module 0: Setup
|
||||
**Goal**: Get your development environment ready
|
||||
**Time**: 30 minutes
|
||||
**Location**: [`modules/setup/`](modules/setup/)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Create virtual environment
|
||||
- [ ] Install dependencies
|
||||
- [ ] Implement `hello_tinytorch()` function
|
||||
- [ ] Pass all setup tests
|
||||
- [ ] Learn the `tito` CLI
|
||||
|
||||
**Verification**:
|
||||
```bash
|
||||
python modules/setup/check_setup.py # Should show all ✅
|
||||
python bin/tito.py test --module setup
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🔢 Module 1: Tensor
|
||||
**Goal**: Build the core tensor system
|
||||
**Prerequisites**: Module 0 complete
|
||||
**Location**: [`modules/tensor/`](modules/tensor/)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Implement `Tensor` class in notebook
|
||||
- [ ] Basic operations (add, mul, reshape)
|
||||
- [ ] Memory management
|
||||
- [ ] Shape validation
|
||||
- [ ] Broadcasting support
|
||||
|
||||
**Verification**:
|
||||
```bash
|
||||
python bin/tito.py test --module tensor
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 🧠 Module 2: MLP
|
||||
**Goal**: Build multi-layer perceptron
|
||||
**Prerequisites**: Module 1 complete
|
||||
**Location**: [`modules/mlp/`](modules/mlp/)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Implement `Linear` layer
|
||||
- [ ] Activation functions (ReLU, Sigmoid)
|
||||
- [ ] Forward pass
|
||||
- [ ] Basic backward pass
|
||||
- [ ] Train on MNIST
|
||||
|
||||
**Target**: >95% accuracy on MNIST
|
||||
|
||||
---
|
||||
|
||||
### 🖼️ Module 3: CNN
|
||||
**Goal**: Build convolutional neural networks
|
||||
**Prerequisites**: Module 2 complete
|
||||
**Location**: [`modules/cnn/`](modules/cnn/)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Implement `Conv2d` layer
|
||||
- [ ] `MaxPool2d` layer
|
||||
- [ ] Padding and stride support
|
||||
- [ ] Train CNN on CIFAR-10
|
||||
|
||||
**Target**: >80% accuracy on CIFAR-10
|
||||
|
||||
---
|
||||
|
||||
### ⚡ Module 4: Autograd
|
||||
**Goal**: Automatic differentiation engine
|
||||
**Prerequisites**: Module 3 complete
|
||||
**Location**: [`modules/autograd/`](modules/autograd/)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Computational graph construction
|
||||
- [ ] Backward pass automation
|
||||
- [ ] Gradient checking
|
||||
- [ ] Memory efficient gradients
|
||||
|
||||
**Verification**: All gradient checks pass
|
||||
|
||||
---
|
||||
|
||||
### 📊 Module 5: Data
|
||||
**Goal**: Efficient data loading
|
||||
**Prerequisites**: Module 4 complete
|
||||
**Location**: [`modules/data/`](modules/data/)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Custom `DataLoader` implementation
|
||||
- [ ] Batch processing
|
||||
- [ ] Data transformations
|
||||
- [ ] Multi-threaded loading
|
||||
|
||||
---
|
||||
|
||||
### 🎯 Module 6: Training
|
||||
**Goal**: Complete training system
|
||||
**Prerequisites**: Module 5 complete
|
||||
**Location**: [`modules/training/`](modules/training/)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Training loop implementation
|
||||
- [ ] SGD optimizer
|
||||
- [ ] Adam optimizer
|
||||
- [ ] Learning rate scheduling
|
||||
- [ ] Metric tracking
|
||||
|
||||
---
|
||||
|
||||
### ⚙️ Module 7: Config
|
||||
**Goal**: Configuration management
|
||||
**Prerequisites**: Module 6 complete
|
||||
**Location**: [`modules/config/`](modules/config/)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] YAML configuration system
|
||||
- [ ] Experiment logging
|
||||
- [ ] Reproducible training
|
||||
- [ ] Hyperparameter management
|
||||
|
||||
---
|
||||
|
||||
### 📊 Module 8: Profiling
|
||||
**Goal**: Performance measurement
|
||||
**Prerequisites**: Module 7 complete
|
||||
**Location**: [`modules/profiling/`](modules/profiling/)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Memory profiler
|
||||
- [ ] Compute profiler
|
||||
- [ ] Bottleneck identification
|
||||
- [ ] Performance visualizations
|
||||
|
||||
---
|
||||
|
||||
### 🗜️ Module 9: Compression
|
||||
**Goal**: Model compression techniques
|
||||
**Prerequisites**: Module 8 complete
|
||||
**Location**: [`modules/compression/`](modules/compression/)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Pruning implementation
|
||||
- [ ] Quantization
|
||||
- [ ] Knowledge distillation
|
||||
- [ ] Compression benchmarks
|
||||
|
||||
---
|
||||
|
||||
### 🔥 Module 10: Kernels
|
||||
**Goal**: Custom compute kernels
|
||||
**Prerequisites**: Module 9 complete
|
||||
**Location**: [`modules/kernels/`](modules/kernels/)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Optimized matrix multiplication
|
||||
- [ ] Vectorized operations
|
||||
- [ ] CPU optimization
|
||||
|
||||
---
|
||||
|
||||
### 📈 Module 11: Benchmarking
|
||||
**Goal**: Performance benchmarking system
|
||||
**Prerequisites**: Module 10 complete
|
||||
**Location**: [`modules/benchmarking/`](modules/benchmarking/)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Benchmark suite implementation
|
||||
- [ ] Performance regression testing
|
||||
- [ ] Comparative analysis
|
||||
- [ ] Automated reporting
|
||||
|
||||
---
|
||||
|
||||
### 🚀 Module 12: MLOps
|
||||
**Goal**: Production monitoring and deployment
|
||||
**Prerequisites**: Module 11 complete
|
||||
**Location**: [`modules/mlops/`](modules/mlops/)
|
||||
|
||||
**Key Tasks**:
|
||||
- [ ] Model monitoring
|
||||
- [ ] Production deployment
|
||||
- [ ] A/B testing framework
|
||||
- [ ] Performance dashboards
|
||||
|
||||
## 🧪 Testing Strategy
|
||||
|
||||
### Module-Level Testing
|
||||
```bash
|
||||
# Test specific module
|
||||
python bin/tito.py test --module tensor
|
||||
|
||||
# Test all modules
|
||||
python bin/tito.py test
|
||||
|
||||
# Check overall status
|
||||
python bin/tito.py info
|
||||
```
|
||||
|
||||
### Integration Testing
|
||||
Each module integrates into the main `tinytorch` package through nbdev:
|
||||
- Notebooks automatically export to `tinytorch/core/`
|
||||
- Integration tests verify cross-module compatibility
|
||||
- Final package assembly validates the complete system
|
||||
|
||||
## 🏆 Completion Criteria
|
||||
|
||||
A module is complete when:
|
||||
- [ ] All notebook cells run without errors
|
||||
- [ ] `tito sync` exports code successfully
|
||||
- [ ] `tito test --module [name]` passes all tests
|
||||
- [ ] Integration with previous modules works
|
||||
- [ ] Ready to proceed to next module
|
||||
|
||||
## 🎓 Learning Philosophy
|
||||
|
||||
This course teaches **systems thinking** by building one cohesive ML system rather than isolated components. Each module contributes essential functionality that later modules depend upon, creating a complete, production-ready machine learning framework.
|
||||
|
||||
## 💡 Need Help?
|
||||
|
||||
- **Quick Start**: See [`modules/setup/QUICKSTART.md`](modules/setup/QUICKSTART.md)
|
||||
- **Development Workflow**: Each module's README.md
|
||||
- **CLI Reference**: `python bin/tito.py --help`
|
||||
- **Integration Issues**: Check `python bin/tito.py info`
|
||||
@@ -1,129 +0,0 @@
|
||||
# TinyTorch Structure Reorganization Proposal
|
||||
|
||||
## Current Problem
|
||||
The current structure is confusing because we have "projects" that are really just parts of one big project (TinyTorch). This makes it unclear that students are building one cohesive ML system.
|
||||
|
||||
## Proposed New Structure
|
||||
|
||||
```
|
||||
TinyTorch/
|
||||
├── parts/ # 🧩 System Components (was "projects")
|
||||
│ ├── 01_setup/ # Environment & onboarding
|
||||
│ ├── 02_tensor/ # Core tensor operations
|
||||
│ ├── 03_mlp/ # Multi-layer perceptron
|
||||
│ ├── 04_cnn/ # Convolutional networks
|
||||
│ ├── 05_autograd/ # Automatic differentiation
|
||||
│ ├── 06_data/ # Data loading pipeline
|
||||
│ ├── 07_training/ # Training loop & optimization
|
||||
│ ├── 08_config/ # Configuration system
|
||||
│ ├── 09_profiling/ # Performance profiling
|
||||
│ ├── 10_compression/ # Model compression
|
||||
│ ├── 11_kernels/ # Custom compute kernels
|
||||
│ ├── 12_benchmarking/ # Performance benchmarking
|
||||
│ └── 13_mlops/ # Production monitoring
|
||||
├── notebooks/ # 📓 Interactive Development
|
||||
│ ├── 01_tensor_dev.ipynb # Tensor development
|
||||
│ ├── 02_mlp_dev.ipynb # MLP development
|
||||
│ ├── 03_cnn_dev.ipynb # CNN development
|
||||
│ └── tutorials/ # Step-by-step guides
|
||||
├── tinytorch/ # 🏗️ Compiled Package
|
||||
│ └── core/ # Generated from notebooks
|
||||
├── docs/ # 📚 Documentation
|
||||
├── examples/ # 💡 Working Examples
|
||||
├── tests/ # 🧪 Test Suite
|
||||
└── bin/ # 🛠️ CLI Tools
|
||||
└── tito.py # Main CLI
|
||||
```
|
||||
|
||||
## Benefits of This Structure
|
||||
|
||||
### Clearer Mental Model
|
||||
- **Parts**: Students understand they're building parts of one system
|
||||
- **Progressive**: Each part builds on the previous ones
|
||||
- **Cohesive**: Clear that everything works together
|
||||
|
||||
### Better Learning Flow
|
||||
1. **Setup** (01_setup) - Get environment ready
|
||||
2. **Core** (02-04) - Build fundamental components
|
||||
3. **Engine** (05-07) - Build the training engine
|
||||
4. **Optimization** (08-12) - Add performance features
|
||||
5. **Production** (13) - Add MLOps capabilities
|
||||
|
||||
### Improved CLI Commands
|
||||
```bash
|
||||
# Instead of confusing "projects"
|
||||
tito test --project tensor
|
||||
|
||||
# Clear "parts"
|
||||
tito test --part tensor
|
||||
tito info --part tensor
|
||||
tito submit --part tensor
|
||||
```
|
||||
|
||||
## Migration Plan
|
||||
|
||||
### 1. Rename Directory
|
||||
```bash
|
||||
mv projects parts
|
||||
```
|
||||
|
||||
### 2. Update CLI Commands
|
||||
- `--project` → `--part`
|
||||
- `projects/` → `parts/`
|
||||
- Update all references in tito.py
|
||||
|
||||
### 3. Update Documentation
|
||||
- README.md references
|
||||
- Course guide references
|
||||
- All documentation paths
|
||||
|
||||
### 4. Update Notebooks
|
||||
- Development notebooks reference parts
|
||||
- Tutorials reference parts
|
||||
- nbdev configuration
|
||||
|
||||
## Example Usage
|
||||
|
||||
### For Students
|
||||
```bash
|
||||
# Clear progression through parts
|
||||
cd parts/01_setup/
|
||||
# ... work on setup
|
||||
|
||||
cd ../02_tensor/
|
||||
# ... work on tensor operations
|
||||
|
||||
cd ../03_mlp/
|
||||
# ... work on MLP implementation
|
||||
```
|
||||
|
||||
### For CLI
|
||||
```bash
|
||||
# Test specific part
|
||||
tito test --part tensor
|
||||
|
||||
# Check status of all parts
|
||||
tito info
|
||||
|
||||
# Submit completed part
|
||||
tito submit --part tensor
|
||||
```
|
||||
|
||||
## Alternative Names
|
||||
|
||||
If "parts" doesn't feel right, other options:
|
||||
- `components/` - System components
|
||||
- `modules/` - System modules
|
||||
- `stages/` - Development stages
|
||||
- `phases/` - Development phases
|
||||
- `steps/` - Implementation steps
|
||||
|
||||
## Implementation
|
||||
|
||||
Would you like me to:
|
||||
1. Create the new directory structure
|
||||
2. Update the CLI commands
|
||||
3. Migrate the existing content
|
||||
4. Update all documentation references
|
||||
|
||||
This would make the learning journey much clearer - students are building one cohesive ML system, not separate projects!
|
||||
33
nbdev.yaml
33
nbdev.yaml
@@ -1,33 +0,0 @@
|
||||
# nbdev configuration for TinyTorch
|
||||
# This file tells nbdev how to compile notebooks into the tinytorch package
|
||||
|
||||
# Package settings
|
||||
lib_name: tinytorch
|
||||
lib_path: tinytorch
|
||||
doc_path: docs
|
||||
nbs_path: modules
|
||||
recursive: true
|
||||
|
||||
# Documentation settings
|
||||
doc_host: https://tinytorch.github.io
|
||||
doc_baseurl: /TinyTorch/
|
||||
git_url: https://github.com/tinytorch/TinyTorch
|
||||
lib_https: https://github.com/tinytorch/TinyTorch/blob/main/
|
||||
baseurl: https://tinytorch.github.io
|
||||
|
||||
# Build settings
|
||||
custom_sidebar: false
|
||||
use_relative_doc_path: true
|
||||
use_github: true
|
||||
github_username: tinytorch
|
||||
github_repo: TinyTorch
|
||||
|
||||
# Notebook settings
|
||||
tst_flags: notest
|
||||
version: 0.1.0
|
||||
title: TinyTorch
|
||||
description: Build ML systems from scratch
|
||||
author: TinyTorch Students
|
||||
author_email: tinytorch@example.com
|
||||
copyright: 2024 TinyTorch
|
||||
license: MIT
|
||||
@@ -13,7 +13,7 @@ black_formatting = False
|
||||
### nbdev ###
|
||||
doc_path = _docs
|
||||
lib_path = tinytorch
|
||||
nbs_path = notebooks
|
||||
nbs_path = modules
|
||||
recursive = True
|
||||
tst_flags = notest
|
||||
put_version_in_init = True
|
||||
@@ -39,5 +39,4 @@ user = tinytorch
|
||||
|
||||
### Optional ###
|
||||
requirements = numpy>=1.20.0 matplotlib>=3.3.0 rich>=10.0.0 jupyter>=1.0.0 pytest>=6.0.0
|
||||
dev_requirements = nbdev>=2.3.0 black>=22.0.0
|
||||
console_scripts = tito=tinytorch.cli:main
|
||||
dev_requirements = nbdev>=2.3.0 black>=22.0.0
|
||||
Reference in New Issue
Block a user