mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2025-12-05 19:17:52 -06:00
Remove temporary planning documents and prepare package for pip install
- Remove DECEMBER_2024_RELEASE.md (temporary release checklist) - Remove module_20_consistency_review.md (internal review document) - Remove PEDAGOGICAL_IMPROVEMENTS_PLAN.md (planning document) - Remove benchmark_results/benchmark_report.md (generated report) - Fix pyproject.toml for proper pip installation: - Update version to 0.1.0 to match __init__.py - Fix package discovery to use tinytorch instead of tinytorch_placeholder - Add proper dependencies (numpy, rich, PyYAML) - Add optional dev dependencies - Update project URLs and metadata - Exclude non-package directories from distribution - Add MANIFEST.in to ensure LICENSE and README are included in package
This commit is contained in:
Vijay Janapa Reddi
@@ -1,314 +0,0 @@
|
||||
# TinyTorch December 2024 Community Release Checklist
|
||||
|
||||
**Target**: December 2024 community launch as a functional educational framework
|
||||
**Focus**: Individual learners (classroom integration coming in future releases)
|
||||
**Goal**: Stable, well-documented system for building ML frameworks from scratch
|
||||
|
||||
---
|
||||
|
||||
## ✅ Documentation (CRITICAL)
|
||||
|
||||
### Core Documentation
|
||||
- [x] **Student workflow documented** - Clear edit → export → validate cycle
|
||||
- [x] **Module count corrected** - All docs show 20 modules consistently
|
||||
- [x] **FAQ created** - Addresses "why TinyTorch vs alternatives"
|
||||
- [x] **Datasets documented** - Clear explanation of shipped vs downloaded data
|
||||
- [ ] **README.md polished** - First impression for GitHub visitors
|
||||
- [ ] **LICENSE verified** - Appropriate open-source license in place
|
||||
- [ ] **CONTRIBUTING.md** - Guidelines for community contributions
|
||||
- [ ] **Installation guide tested** - Setup works on Mac/Linux/Windows
|
||||
|
||||
### Module Documentation
|
||||
- [ ] **All 20 ABOUT.md files complete** - Each module has learning objectives
|
||||
- [ ] **Module numbering verified** - 01-20 with correct tier assignments
|
||||
- [ ] **Prerequisites documented** - Clear dependency chains
|
||||
- [ ] **Time estimates realistic** - Accurate completion time expectations
|
||||
|
||||
### Milestone Documentation
|
||||
- [x] **All 6 milestone READMEs standardized** - Historical context + requirements
|
||||
- [ ] **Expected results documented** - Clear success criteria per milestone
|
||||
- [ ] **Troubleshooting sections** - Common issues and solutions
|
||||
- [ ] **Dataset requirements clear** - Which datasets needed per milestone
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Technical Validation (CRITICAL)
|
||||
|
||||
### Environment Setup
|
||||
- [ ] **setup-environment.sh tested** on:
|
||||
- [ ] macOS (M1/M2 arm64)
|
||||
- [ ] macOS (Intel x86_64)
|
||||
- [ ] Linux (Ubuntu 22.04)
|
||||
- [ ] Linux (Ubuntu 20.04)
|
||||
- [ ] Windows (WSL2)
|
||||
- [ ] **Dependencies verified** - All packages install correctly
|
||||
- [ ] **Version pins checked** - Compatible NumPy, Jupyter, etc.
|
||||
- [ ] **Virtual environment isolation** - No conflicts with system Python
|
||||
|
||||
### TITO CLI Commands
|
||||
- [ ] **`tito system doctor`** - Comprehensive environment checks
|
||||
- [ ] **`tito system info`** - Shows correct configuration
|
||||
- [ ] **`tito module complete N`** - Exports work correctly for all 20 modules
|
||||
- [ ] **`tito checkpoint status`** - Optional checkpoint tracking works
|
||||
- [ ] **Error messages helpful** - Clear guidance when things fail
|
||||
|
||||
### Module Export System
|
||||
- [ ] **Export validates** - All 20 modules export without errors
|
||||
- [ ] **Import verification** - Exported modules importable from tinytorch.*
|
||||
- [ ] **Dependency handling** - Modules export in correct order
|
||||
- [ ] **File structure correct** - Modules land in right package locations
|
||||
|
||||
### Milestone Execution
|
||||
- [ ] **M01: Perceptron** - Runs successfully with module 07 exports
|
||||
- [ ] **M02: XOR** - Trains and solves XOR problem
|
||||
- [ ] **M03: MLP** - Achieves 85%+ on TinyDigits, 90%+ on MNIST
|
||||
- [ ] **M04: CNN** - Achieves 70%+ on CIFAR-10
|
||||
- [ ] **M05: Transformer** - Generates coherent text
|
||||
- [ ] **M06: MLPerf** - Benchmarking completes successfully
|
||||
|
||||
---
|
||||
|
||||
## 📦 Repository Health (HIGH PRIORITY)
|
||||
|
||||
### Git Repository
|
||||
- [ ] **.gitignore complete** - No datasets/checkpoints/cache in repo
|
||||
- [ ] **No large files** - Repository under 50 MB
|
||||
- [ ] **Clean history** - No sensitive data in commits
|
||||
- [ ] **Branch strategy** - main/dev branches clear
|
||||
- [ ] **Tags for release** - v0.9.0 tag created
|
||||
|
||||
### Repository Structure
|
||||
- [ ] **Directory organization clear**:
|
||||
- `modules/` - 20 module directories
|
||||
- `milestones/` - 6 milestone directories
|
||||
- `datasets/` - TinyDigits, TinyTalks (shipped)
|
||||
- `site/` - Documentation website
|
||||
- `tinytorch/` - Package code (generated from modules)
|
||||
- `tests/` - Test suite
|
||||
- [ ] **README files present** - Key directories have README.md
|
||||
- [ ] **No orphaned files** - Old experiments cleaned up
|
||||
|
||||
### Code Quality
|
||||
- [ ] **Python 3.9+ compatibility** - Works on modern Python
|
||||
- [ ] **Type hints** - Critical functions annotated
|
||||
- [ ] **Docstrings present** - Public APIs documented
|
||||
- [ ] **Code formatting** - Consistent style (black/ruff)
|
||||
- [ ] **No obvious bugs** - Core functionality works
|
||||
|
||||
---
|
||||
|
||||
## 🌐 Website/Documentation Site (HIGH PRIORITY)
|
||||
|
||||
### Website Build
|
||||
- [ ] **Site builds successfully** - `jupyter-book build site/` works
|
||||
- [ ] **All pages render** - No broken markdown/formatting
|
||||
- [ ] **Navigation clear** - Easy to find information
|
||||
- [ ] **Mobile-friendly** - Responsive design works
|
||||
|
||||
### Critical Pages
|
||||
- [x] **intro.md** - Landing page with clear value proposition
|
||||
- [x] **quickstart-guide.md** - 15-minute getting started
|
||||
- [x] **student-workflow.md** - Core development cycle
|
||||
- [x] **tito-essentials.md** - Command reference
|
||||
- [x] **learning-progress.md** - Module progression guide
|
||||
- [x] **faq.md** - Answers common questions
|
||||
- [x] **datasets.md** - Dataset documentation
|
||||
- [ ] **chapters/** - All chapter content complete
|
||||
|
||||
### Internal Links
|
||||
- [ ] **All internal links work** - No broken cross-references
|
||||
- [ ] **Code references formatted** - Syntax highlighting works
|
||||
- [ ] **Images display** - If any diagrams/screenshots present
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing (MEDIUM PRIORITY)
|
||||
|
||||
### Automated Tests
|
||||
- [ ] **Test suite exists** - tests/ directory has comprehensive coverage
|
||||
- [ ] **Tests pass** - `pytest tests/` succeeds
|
||||
- [ ] **Coverage reasonable** - Core functionality tested
|
||||
- [ ] **CI/CD configured** - GitHub Actions run tests (optional for v0.9)
|
||||
|
||||
### Manual Testing
|
||||
- [ ] **Fresh install tested** - New user can complete Module 01
|
||||
- [ ] **Module 01-07 validated** - Foundation tier works end-to-end
|
||||
- [ ] **Module 08-13 validated** - Architecture tier works
|
||||
- [ ] **Module 14-20 validated** - Optimization tier works
|
||||
- [ ] **Cross-platform tested** - Works on Mac/Linux at minimum
|
||||
|
||||
### Edge Cases
|
||||
- [ ] **Missing dependencies handled** - Clear error messages
|
||||
- [ ] **Network failures graceful** - MNIST/CIFAR download errors handled
|
||||
- [ ] **Disk space issues** - Helpful messages if space low
|
||||
- [ ] **Permission errors** - Guide users to fix permissions
|
||||
|
||||
---
|
||||
|
||||
## 📢 Community Preparation (MEDIUM PRIORITY)
|
||||
|
||||
### GitHub Repository
|
||||
- [ ] **Description clear** - "Educational ML framework built from scratch"
|
||||
- [ ] **Topics tagged** - machine-learning, education, pytorch-alternative, etc.
|
||||
- [ ] **GitHub Pages enabled** - Documentation site live
|
||||
- [ ] **Issues template** - Bug report and feature request templates
|
||||
- [ ] **PR template** - Contribution guidelines template
|
||||
- [ ] **Code of Conduct** - Community standards documented
|
||||
|
||||
### Communication
|
||||
- [ ] **Release announcement drafted** - What, why, how to get started
|
||||
- [ ] **Social media prepared** - Twitter/LinkedIn posts ready
|
||||
- [ ] **README badges** - Build status, license, etc.
|
||||
- [ ] **Changelog started** - CHANGELOG.md for v0.9.0
|
||||
|
||||
### Community Resources
|
||||
- [ ] **GitHub Discussions enabled** - Q&A and community space
|
||||
- [ ] **Discord/Slack** (optional) - Real-time community chat
|
||||
- [ ] **Leaderboard** (optional) - Module 20 competition results
|
||||
- [ ] **Contributor guide** - How to contribute code/docs
|
||||
|
||||
---
|
||||
|
||||
## 🎓 Educational Quality (MEDIUM PRIORITY)
|
||||
|
||||
### Pedagogical Soundness
|
||||
- [ ] **Learning objectives clear** - Each module states what you'll learn
|
||||
- [ ] **Prerequisites documented** - Students know what's required
|
||||
- [ ] **Scaffolding effective** - Modules build on previous work
|
||||
- [ ] **Systems focus maintained** - Profiling/performance emphasized
|
||||
|
||||
### Student Experience
|
||||
- [ ] **First module polished** - Module 01 is excellent intro
|
||||
- [ ] **Error messages helpful** - Students not blocked by cryptic errors
|
||||
- [ ] **Success feedback** - Celebrate completions appropriately
|
||||
- [ ] **Realistic expectations** - Time estimates accurate
|
||||
|
||||
### Reference Materials
|
||||
- [ ] **Production comparisons** - How TinyTorch relates to PyTorch/TF
|
||||
- [ ] **Historical context** - Why each milestone matters
|
||||
- [ ] **Career connections** - Job relevance clear
|
||||
- [ ] **Further reading** - Links to deepen understanding
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Launch Readiness (LOW PRIORITY - Nice to Have)
|
||||
|
||||
### Optional Enhancements
|
||||
- [ ] **Video walkthrough** - 5-minute intro video
|
||||
- [ ] **Blog post** - Detailed launch article
|
||||
- [ ] **Academic paper** - Pedagogy research paper (future)
|
||||
- [ ] **Conference submission** - SIGCSE/ICER presentation (future)
|
||||
|
||||
### Future Features (Mark as "Coming Soon")
|
||||
- [x] **NBGrader integration** - Marked as coming soon in docs
|
||||
- [x] **Classroom tooling** - Instructor guide states under development
|
||||
- [ ] **Advanced modules** - 21-25 as extension (future)
|
||||
- [ ] **GPU support** - CUDA implementation (future)
|
||||
|
||||
---
|
||||
|
||||
## Final Pre-Launch Checklist
|
||||
|
||||
**Run through this sequence 1 week before launch:**
|
||||
|
||||
### Day -7: Documentation Review
|
||||
- [ ] Read entire documentation site as a new user
|
||||
- [ ] Fix all typos, broken links, unclear sections
|
||||
- [ ] Verify all code examples run correctly
|
||||
|
||||
### Day -5: Technical Validation
|
||||
- [ ] Fresh install on 3 different machines
|
||||
- [ ] Complete Module 01 on each platform
|
||||
- [ ] Run all 6 milestones successfully
|
||||
- [ ] Verify all TITO commands work
|
||||
|
||||
### Day -3: Community Prep
|
||||
- [ ] Finalize GitHub repository settings
|
||||
- [ ] Prepare announcement posts
|
||||
- [ ] Set up community channels (Discussions/Discord)
|
||||
- [ ] Test contributor workflow
|
||||
|
||||
### Day -1: Final Polish
|
||||
- [ ] Create v0.9.0 release tag
|
||||
- [ ] Deploy documentation site
|
||||
- [ ] Queue social media announcements
|
||||
- [ ] Prepare for launch day support
|
||||
|
||||
### Launch Day
|
||||
- [ ] Publish release on GitHub
|
||||
- [ ] Post announcements (social media, forums)
|
||||
- [ ] Monitor issues/discussions
|
||||
- [ ] Celebrate! 🎉
|
||||
|
||||
---
|
||||
|
||||
## Version Recommendation
|
||||
|
||||
**Proposed**: **v0.9.0** for December 2024 release
|
||||
|
||||
**Rationale:**
|
||||
- v1.0 implies "production complete" - saves that for classroom integration
|
||||
- v0.9 signals "feature-complete for individual learners, refinements ongoing"
|
||||
- Allows v0.9.x patches for bugs discovered post-launch
|
||||
- v1.0 can mark full classroom integration milestone (Spring 2025?)
|
||||
|
||||
**Version Roadmap:**
|
||||
- **v0.9.0** (Dec 2024) - Community launch for individual learners
|
||||
- **v0.9.x** (Dec-Feb) - Bug fixes and documentation improvements
|
||||
- **v1.0.0** (Spring 2025?) - NBGrader integration + full classroom support
|
||||
- **v1.x.x** - Advanced modules, GPU support, additional features
|
||||
|
||||
---
|
||||
|
||||
## Success Metrics (Post-Launch)
|
||||
|
||||
Track these after release:
|
||||
|
||||
**Technical:**
|
||||
- Setup success rate (% users completing Module 01)
|
||||
- Platform coverage (macOS/Linux/Windows compatibility)
|
||||
- Bug report frequency
|
||||
- Milestone completion rates
|
||||
|
||||
**Community:**
|
||||
- GitHub stars/forks
|
||||
- Documentation page views
|
||||
- Community discussions activity
|
||||
- Contribution rate
|
||||
|
||||
**Educational:**
|
||||
- Module completion rates
|
||||
- Time-to-complete estimates validated
|
||||
- Learning objective achievement
|
||||
- Student feedback quality
|
||||
|
||||
---
|
||||
|
||||
## Notes
|
||||
|
||||
**Current Status (as of checklist creation):**
|
||||
- ✅ Documentation structure complete and consistent
|
||||
- ✅ Module count corrected to 20
|
||||
- ✅ FAQ and datasets documented
|
||||
- ⏳ Need comprehensive testing across platforms
|
||||
- ⏳ Need community infrastructure setup
|
||||
- ⏳ Need final polish pass
|
||||
|
||||
**Estimated time to launch-ready:** 2-3 weeks of focused work
|
||||
|
||||
**Critical path items:**
|
||||
1. Technical validation (test on multiple platforms)
|
||||
2. Module/milestone execution verification
|
||||
3. Documentation final polish
|
||||
4. Community infrastructure setup
|
||||
5. Release announcement preparation
|
||||
|
||||
**Non-blocking items (can be post-launch):**
|
||||
- Video tutorials
|
||||
- Advanced test coverage
|
||||
- Performance optimizations
|
||||
- Additional example notebooks
|
||||
|
||||
|
||||
|
||||
|
||||
8
MANIFEST.in
Normal file
8
MANIFEST.in
Normal file
@@ -0,0 +1,8 @@
|
||||
include README.md
|
||||
include LICENSE
|
||||
include pyproject.toml
|
||||
recursive-include tinytorch *.py
|
||||
recursive-exclude * __pycache__
|
||||
recursive-exclude * *.py[co]
|
||||
recursive-exclude * .DS_Store
|
||||
|
||||
@@ -1,546 +0,0 @@
|
||||
# Module 20 (Capstone) Consistency Review
|
||||
|
||||
**Review Date:** 2025-01-12
|
||||
**Reviewer:** Claude (TinyTorch Module Development Agent)
|
||||
**Files Reviewed:**
|
||||
- `/Users/VJ/GitHub/TinyTorch/modules/20_capstone/capstone.py` (2225 lines)
|
||||
- `/Users/VJ/GitHub/TinyTorch/modules/20_capstone/capstone_dev.py` (1016 lines)
|
||||
- `/Users/VJ/GitHub/TinyTorch/modules/20_capstone/ABOUT.md` (441 lines)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
**CRITICAL INCONSISTENCY FOUND:** Module 20 has **TWO DIFFERENT IMPLEMENTATIONS** with conflicting purposes:
|
||||
|
||||
1. **capstone.py**: Builds TinyGPT (complete transformer language model)
|
||||
2. **capstone_dev.py**: Competition submission workflow (TinyTorch Olympics)
|
||||
|
||||
**Impact:** HIGH - Students receive conflicting educational objectives and module descriptions don't match implementation.
|
||||
|
||||
**Recommendation:** URGENT - Decide which capstone to keep and remove/archive the other.
|
||||
|
||||
---
|
||||
|
||||
## 🚨 CRITICAL ISSUES
|
||||
|
||||
### Issue 1: Dual Competing Implementations (CRITICAL)
|
||||
|
||||
**Location:** Both `capstone.py` and `capstone_dev.py` exist with different purposes
|
||||
|
||||
**Problem:**
|
||||
```python
|
||||
# capstone.py (Line 17-20)
|
||||
"""
|
||||
# Module 20: Capstone - Building TinyGPT End-to-End
|
||||
Welcome to the capstone project of TinyTorch! [...] build **TinyGPT** -
|
||||
a complete transformer-based language model.
|
||||
"""
|
||||
|
||||
# capstone_dev.py (Line 17-19)
|
||||
"""
|
||||
# Module 20: TinyTorch Olympics - Competition & Submission
|
||||
Welcome to the capstone module of TinyTorch! [...] it's time to compete in
|
||||
**TinyTorch Olympics**
|
||||
"""
|
||||
```
|
||||
|
||||
**Why This Is Critical:**
|
||||
- **Confuses students**: Two different learning objectives
|
||||
- **Breaks module system**: `tito module complete 20` unclear which to export
|
||||
- **Documentation mismatch**: ABOUT.md describes Olympics but capstone.py builds TinyGPT
|
||||
- **Testing conflicts**: Different test patterns and validation
|
||||
|
||||
**TinyTorch Pattern Violation:**
|
||||
- Each module should have ONE clear purpose
|
||||
- Module number should map to ONE implementation
|
||||
- _dev.py should be development version of final .py, not a different module
|
||||
|
||||
---
|
||||
|
||||
### Issue 2: ABOUT.md Describes Olympics, but capstone.py Builds TinyGPT
|
||||
|
||||
**Location:** `ABOUT.md` vs `capstone.py` title mismatch
|
||||
|
||||
**ABOUT.md says:**
|
||||
```yaml
|
||||
title: "Torch Olympics - ML Systems Competition"
|
||||
description: "Combine all optimization techniques and compete on standardized benchmarks"
|
||||
```
|
||||
|
||||
**capstone.py says:**
|
||||
```python
|
||||
# Module 20: Capstone - Building TinyGPT End-to-End
|
||||
```
|
||||
|
||||
**Impact:**
|
||||
- Students reading ABOUT.md expect competition workflow
|
||||
- Opening capstone.py shows transformer implementation
|
||||
- Learning objectives don't align with implementation
|
||||
|
||||
**Pattern Violation:**
|
||||
- ABOUT.md MUST match the actual module implementation
|
||||
- Module title should be consistent across all files
|
||||
|
||||
---
|
||||
|
||||
### Issue 3: Duplicate test_module() and MODULE SUMMARY (CRITICAL)
|
||||
|
||||
**Location:** `capstone_dev.py` lines 656-720 and 840-910
|
||||
|
||||
**Problem:** Two identical `test_module()` functions and two MODULE SUMMARY sections
|
||||
|
||||
```python
|
||||
# First test_module() at line 656
|
||||
def test_module():
|
||||
"""Comprehensive test of entire competition module functionality."""
|
||||
# ... implementation
|
||||
|
||||
# Second test_module() at line 840 (DUPLICATE)
|
||||
def test_module():
|
||||
"""Comprehensive test of entire competition module functionality."""
|
||||
# ... identical implementation
|
||||
```
|
||||
|
||||
**Why This Breaks:**
|
||||
- Python will only recognize the second definition
|
||||
- First test_module() becomes dead code
|
||||
- NBGrader will fail with duplicate grade_ids
|
||||
- MODULE SUMMARY appears twice (lines 783 and 967)
|
||||
|
||||
**Pattern Violation:**
|
||||
- Modules MUST have exactly ONE test_module()
|
||||
- Modules MUST have exactly ONE MODULE SUMMARY
|
||||
- Duplicates break automated grading and testing
|
||||
|
||||
---
|
||||
|
||||
### Issue 4: Inconsistent Jupytext Version Numbers
|
||||
|
||||
**Location:** File headers
|
||||
|
||||
**capstone.py header:**
|
||||
```python
|
||||
# jupytext_version: 1.17.1 # ✅ CORRECT
|
||||
```
|
||||
|
||||
**capstone_dev.py header:**
|
||||
```python
|
||||
# jupytext_version: 1.18.1 # ❌ INCONSISTENT
|
||||
```
|
||||
|
||||
**Impact:**
|
||||
- Notebook conversion may fail
|
||||
- Version mismatch can cause formatting issues
|
||||
- Team should standardize on one version
|
||||
|
||||
**Pattern Violation:**
|
||||
- All modules should use same jupytext version
|
||||
- Module 19 uses 1.17.1 (correct standard)
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ STRUCTURAL ISSUES
|
||||
|
||||
### Issue 5: Missing Immediate Unit Test Execution
|
||||
|
||||
**Location:** `capstone_dev.py` line 540
|
||||
|
||||
**Problem:**
|
||||
```python
|
||||
def test_unit_submission_generation():
|
||||
"""🔬 Test submission generation."""
|
||||
# ... test implementation
|
||||
|
||||
test_unit_submission_generation() # ✅ Called immediately
|
||||
|
||||
# BUT this pattern is missing __main__ guard like other modules
|
||||
```
|
||||
|
||||
**Correct Pattern (from Module 19 and template):**
|
||||
```python
|
||||
def test_unit_submission_generation():
|
||||
"""🔬 Test submission generation."""
|
||||
# ... test implementation
|
||||
|
||||
# Run test immediately when developing this module
|
||||
if __name__ == "__main__":
|
||||
test_unit_submission_generation()
|
||||
```
|
||||
|
||||
**Impact:**
|
||||
- Test runs on every import (breaks dependency chain)
|
||||
- Violates CRITICAL FIRST RULE about protecting test code
|
||||
- Module 19 and others follow correct pattern
|
||||
|
||||
**Pattern Violation:**
|
||||
- All unit tests MUST be protected by `if __name__ == "__main__":`
|
||||
- Running tests at module level breaks imports
|
||||
|
||||
---
|
||||
|
||||
### Issue 6: capstone.py Has Correct Structure, capstone_dev.py Simplified
|
||||
|
||||
**Location:** Comparing both files
|
||||
|
||||
**Observation:**
|
||||
- **capstone.py (2225 lines)**: Full implementation with proper staging, comprehensive systems analysis, detailed ASCII diagrams
|
||||
- **capstone_dev.py (1016 lines)**: Simplified competition workflow, minimal implementation
|
||||
|
||||
**Analysis:**
|
||||
- capstone.py follows full TinyTorch template (Advanced module pattern)
|
||||
- capstone_dev.py appears to be a "simpler" version focusing only on competition
|
||||
- BUT _dev.py should be DEVELOPMENT version, not simplified version
|
||||
|
||||
**Pattern Violation:**
|
||||
- _dev.py files should be where students work (scaffolded with TODOs)
|
||||
- Final .py should be exported clean version
|
||||
- NOT two completely different modules
|
||||
|
||||
---
|
||||
|
||||
## ✅ POSITIVE FINDINGS
|
||||
|
||||
### What capstone.py Does Well:
|
||||
|
||||
1. **Comprehensive NBGrader Integration**
|
||||
- Proper cell metadata
|
||||
- BEGIN/END SOLUTION blocks correctly placed
|
||||
- Scaffolding outside solution blocks
|
||||
|
||||
2. **Proper Testing Pattern**
|
||||
- Unit tests immediately after implementations
|
||||
- `test_module()` integration test before summary
|
||||
- Protected test execution with `if __name__ == "__main__"`
|
||||
|
||||
3. **Systems Analysis Included**
|
||||
- Memory footprint tables
|
||||
- Performance complexity analysis (O(n²) attention)
|
||||
- Training vs inference memory breakdown
|
||||
|
||||
4. **Documentation Structure**
|
||||
- Clear ASCII diagrams showing architecture
|
||||
- Prerequisites section well-defined
|
||||
- Module Summary follows template
|
||||
|
||||
5. **ML Systems Questions**
|
||||
- Questions use only current module knowledge
|
||||
- Proper scoping (no forward references)
|
||||
- Reflection on systems concepts
|
||||
|
||||
### What capstone_dev.py Does Well:
|
||||
|
||||
1. **Clean Competition Workflow**
|
||||
- Clear 5-step process (Choose → Measure → Optimize → Validate → Submit)
|
||||
- Event-based structure (Latency Sprint, Memory Challenge, etc.)
|
||||
- Integration with Module 19's Benchmark class
|
||||
|
||||
2. **Proper Module Dependencies**
|
||||
- Uses Benchmark from Module 19
|
||||
- Uses optimization techniques from Modules 14-18
|
||||
- Clear dependency chain
|
||||
|
||||
---
|
||||
|
||||
## 📋 RECOMMENDATIONS
|
||||
|
||||
### URGENT: Resolve Dual Implementation Conflict
|
||||
|
||||
**Option A: Keep TinyGPT Capstone (capstone.py)**
|
||||
- **Rationale:** More comprehensive, demonstrates full systems integration
|
||||
- **Actions:**
|
||||
1. Rename `capstone_dev.py` → `competition_dev.py` (separate module)
|
||||
2. Update ABOUT.md to describe TinyGPT implementation
|
||||
3. Create separate Module 21 (or separate milestone) for competition
|
||||
4. Move Olympics content to milestone instead of module
|
||||
|
||||
**Option B: Keep Olympics Competition (capstone_dev.py)**
|
||||
- **Rationale:** Aligns with ABOUT.md, practical competition focus
|
||||
- **Actions:**
|
||||
1. Remove/archive `capstone.py` → `ABOUT_old.md` pattern
|
||||
2. Keep current ABOUT.md (already matches Olympics)
|
||||
3. Expand `capstone_dev.py` to include more implementation detail
|
||||
4. Move TinyGPT to Milestone 05 (already mentioned in journey)
|
||||
|
||||
**RECOMMENDED: Option B** because:
|
||||
- ABOUT.md already describes Olympics (less to change)
|
||||
- Aligns with "TorchPerf Olympics" mentioned in Module 19
|
||||
- Competition workflow is clearer learning objective
|
||||
- TinyGPT better fits as milestone (integration project)
|
||||
- Less disruptive to existing documentation
|
||||
|
||||
---
|
||||
|
||||
### REQUIRED FIXES (If Keeping capstone_dev.py)
|
||||
|
||||
#### Fix 1: Remove Duplicate test_module() and MODULE SUMMARY
|
||||
|
||||
**File:** `capstone_dev.py`
|
||||
|
||||
**Action:** Delete lines 840-1015 (second occurrence)
|
||||
|
||||
**Before:**
|
||||
```python
|
||||
# Line 656: First test_module()
|
||||
def test_module():
|
||||
# ... implementation
|
||||
|
||||
# Line 719: First call
|
||||
test_module()
|
||||
|
||||
# Line 783: First MODULE SUMMARY
|
||||
## 🎯 MODULE SUMMARY: ...
|
||||
|
||||
# Line 840: DUPLICATE test_module() ❌ DELETE THIS
|
||||
def test_module():
|
||||
# ... duplicate implementation
|
||||
|
||||
# Line 967: DUPLICATE MODULE SUMMARY ❌ DELETE THIS
|
||||
## 🎯 MODULE SUMMARY: ...
|
||||
```
|
||||
|
||||
**After:**
|
||||
```python
|
||||
# Keep only lines 656-783 (first occurrence)
|
||||
def test_module():
|
||||
# ... implementation
|
||||
|
||||
test_module()
|
||||
|
||||
## 🎯 MODULE SUMMARY: ...
|
||||
|
||||
# Delete everything after line 783 that duplicates earlier content
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Fix 2: Add __main__ Guards to Unit Tests
|
||||
|
||||
**File:** `capstone_dev.py` line 540
|
||||
|
||||
**Before:**
|
||||
```python
|
||||
def test_unit_submission_generation():
|
||||
"""🔬 Test submission generation."""
|
||||
# ... implementation
|
||||
|
||||
test_unit_submission_generation() # ❌ Runs on import
|
||||
```
|
||||
|
||||
**After:**
|
||||
```python
|
||||
def test_unit_submission_generation():
|
||||
"""🔬 Test submission generation."""
|
||||
# ... implementation
|
||||
|
||||
# Run test immediately when developing this module
|
||||
if __name__ == "__main__":
|
||||
test_unit_submission_generation() # ✅ Protected
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Fix 3: Fix Jupytext Version
|
||||
|
||||
**File:** `capstone_dev.py` line 8
|
||||
|
||||
**Before:**
|
||||
```python
|
||||
# jupytext_version: 1.18.1
|
||||
```
|
||||
|
||||
**After:**
|
||||
```python
|
||||
# jupytext_version: 1.17.1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Fix 4: Update Module Number References (If This Becomes Module 21)
|
||||
|
||||
**If competition becomes separate module:**
|
||||
|
||||
**Search for:** "Module 20"
|
||||
**Replace with:** "Module 21"
|
||||
|
||||
**Update journey maps:**
|
||||
```python
|
||||
# Before:
|
||||
Module 19: Benchmarking → Module 20: Competition
|
||||
|
||||
# After:
|
||||
Module 19: Benchmarking → Module 20: Capstone → Competition
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### REQUIRED FIXES (If Keeping capstone.py)
|
||||
|
||||
#### Fix 1: Update ABOUT.md to Match TinyGPT
|
||||
|
||||
**File:** `ABOUT.md`
|
||||
|
||||
**Before:**
|
||||
```yaml
|
||||
title: "Torch Olympics - ML Systems Competition"
|
||||
description: "Combine all optimization techniques and compete on standardized benchmarks"
|
||||
```
|
||||
|
||||
**After:**
|
||||
```yaml
|
||||
title: "TinyGPT Capstone - Building Intelligence from Scratch"
|
||||
description: "Integrate all 19 TinyTorch modules to build a complete transformer-based language model"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### Fix 2: Archive or Move capstone_dev.py
|
||||
|
||||
**Action:** Rename to preserve history
|
||||
|
||||
```bash
|
||||
cd modules/20_capstone
|
||||
mv capstone_dev.py competition_workflow_reference.py
|
||||
# Or move to separate module/milestone
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 CONSISTENCY SCORECARD
|
||||
|
||||
### Code Structure: 7/10
|
||||
- ✅ Proper NBGrader integration in capstone.py
|
||||
- ✅ Test patterns follow template
|
||||
- ⚠️ Dual implementations confuse structure
|
||||
- ❌ Missing __main__ guards in capstone_dev.py
|
||||
- ❌ Duplicate functions in capstone_dev.py
|
||||
|
||||
### Testing Patterns: 6/10
|
||||
- ✅ Unit tests present
|
||||
- ✅ test_module() integration test
|
||||
- ❌ Duplicate test_module() in capstone_dev.py
|
||||
- ⚠️ Tests not protected by __main__ in capstone_dev.py
|
||||
|
||||
### Documentation Structure: 5/10
|
||||
- ✅ Learning objectives clear
|
||||
- ✅ Prerequisites well-defined
|
||||
- ❌ ABOUT.md doesn't match capstone.py
|
||||
- ❌ Duplicate MODULE SUMMARY sections
|
||||
- ⚠️ Two different educational narratives
|
||||
|
||||
### NBGrader Integration: 8/10
|
||||
- ✅ Proper cell metadata
|
||||
- ✅ BEGIN/END SOLUTION blocks
|
||||
- ✅ Scaffolding outside solutions
|
||||
- ⚠️ Duplicate grade_ids risk (if both files used)
|
||||
|
||||
### Memory/Performance Analysis: 9/10 (capstone.py only)
|
||||
- ✅ Comprehensive memory tables
|
||||
- ✅ Complexity analysis (O(n²) attention)
|
||||
- ✅ Training vs inference comparison
|
||||
- ✅ Optimization impact quantified
|
||||
|
||||
### Production Context: 8/10
|
||||
- ✅ Real-world ML systems workflow
|
||||
- ✅ Integration with all previous modules
|
||||
- ✅ Proper dependency chain
|
||||
- ⚠️ Confusion about end goal (TinyGPT vs Competition)
|
||||
|
||||
### Module Consistency: 4/10
|
||||
- ❌ Two different implementations
|
||||
- ❌ ABOUT.md mismatch with capstone.py
|
||||
- ❌ Unclear which file is canonical
|
||||
- ❌ Inconsistent jupytext versions
|
||||
|
||||
---
|
||||
|
||||
## 🎯 PRIORITY ACTION ITEMS
|
||||
|
||||
### P0 (Critical - Do First):
|
||||
1. **DECIDE:** TinyGPT capstone OR Olympics competition
|
||||
2. **ARCHIVE:** Non-chosen implementation (preserve for reference)
|
||||
3. **UPDATE:** ABOUT.md to match chosen implementation
|
||||
4. **REMOVE:** Duplicate test_module() and MODULE SUMMARY in capstone_dev.py
|
||||
|
||||
### P1 (High - Do Soon):
|
||||
5. **FIX:** Add __main__ guards to all unit tests in capstone_dev.py
|
||||
6. **STANDARDIZE:** Jupytext version to 1.17.1
|
||||
7. **VERIFY:** NBGrader grade_ids are unique (no duplicates)
|
||||
|
||||
### P2 (Medium - Do Before Release):
|
||||
8. **TEST:** Run `tito test --module 20` to verify no breakage
|
||||
9. **DOCUMENT:** Update site/chapters to reflect chosen approach
|
||||
10. **VALIDATE:** Student workflow makes sense end-to-end
|
||||
|
||||
---
|
||||
|
||||
## 🔍 DETAILED LINE-BY-LINE ISSUES
|
||||
|
||||
### capstone_dev.py Issues:
|
||||
|
||||
| Line | Issue | Severity | Fix |
|
||||
|------|-------|----------|-----|
|
||||
| 8 | `jupytext_version: 1.18.1` | Medium | Change to 1.17.1 |
|
||||
| 540 | Test called without __main__ guard | High | Add if __name__ check |
|
||||
| 656-720 | First test_module() | N/A | Keep this one |
|
||||
| 783-832 | First MODULE SUMMARY | N/A | Keep this one |
|
||||
| 840-910 | Duplicate test_module() | Critical | DELETE |
|
||||
| 967-1015 | Duplicate MODULE SUMMARY | Critical | DELETE |
|
||||
|
||||
### capstone.py Issues:
|
||||
|
||||
| Line | Issue | Severity | Fix |
|
||||
|------|-------|----------|-----|
|
||||
| 17-20 | Title conflicts with ABOUT.md | Critical | Update ABOUT.md OR archive this file |
|
||||
| 2176 | MODULE SUMMARY present | N/A | Correct if keeping this file |
|
||||
|
||||
### ABOUT.md Issues:
|
||||
|
||||
| Line | Issue | Severity | Fix |
|
||||
|------|-------|----------|-----|
|
||||
| 1-2 | Title describes Olympics not TinyGPT | Critical | Match chosen implementation |
|
||||
| 8-13 | Learning objectives for Olympics | Medium | Update if keeping TinyGPT |
|
||||
|
||||
---
|
||||
|
||||
## 📝 CONCLUSION
|
||||
|
||||
Module 20 has **two high-quality but incompatible implementations**. Both follow TinyTorch patterns well individually, but their coexistence creates critical confusion.
|
||||
|
||||
**Immediate Action Required:**
|
||||
1. Team decision: TinyGPT OR Olympics
|
||||
2. Archive non-chosen implementation
|
||||
3. Fix duplicate test_module() and MODULE SUMMARY
|
||||
4. Update ABOUT.md to match
|
||||
|
||||
**Timeline Recommendation:**
|
||||
- **Week 1:** Make decision and archive
|
||||
- **Week 2:** Fix duplicates and test guards
|
||||
- **Week 3:** Validate with student testing
|
||||
|
||||
**Risk Assessment:**
|
||||
- **High Risk:** Students currently receive conflicting instructions
|
||||
- **Medium Risk:** NBGrader may fail with duplicate grade_ids
|
||||
- **Low Risk:** Once fixed, either implementation is solid
|
||||
|
||||
---
|
||||
|
||||
## 📚 REFERENCE: Correct Module Pattern
|
||||
|
||||
For reference, here's what a consistent Module 20 should look like:
|
||||
|
||||
```
|
||||
modules/20_capstone/
|
||||
├── ABOUT.md # Describes actual implementation
|
||||
├── capstone.py # Clean exported version (OR competition.py)
|
||||
├── capstone_dev.py # Development version with TODOs (OR competition_dev.py)
|
||||
├── test_capstone.py # Pytest tests
|
||||
└── reference_solution.py # Hidden instructor solution
|
||||
```
|
||||
|
||||
**Key Principle:** _dev.py is scaffolded version of .py, NOT a different module.
|
||||
|
||||
---
|
||||
|
||||
**Review Complete**
|
||||
**Recommendation:** Address P0 items immediately before next release.
|
||||
@@ -4,35 +4,50 @@ build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name="tinytorch"
|
||||
version = "0.0.1"
|
||||
description = "🚧 TinyTorch: Educational Deep Learning Framework (Coming Soon)"
|
||||
version = "0.1.0"
|
||||
description = "Build ML Systems from Scratch - Educational Deep Learning Framework"
|
||||
readme = "README.md"
|
||||
requires-python=">=3.8"
|
||||
authors = [
|
||||
{name = "Vijay Janapa Reddi", email = "vj@eecs.harvard.edu"}
|
||||
]
|
||||
license = "MIT"
|
||||
license = {text = "MIT"}
|
||||
classifiers = [
|
||||
"Development Status :: 2 - Pre-Alpha",
|
||||
"Development Status :: 3 - Alpha",
|
||||
"Intended Audience :: Education",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.8",
|
||||
"Programming Language :: Python :: 3.9",
|
||||
"Programming Language :: Python :: 3.10",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Programming Language :: Python :: 3.12",
|
||||
"Programming Language :: Python :: 3.13",
|
||||
"Topic :: Scientific/Engineering :: Artificial Intelligence",
|
||||
"Topic :: Education",
|
||||
]
|
||||
dependencies = []
|
||||
dependencies = [
|
||||
"numpy>=1.24.0,<3.0.0",
|
||||
"rich>=13.0.0",
|
||||
"PyYAML>=6.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=8.0.0",
|
||||
"pytest-cov>=4.0.0",
|
||||
"jupytext>=1.16.0",
|
||||
"nbformat>=5.10.0",
|
||||
"jupyter>=1.1.0",
|
||||
"jupyterlab>=4.2.0",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://github.com/VJ/TinyTorch"
|
||||
Repository = "https://github.com/VJ/TinyTorch"
|
||||
Issues = "https://github.com/VJ/TinyTorch/issues"
|
||||
Homepage = "https://github.com/mlsysbook/TinyTorch"
|
||||
Repository = "https://github.com/mlsysbook/TinyTorch"
|
||||
Documentation = "https://mlsysbook.github.io/TinyTorch/"
|
||||
Issues = "https://github.com/mlsysbook/TinyTorch/issues"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["."]
|
||||
include = ["tinytorch_placeholder*"]
|
||||
|
||||
[tool.setuptools.package-dir]
|
||||
tinytorch = "tinytorch_placeholder"
|
||||
include = ["tinytorch*"]
|
||||
exclude = ["tests*", "modules*", "site*", "docs*", "milestones*", "assignments*", "tito*"]
|
||||
|
||||
Reference in New Issue
Block a user