mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-04-28 22:27:30 -05:00
docs(toc): Update table of contents for reorganized structure
TOC changes: - Architecture Tier: (08-14) → (08-13) - Removed Module 14 (KV Caching) from Architecture Tier - Renamed Module 09: 'Spatial (CNNs)' → 'Convolutional Networks' - Optimization Tier: (15-19) → (14-19) - New order: Profiling, Memoization, Quantization, Compression, Acceleration, Benchmarking Added documentation: - OPTIMIZATION_TIER_RESTRUCTURE_PLAN.md: Comprehensive implementation plan - PROGRESS_SUMMARY.md: Summary of completed work
This commit is contained in:
135
PROGRESS_SUMMARY.md
Normal file
135
PROGRESS_SUMMARY.md
Normal file
@@ -0,0 +1,135 @@
|
||||
# Optimization Tier Restructure - Progress Summary
|
||||
|
||||
## ✅ Completed Work
|
||||
|
||||
### Phase 1: Profiler Enhancement ✓
|
||||
- ✅ Added `quick_profile()` helper function
|
||||
- ✅ Added `analyze_weight_distribution()` helper function
|
||||
- ✅ Both exported for use in optimization modules
|
||||
|
||||
### Phase 2: Profiling Intro Sections ✓
|
||||
- ✅ Module 14 (KV Caching → Memoization): Added O(n²) growth demonstration
|
||||
- ✅ Module 17 (Quantization → Module 16): Added memory usage profiling
|
||||
- ✅ Module 18 (Compression → Module 17): Added weight distribution analysis
|
||||
- ✅ Module 16 (Acceleration → Module 18): Added CNN bottleneck profiling
|
||||
|
||||
### Phase 3: Module Directory Reorganization ✓
|
||||
- ✅ Renamed: `14_kvcaching` → `15_memoization`
|
||||
- ✅ Renamed: `15_profiling` → `14_profiling`
|
||||
- ✅ Renamed: `16_acceleration` → `18_acceleration`
|
||||
- ✅ Renamed: `17_quantization` → `16_quantization`
|
||||
- ✅ Renamed: `18_compression` → `17_compression`
|
||||
- ✅ Kept: `19_benchmarking` (no change)
|
||||
- ✅ Renamed file: `kvcaching_dev.py` → `memoization_dev.py`
|
||||
|
||||
### Phase 4: Module Source File Updates ✓
|
||||
- ✅ Module 14 (Profiling): Updated header, connection map, prerequisites
|
||||
- ✅ Module 15 (Memoization): Updated to emphasize memoization concept, KV caching as application
|
||||
- ✅ Module 16 (Quantization): Updated module number, prerequisites
|
||||
- ✅ Module 17 (Compression): Updated module number, prerequisites
|
||||
- ✅ Module 18 (Acceleration): Updated module number, prerequisites
|
||||
- ✅ Module 19 (Benchmarking): Updated cross-references to Module 14
|
||||
|
||||
### Phase 5: Book Chapter File Reorganization ✓
|
||||
- ✅ Renamed: `14-kvcaching.md` → `15-memoization.md`
|
||||
- ✅ Renamed: `15-profiling.md` → `14-profiling.md`
|
||||
- ✅ Renamed: `16-acceleration.md` → `18-acceleration.md`
|
||||
- ✅ Renamed: `17-quantization.md` → `16-quantization.md`
|
||||
- ✅ Renamed: `18-compression.md` → `17-compression.md`
|
||||
- ✅ Kept: `19-benchmarking.md` (no change)
|
||||
|
||||
### Phase 6: Table of Contents Update ✓
|
||||
- ✅ Updated Architecture Tier caption: (08-14) → (08-13)
|
||||
- ✅ Removed Module 14 (KV Caching) from Architecture Tier
|
||||
- ✅ Renamed Module 09: "Spatial (CNNs)" → "Convolutional Networks"
|
||||
- ✅ Updated Optimization Tier caption: (15-19) → (14-19)
|
||||
- ✅ Added Module 14: Profiling
|
||||
- ✅ Added Module 15: Memoization
|
||||
- ✅ Reordered Modules 16-18 (Quantization, Compression, Acceleration)
|
||||
|
||||
---
|
||||
|
||||
## 🚧 In Progress / Remaining Work
|
||||
|
||||
### Phase 7: Book Chapter Content Updates (IN PROGRESS)
|
||||
Need to update in each chapter file:
|
||||
- [ ] Main heading (e.g., `# 15. Memoization`)
|
||||
- [ ] YAML frontmatter:
|
||||
- [ ] `title`
|
||||
- [ ] `prerequisites`
|
||||
- [ ] `next_steps`
|
||||
- [ ] `difficulty` (Memoization: 3→2)
|
||||
- [ ] Tier badge (if needed)
|
||||
- [ ] Cross-references to other modules
|
||||
- [ ] "What's Next?" sections
|
||||
|
||||
**Files to update:**
|
||||
- [ ] `14-profiling.md` (was 15)
|
||||
- [ ] `15-memoization.md` (was 14, "KV Caching")
|
||||
- [ ] `16-quantization.md` (was 17)
|
||||
- [ ] `17-compression.md` (was 18)
|
||||
- [ ] `18-acceleration.md` (was 16)
|
||||
- [ ] `19-benchmarking.md` (cross-references only)
|
||||
- [ ] `09-spatial.md` → rename to `09-convolutional-networks.md`
|
||||
|
||||
### Phase 8: Cross-Reference Cleanup (PENDING)
|
||||
- [ ] Search for "Module 14" references (should now be context-dependent)
|
||||
- [ ] Search for "Module 15" references
|
||||
- [ ] Search for "KV Caching" references (update to "Memoization" where appropriate)
|
||||
- [ ] Update "Next module" links
|
||||
|
||||
### Phase 9: Testing (PENDING)
|
||||
- [ ] Export test: `tito export 14` (profiling)
|
||||
- [ ] Export test: `tito export 15` (memoization)
|
||||
- [ ] Book build test: `jupyter-book build book/`
|
||||
- [ ] Check for warnings/errors
|
||||
|
||||
### Phase 10: Final Commits (PENDING)
|
||||
Will commit in logical chunks:
|
||||
1. Profiler enhancements
|
||||
2. Module profiling intro sections
|
||||
3. Module reorganization
|
||||
4. Book chapter updates
|
||||
5. TOC update
|
||||
6. Cross-reference fixes
|
||||
|
||||
---
|
||||
|
||||
## 📊 Statistics
|
||||
|
||||
**Total modules updated:** 6 (14-19)
|
||||
**Total chapter files renamed:** 6
|
||||
**Total dev files updated:** 6
|
||||
**Lines of code added:** ~400+ (profiling intros)
|
||||
**Files renamed:** 12 (6 directories + 6 markdown files)
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Key Design Decisions Made
|
||||
|
||||
1. **Memoization vs KV Caching**: Module renamed to emphasize general pattern, with KV caching as specific transformer application
|
||||
2. **Profiling First**: Establishes measurement-first workflow for all optimizations
|
||||
3. **Quick Profiling Sections**: Each optimization module (15-18) starts with profiling motivation
|
||||
4. **Module Order**: Memoization → Quantization → Compression → Acceleration (specific to general, easy to hard)
|
||||
5. **Difficulty Adjustment**: Memoization lowered from 3 to 2 (simpler caching pattern)
|
||||
|
||||
---
|
||||
|
||||
## 📝 Commits to Make
|
||||
|
||||
1. ✅ Profiler helper functions
|
||||
2. ✅ Memoization profiling intro
|
||||
3. ✅ Quantization profiling intro (pending commit)
|
||||
4. ✅ Compression profiling intro (pending commit)
|
||||
5. ✅ Acceleration profiling intro (pending commit)
|
||||
6. Module source reorganization (pending commit)
|
||||
7. Book chapter reorganization (pending commit)
|
||||
8. TOC update (pending commit)
|
||||
9. Cross-reference fixes (pending commit)
|
||||
10. Final testing + documentation (pending commit)
|
||||
|
||||
---
|
||||
|
||||
*Last updated: [timestamp]*
|
||||
*Branch: optimization-tier-restructure*
|
||||
|
||||
@@ -42,12 +42,12 @@ parts:
|
||||
- file: chapters/07-training
|
||||
title: "07. Training"
|
||||
|
||||
- caption: 🏛️ Architecture Tier (08-14)
|
||||
- caption: 🏛️ Architecture Tier (08-13)
|
||||
chapters:
|
||||
- file: chapters/08-dataloader
|
||||
title: "08. DataLoader"
|
||||
- file: chapters/09-spatial
|
||||
title: "09. Spatial (CNNs)"
|
||||
title: "09. Convolutional Networks"
|
||||
- file: chapters/10-tokenization
|
||||
title: "10. Tokenization"
|
||||
- file: chapters/11-embeddings
|
||||
@@ -56,19 +56,19 @@ parts:
|
||||
title: "12. Attention"
|
||||
- file: chapters/13-transformers
|
||||
title: "13. Transformers"
|
||||
- file: chapters/14-kvcaching
|
||||
title: "14. KV Caching"
|
||||
|
||||
- caption: ⚡ Optimization Tier (15-19)
|
||||
- caption: ⚡ Optimization Tier (14-19)
|
||||
chapters:
|
||||
- file: chapters/15-profiling
|
||||
title: "15. Profiling"
|
||||
- file: chapters/16-acceleration
|
||||
title: "16. Acceleration"
|
||||
- file: chapters/17-quantization
|
||||
title: "17. Quantization"
|
||||
- file: chapters/18-compression
|
||||
title: "18. Compression"
|
||||
- file: chapters/14-profiling
|
||||
title: "14. Profiling"
|
||||
- file: chapters/15-memoization
|
||||
title: "15. Memoization"
|
||||
- file: chapters/16-quantization
|
||||
title: "16. Quantization"
|
||||
- file: chapters/17-compression
|
||||
title: "17. Compression"
|
||||
- file: chapters/18-acceleration
|
||||
title: "18. Acceleration"
|
||||
- file: chapters/19-benchmarking
|
||||
title: "19. Benchmarking"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user