- Part I: Foundations (Modules 1-5) - Build MLPs, solve XOR
- Part II: Computer Vision (Modules 6-11) - Build CNNs, classify CIFAR-10
- Part III: Language Models (Modules 12-17) - Build transformers, generate text
Key changes:
- Renamed 05_dense to 05_networks for clarity
- Moved 08_dataloader to 07_dataloader (swap with attention)
- Moved 07_attention to 13_attention (Part III)
- Renamed 12_compression to 16_regularization
- Created placeholder dirs for new language modules (12,14,15,17)
- Moved old modules 13-16 to temp_holding for content migration
- Updated README with three-part structure
- Added comprehensive documentation in docs/three-part-structure.md
This structure gives students three natural exit points with concrete achievements at each level.
Major changes:
- Moved TinyGPT from Module 16 to examples/tinygpt (capstone demo)
- Fixed Module 10 (optimizers) and Module 11 (training) bugs
- All 16 modules now passing tests (100% health)
- Added comprehensive testing with 'tito test --comprehensive'
- Renamed example files for clarity (train_xor_network.py, etc.)
- Created working TinyGPT example structure
- Updated documentation to reflect 15 core modules + examples
- Added KISS principle and testing framework documentation
- Add MIT License with academic use notice and citation info
- Create comprehensive CONTRIBUTING.md with educational focus
- Emphasize systems thinking and pedagogical value
- Include mandatory git workflow standards from CLAUDE.md
- Restore proper file references in README.md
Repository now has complete contribution guidelines and licensing!
- Fix testing section with accurate demo/checkpoint counts (9 demos, 16 checkpoints)
- Update documentation links to point to existing files
- Remove references to missing CONTRIBUTING.md and LICENSE files
- Add reference to comprehensive test suite structure
- Point to actual documentation files in docs/ directory
- Ensure all claims match current reality
README now accurately reflects the actual TinyTorch structure!
- Update EXAMPLES mapping in tito to use new exciting names
- Add prominent examples section to main README
- Show clear progression: Module 05 → xornet, Module 11 → cifar10
- Update accuracy claims to realistic 57% (not aspirational 75%)
- Emphasize that examples are unlocked after module completion
- Connect examples to the learning journey
Students now understand when they can run exciting examples!
- Streamlined from 970 to 175 lines for clarity
- Focused on key information developers need
- Clear quick start instructions
- Concise module overview table
- Removed redundant FAQ section
- Simplified examples to essentials
- Better visual hierarchy with sections
- Professional badge presentation
- Maintained all critical information
The README is now more scannable and GitHub-friendly while
preserving the educational value and project overview.
- Corrected module dependencies based on actual YAML files
- Fixed diagram to show accurate prerequisite relationships:
- Tensor directly enables both Activations and Autograd
- DataLoader depends directly on Tensor (not through Spatial)
- Training depends on Dense, Spatial, Attention, Optimizers, and DataLoader
- TinyGPT depends on Attention, Optimizers, and Training
- Added sphinxcontrib-mermaid to requirements for diagram rendering
- Updated both intro.md and README.md with corrected diagrams
- Ensured mermaid extension is configured in _config.yml
- Add Harvard University badge and attribution
- Document professional academic design improvements
- Update quick start with virtual environment setup
- Add Jupyter Book website information
- Include instructor grading workflow with NBGrader
- Add prerequisites and learning resources section
- Update contributing and support information
- Add citation format for academic use
- Reflect 95% component reuse for TinyGPT
- Clean title format (TinyTorch with fire emoji)
Major Educational Framework Enhancements:
• Deploy interactive NBGrader text response questions across ALL modules
• Replace passive question lists with active 150-300 word student responses
• Enable comprehensive ML Systems learning assessment and grading
TinyGPT Integration (Module 16):
• Complete TinyGPT implementation showing 70% component reuse from TinyTorch
• Demonstrates vision-to-language framework generalization principles
• Full transformer architecture with attention, tokenization, and generation
• Shakespeare demo showing autoregressive text generation capabilities
Module Structure Standardization:
• Fix section ordering across all modules: Tests → Questions → Summary
• Ensure Module Summary is always the final section for consistency
• Standardize comprehensive testing patterns before educational content
Interactive Question Implementation:
• 3 focused questions per module replacing 10-15 passive questions
• NBGrader integration with manual grading workflow for text responses
• Questions target ML Systems thinking: scaling, deployment, optimization
• Cumulative knowledge building across the 16-module progression
Technical Infrastructure:
• TPM agent for coordinated multi-agent development workflows
• Enhanced documentation with pedagogical design principles
• Updated book structure to include TinyGPT as capstone demonstration
• Comprehensive QA validation of all module structures
Framework Design Insights:
• Mathematical unity: Dense layers power both vision and language models
• Attention as key innovation for sequential relationship modeling
• Production-ready patterns: training loops, optimization, evaluation
• System-level thinking: memory, performance, scaling considerations
Educational Impact:
• Transform passive learning to active engagement through written responses
• Enable instructors to assess deep ML Systems understanding
• Provide clear progression from foundations to complete language models
• Demonstrate real-world framework design principles and trade-offs
* Update README.md to lead with ML Systems value proposition
- Lead with "Build ML Systems From First Principles"
- Emphasize systems understanding through implementation
- Add learning path progression to TinyGPT
- Make MLSys book connection secondary/optional
- Focus on memory analysis, compute patterns, bottlenecks
* Update CLAUDE.md agent instructions for ML Systems focus
- Module Developer: Must include ML Systems analysis in every module
- Documentation Publisher: Must add systems insights sections
- QA Agent: Must test performance characteristics, not just correctness
- Add principle: "Every module teaches systems thinking through implementation"
- Require memory profiling, complexity analysis, scaling behavior
- Mandate production context and hardware implications
* Key positioning changes:
- TinyTorch = ML SYSTEMS course, not just ML algorithms
- Understanding comes through building complete systems
- Every implementation teaches memory, performance, scaling
- Bridge academic rigor with production engineering reality
This repositions TinyTorch as the definitive hands-on ML Systems engineering course.
- Add comprehensive README section showcasing 75% accuracy goal
- Update dataloader module README with CIFAR-10 support details
- Update training module README with checkpointing features
- Create complete CIFAR-10 training guide for students
- Document all north star implementations in CLAUDE.md
Students can now train real CNNs on CIFAR-10 using 100% TinyTorch code.
- Update Quick Start to show clear 3-step progression: Setup → Module 0 → Module 1
- Restructure module listing to highlight "START HERE!" for Module 0
- Add explicit "Module Progression" showing 0 → 1-16 flow
- Expand Module 0 description with bullet points about what users will explore
- Make it crystal clear that everyone should begin with Module 0 (Introduction)
The introduction module provides crucial system understanding before diving into implementation,
ensuring users understand the architecture and dependencies before building.
- Add virtual environment requirements and standards to CLAUDE.md
- Update README.md with new 00_introduction module overview
- Include visual system architecture and dependency analysis features
- Document proper development environment setup requirements
- Add troubleshooting guidance for environment issues
📖 Enhanced Visual Design:
- Wrapped entire FAQ content in blockquotes (>) for consistent grey background
- All bullet points, headers, and content now have improved readability
- Code blocks within blockquotes maintain proper formatting
- Consistent visual styling across all 8 FAQ entries
✨ User Experience Benefits:
- Grey background makes content much easier to read when expanded
- Better visual separation from surrounding text
- Professional appearance with improved contrast
- Reduces eye strain and improves content scanning
🎯 Technical Implementation:
- Added > prefix to all content lines within FAQ answers
- Maintained proper markdown formatting for headers, lists, and code
- Preserved existing structure while enhancing visual presentation
Result: FAQ dropdowns now have beautiful, consistent grey styling
that makes expanded content significantly easier to read and scan.
📱 New Access Method:
- Added Binder badge linking to mybinder.org launch
- Users can now run TinyTorch directly in browser without local setup
- Links to main branch: mybinder.org/v2/gh/MLSysBook/TinyTorch/main
🎯 User Experience Benefits:
- Zero-installation access for quick exploration
- Perfect for workshops, demos, and trying before installing
- Complements existing Jupyter Book documentation
- Positioned logically between Python and Jupyter Book badges
Result: Users now have multiple ways to engage with TinyTorch -
local installation, online documentation, and live interactive environment.
✨ Tone Improvements:
- Removed dismissive 'build toys' language about other tutorials
- Reframed as 'isolated components vs integrated systems' approach
- Much more respectful to other educators and learning resources
🏗️ Better Systems Engineering Analogy:
- Added compiler/OS analogy to explain systems thinking
- Helps readers understand why building integrated systems matters
- Concrete example: 'like understanding how every part of a compiler interacts'
📊 Enhanced Comparison:
- Updated comparison table to be more constructive
- Focus on 'Component vs Systems Approach' rather than dismissive contrasts
- Emphasizes integration and how everything connects
🎯 Educational Value:
- Explains WHY systems engineering matters without putting down alternatives
- Shows TinyTorch's unique value through positive comparison
- Maintains respectful tone while highlighting differentiating approach
Result: FAQ now educates about systems thinking benefits without
disrespecting other valuable learning resources. Much more professional
and constructive messaging.
🎨 Visual Design Improvements:
- Added proper spacing with <br> tags after each summary
- Used blockquotes (>) for key opening statements
- Added emoji section headers for better visual organization
- Added horizontal rules (---) to separate content sections
📖 Content Organization:
- Restructured answers with clear section headers
- Improved bullet point formatting and emphasis
- Added context headers like '🧪 Challenge Test', '🎯 Key Outcome'
- Made key phrases bold for easier scanning
🔍 Readability Enhancements:
- Eliminated wall-of-text appearance when expanded
- Created clear visual hierarchy within each answer
- Consistent formatting pattern across all FAQ entries
- Better information architecture for quick scanning
Result: FAQ dropdowns now transform from dense text blocks into
well-organized, scannable content that's actually pleasant to read
when expanded. Much better user experience
🔧 Module Structure Updates:
- Updated from 15 to 16 modules throughout documentation
- Fixed module names: 05_networks → 05_dense, 06_cnn → 06_spatial
- Added 07_attention module to documentation and flowchart
- Corrected module numbering in all sections (Deep Learning now 06-10, Production 11-15)
📊 Course Organization:
- Updated repository structure diagram with correct module names
- Fixed mermaid flowchart to show actual module dependencies
- Updated capstone references (15 core modules → 15 core modules + capstone = 16 total)
- Corrected learning path recommendations (core modules 01-10 for foundations)
📦 Package References:
- Added exports for dense.py, spatial.py, attention.py in tinytorch/core/
- Updated all module counts and difficulty progressions
- Fixed references to complete framework capabilities
Result: README now accurately reflects the actual 16-module structure with
correct naming, dependencies, and learning progression. No more confusion
between documentation and actual repository state.
📦 Dependency Management Fix:
- Added 'pip install -r requirements.txt' before 'pip install -e .'
- Explains that requirements.txt has all dependencies (numpy, jupyter, pytest, etc.)
- Clarifies that 'pip install -e .' installs TinyTorch package in editable mode
🐛 Problem Solved:
- Previously: 'pip install -e .' only installed numpy (from pyproject.toml)
- Students were missing matplotlib, PyYAML, pytest, rich, jupyter, nbdev, etc.
- Now: Proper two-step installation ensures all dependencies are available
Result: Students get working installation with all required dependencies
- Remove career projections and salary mentions (too sales-y)
- Add dropdown format for compact presentation
- Logical order: basic skepticism → advanced concerns → practical details
- Focus on learning benefits and technical substance
- More concise and scannable format
- Address Transformer dominance vs foundations learning
- Explain why not just use PyTorch/TensorFlow
- Differentiate from basic tutorials - emphasize systems thinking
- Show concrete ROI and career impact
- Bridge academic vs practical concerns
- Provide realistic time investment and career paths
- Address common objections with evidence-based responses
- Replace dry text description with engaging Mermaid flowchart
- Show clear progression through 4 educational layers: Foundation → Deep Learning → Production → Mastery
- Use color coding and visual flow arrows to demonstrate module dependencies
- Make it immediately clear how each module builds into the next
- Added detailed file hierarchy showing modules/source/, tinytorch/, book/, tito/ organization
- Included workflow explanation from development to testing to deployment
- Added difficulty progression visualization (⭐ to ⭐⭐⭐⭐⭐🥷)
- Enhanced module descriptions with clear learning objectives
- Improved onboarding experience for new contributors and students
- Changed from ambitious app development (computer vision, NLP, etc.) to realistic framework engineering
- New focus areas: performance optimization, algorithm extensions, systems engineering, benchmarking analysis, developer tools
- Projects now align with what students actually built: a complete ML framework
- Emphasizes systems engineering and optimization skills rather than application development
- Maintains 'no PyTorch imports' constraint to prove deep framework understanding
- Added 'Complete System Integration' section emphasizing how all 14 modules connect
- Highlighted that students build ONE cohesive ML framework, not isolated exercises
- Added capstone project section encouraging real applications using only TinyTorch
- Updated README.md 'What You'll Build' to emphasize system integration
- Added visual flow diagram showing module dependencies and connections
- Emphasized 'no PyTorch imports' constraint to prove framework completeness
Changed main tagline from:
'Most ML education teaches you to use frameworks. TinyTorch teaches you to understand them.'
To:
'Most ML education teaches you to use frameworks. TinyTorch teaches you to build them.'
Rationale:
- 'Understand' is vague and passive
- 'Build' is concrete and action-oriented
- Aligns perfectly with engineering focus we just established
- Reinforces the hands-on, construction-based learning approach
- More compelling for engineering-minded learners
Updated in both README.md and book/intro.md for consistency.
- Move 'The Big Picture: Why Build from Scratch?' to the top
- Add prominent 'What Makes TinyTorch Different' section highlighting unique value
- Emphasize build-first philosophy vs traditional 'use' frameworks approach
- Show concrete code comparison: traditional vs TinyTorch approach
- Better highlight real production skills, progressive mastery, instant feedback
- Reorganize content flow: vision → differentiators → practical details
- Updated all module references to start from 01 instead of 00
- Changed tagline to 'Build your own ML framework. Start small. Go deep.'
- Added educational foundation section linking to ML Systems book
- Updated README, documentation, CLI examples, and prerequisites
- Regenerated book content with consistent numbering throughout
- Maintains 14 modules total but with natural numbering (01-14)
- Fix repository URL and directory structure
- Add prominent Jupyter Book documentation link
- List all 14 complete modules with proper organization
- Update installation and workflow instructions
- Add dev/main branch git workflow documentation
- Include modern badges and three user onboarding paths
- Emphasize production ML and inline testing approach
- Reflect current tech stack and learning outcomes
- Remove 5 outdated development guides that contradicted clean NBGrader/nbdev architecture
- Update all documentation to reflect assignments/ directory structure
- Remove references to deprecated #| hide approach and old command patterns
- Ensure clean separation: NBGrader for assignments, nbdev for package export
- Update README, Student Guide, and Instructor Guide with current workflows
✅ PYTHON-FIRST DEVELOPMENT:
- Always work in raw Python files (modules/XX/XX_dev.py)
- Generate Jupyter notebooks on demand using Jupytext
- NBGrader compliance through automated cell metadata
- nbdev for package building and exports
🔧 WORKFLOW IMPROVEMENTS:
- Fixed file priority: use XX_dev.py over XX_dev_enhanced.py
- Clean up enhanced files to use standard files as source of truth
- Updated documentation to highlight Python-first approach
📚 COMPLETE INSTRUCTOR WORKFLOW:
1. Edit modules/XX/XX_dev.py (Python source of truth)
2. Export to package: tito module export XX (nbdev)
3. Generate assignment: tito nbgrader generate XX (Python→Jupyter→NBGrader)
4. Release to students: tito nbgrader release XX
5. Auto-grade with pytest: tito nbgrader autograde XX
✅ VERIFIED WORKING:
- Python file editing ✅
- nbdev export to tinytorch package ✅
- Jupytext conversion to notebooks ✅
- NBGrader assignment generation ✅
- pytest integration for auto-grading ✅🎯 TOOLS INTEGRATION:
- Raw Python development (version control friendly)
- Jupytext (Python ↔ Jupyter conversion)
- nbdev (package building and exports)
- NBGrader (student assignments and auto-grading)
- pytest (testing within notebooks)
Perfect implementation of user's ideal workflow
- Move development artifacts to development/archived/ directory
- Remove NBGrader artifacts (assignments/, testing/, gradebook.db, logs)
- Update root README.md to match actual repository structure
- Provide clear navigation paths for instructors and students
- Remove outdated documentation references
- Clean root directory while preserving essential files
- Maintain all functionality while improving organization
Repository is now optimally structured for classroom use with clear entry points:
- Instructors: docs/INSTRUCTOR_GUIDE.md
- Students: docs/STUDENT_GUIDE.md
- Developers: docs/development/
✅ All functionality verified working after restructuring
Introduces a README with project overview, setup instructions,
and course structure.
Adds a VISION document outlining the project's goals, conventions,
and architecture.
Includes updates to the setup module's README to clarify module to
package mapping.
Updates references to the development notebook naming convention from `[module].ipynb` to `[module]_dev.ipynb` in documentation. This change ensures consistency across the project and aligns with the intended naming scheme for development notebooks.
