- Add last commit badge to show project is actively maintained
- Add commit activity badge to show consistent development
- Add GitHub stars badge for social proof
- Add contributors badge to highlight collaboration
- Updated main README to prominently feature historical milestones (1957-2024)
- Added new 'Journey Through ML History' section to book navigation
- Created comprehensive milestones-overview.md chapter explaining the progression
- Updated intro.md with milestone achievements section
- Enhanced quickstart-guide.md with milestone unlock information
- Reflects working milestones/ directory structure with 6 historical demonstrations
- Clear progression: Perceptron (1957) → XOR (1969) → MLP (1986) → CNN (1998) → Transformers (2017) → Systems (2024)
- Emphasizes proof-of-mastery approach with real achievements
Major fixes for complete training pipeline functionality:
Core Components Fixed:
- Parameter class: Now wraps Variables with requires_grad=True for proper gradient tracking
- Variable.sum(): Essential for scalar loss computation from multi-element tensors
- Gradient handling: Fixed memoryview issues in autograd and activations
- Tensor indexing: Added __getitem__ support for weight inspection
Training Results:
- XOR learning: 100% accuracy (4/4) - network successfully learns XOR function
- Linear regression: Weight=1.991 (target=2.0), Bias=0.980 (target=1.0)
- Integration tests: 21/22 passing (95.5% success rate)
- Module tests: All individual modules passing
- General functionality: 4/5 tests passing with core training working
Technical Details:
- Fixed gradient data access patterns throughout activations.py
- Added safe memoryview handling in Variable.backward()
- Implemented proper Parameter-Variable delegation
- Added Tensor subscripting for debugging access
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
MLOps Module Removal:
- Remove deleted Module 21 (MLOps) from all documentation
- Update TOC to end at Module 20 (Benchmarking)
- Fix references in intro.md and README.md
- Clean up learning timeline to reflect 20-module structure
TinyMLPerf Leaderboard Addition:
- Create comprehensive leaderboard placeholder page at /leaderboard
- Detail competition categories: MLP Sprint, CNN Marathon, Transformer Decathlon
- Outline benchmark specifications and fair competition guidelines
- Reference future tinytorch.org/leaderboard domain
- Add leaderboard to main navigation under Resources & Tools
- Update README to point to leaderboard page
The website now accurately represents our 20-module curriculum
without premature MLOps references and includes exciting
competition framework for student engagement.
Major improvements:
- Fix module ordering to match actual 20-module progression (01-20 + MLOps)
- Clarify DataLoader as generic batching tool (not just CIFAR-10)
- Add work-in-progress banner with compelling 'Why TinyTorch?' message
- Add TinyMLPerf competition and leaderboard section
- Remove premature industry feedback section
- Acknowledge other TinyTorch/MiniTorch projects
- Simplify additional resources section
- Update Mermaid diagram to show DataLoader correctly
- Ensure git URL points to mlsysbook/TinyTorch
The website now accurately reflects our 20-module structure with proper
categorization and professional presentation ready for Spring 2025 launch.
- Update README and website to be more professional while staying welcoming
- Remove excessive emojis from headers and tables
- Keep strategic emoji usage for emphasis (checkmarks, warnings)
- Clean up module tables and section headers
- Update Mermaid diagrams to be cleaner
- Fix module count (20 not 16) and accuracy claims (75%+ CIFAR-10)
- Strengthen ML Systems engineering messaging throughout
- Update milestone examples with correct historical references
- Maintain accessibility and professional tone
- Part I: Foundations (Modules 1-5) - Build MLPs, solve XOR
- Part II: Computer Vision (Modules 6-11) - Build CNNs, classify CIFAR-10
- Part III: Language Models (Modules 12-17) - Build transformers, generate text
Key changes:
- Renamed 05_dense to 05_networks for clarity
- Moved 08_dataloader to 07_dataloader (swap with attention)
- Moved 07_attention to 13_attention (Part III)
- Renamed 12_compression to 16_regularization
- Created placeholder dirs for new language modules (12,14,15,17)
- Moved old modules 13-16 to temp_holding for content migration
- Updated README with three-part structure
- Added comprehensive documentation in docs/three-part-structure.md
This structure gives students three natural exit points with concrete achievements at each level.
Major changes:
- Moved TinyGPT from Module 16 to examples/tinygpt (capstone demo)
- Fixed Module 10 (optimizers) and Module 11 (training) bugs
- All 16 modules now passing tests (100% health)
- Added comprehensive testing with 'tito test --comprehensive'
- Renamed example files for clarity (train_xor_network.py, etc.)
- Created working TinyGPT example structure
- Updated documentation to reflect 15 core modules + examples
- Added KISS principle and testing framework documentation
- Add MIT License with academic use notice and citation info
- Create comprehensive CONTRIBUTING.md with educational focus
- Emphasize systems thinking and pedagogical value
- Include mandatory git workflow standards from CLAUDE.md
- Restore proper file references in README.md
Repository now has complete contribution guidelines and licensing!
- Fix testing section with accurate demo/checkpoint counts (9 demos, 16 checkpoints)
- Update documentation links to point to existing files
- Remove references to missing CONTRIBUTING.md and LICENSE files
- Add reference to comprehensive test suite structure
- Point to actual documentation files in docs/ directory
- Ensure all claims match current reality
README now accurately reflects the actual TinyTorch structure!
- Update EXAMPLES mapping in tito to use new exciting names
- Add prominent examples section to main README
- Show clear progression: Module 05 → xornet, Module 11 → cifar10
- Update accuracy claims to realistic 57% (not aspirational 75%)
- Emphasize that examples are unlocked after module completion
- Connect examples to the learning journey
Students now understand when they can run exciting examples!
- Streamlined from 970 to 175 lines for clarity
- Focused on key information developers need
- Clear quick start instructions
- Concise module overview table
- Removed redundant FAQ section
- Simplified examples to essentials
- Better visual hierarchy with sections
- Professional badge presentation
- Maintained all critical information
The README is now more scannable and GitHub-friendly while
preserving the educational value and project overview.
- Corrected module dependencies based on actual YAML files
- Fixed diagram to show accurate prerequisite relationships:
- Tensor directly enables both Activations and Autograd
- DataLoader depends directly on Tensor (not through Spatial)
- Training depends on Dense, Spatial, Attention, Optimizers, and DataLoader
- TinyGPT depends on Attention, Optimizers, and Training
- Added sphinxcontrib-mermaid to requirements for diagram rendering
- Updated both intro.md and README.md with corrected diagrams
- Ensured mermaid extension is configured in _config.yml
- Add Harvard University badge and attribution
- Document professional academic design improvements
- Update quick start with virtual environment setup
- Add Jupyter Book website information
- Include instructor grading workflow with NBGrader
- Add prerequisites and learning resources section
- Update contributing and support information
- Add citation format for academic use
- Reflect 95% component reuse for TinyGPT
- Clean title format (TinyTorch with fire emoji)
Major Educational Framework Enhancements:
• Deploy interactive NBGrader text response questions across ALL modules
• Replace passive question lists with active 150-300 word student responses
• Enable comprehensive ML Systems learning assessment and grading
TinyGPT Integration (Module 16):
• Complete TinyGPT implementation showing 70% component reuse from TinyTorch
• Demonstrates vision-to-language framework generalization principles
• Full transformer architecture with attention, tokenization, and generation
• Shakespeare demo showing autoregressive text generation capabilities
Module Structure Standardization:
• Fix section ordering across all modules: Tests → Questions → Summary
• Ensure Module Summary is always the final section for consistency
• Standardize comprehensive testing patterns before educational content
Interactive Question Implementation:
• 3 focused questions per module replacing 10-15 passive questions
• NBGrader integration with manual grading workflow for text responses
• Questions target ML Systems thinking: scaling, deployment, optimization
• Cumulative knowledge building across the 16-module progression
Technical Infrastructure:
• TPM agent for coordinated multi-agent development workflows
• Enhanced documentation with pedagogical design principles
• Updated book structure to include TinyGPT as capstone demonstration
• Comprehensive QA validation of all module structures
Framework Design Insights:
• Mathematical unity: Dense layers power both vision and language models
• Attention as key innovation for sequential relationship modeling
• Production-ready patterns: training loops, optimization, evaluation
• System-level thinking: memory, performance, scaling considerations
Educational Impact:
• Transform passive learning to active engagement through written responses
• Enable instructors to assess deep ML Systems understanding
• Provide clear progression from foundations to complete language models
• Demonstrate real-world framework design principles and trade-offs
* Update README.md to lead with ML Systems value proposition
- Lead with "Build ML Systems From First Principles"
- Emphasize systems understanding through implementation
- Add learning path progression to TinyGPT
- Make MLSys book connection secondary/optional
- Focus on memory analysis, compute patterns, bottlenecks
* Update CLAUDE.md agent instructions for ML Systems focus
- Module Developer: Must include ML Systems analysis in every module
- Documentation Publisher: Must add systems insights sections
- QA Agent: Must test performance characteristics, not just correctness
- Add principle: "Every module teaches systems thinking through implementation"
- Require memory profiling, complexity analysis, scaling behavior
- Mandate production context and hardware implications
* Key positioning changes:
- TinyTorch = ML SYSTEMS course, not just ML algorithms
- Understanding comes through building complete systems
- Every implementation teaches memory, performance, scaling
- Bridge academic rigor with production engineering reality
This repositions TinyTorch as the definitive hands-on ML Systems engineering course.
- Add comprehensive README section showcasing 75% accuracy goal
- Update dataloader module README with CIFAR-10 support details
- Update training module README with checkpointing features
- Create complete CIFAR-10 training guide for students
- Document all north star implementations in CLAUDE.md
Students can now train real CNNs on CIFAR-10 using 100% TinyTorch code.
- Update Quick Start to show clear 3-step progression: Setup → Module 0 → Module 1
- Restructure module listing to highlight "START HERE!" for Module 0
- Add explicit "Module Progression" showing 0 → 1-16 flow
- Expand Module 0 description with bullet points about what users will explore
- Make it crystal clear that everyone should begin with Module 0 (Introduction)
The introduction module provides crucial system understanding before diving into implementation,
ensuring users understand the architecture and dependencies before building.
- Add virtual environment requirements and standards to CLAUDE.md
- Update README.md with new 00_introduction module overview
- Include visual system architecture and dependency analysis features
- Document proper development environment setup requirements
- Add troubleshooting guidance for environment issues
📖 Enhanced Visual Design:
- Wrapped entire FAQ content in blockquotes (>) for consistent grey background
- All bullet points, headers, and content now have improved readability
- Code blocks within blockquotes maintain proper formatting
- Consistent visual styling across all 8 FAQ entries
✨ User Experience Benefits:
- Grey background makes content much easier to read when expanded
- Better visual separation from surrounding text
- Professional appearance with improved contrast
- Reduces eye strain and improves content scanning
🎯 Technical Implementation:
- Added > prefix to all content lines within FAQ answers
- Maintained proper markdown formatting for headers, lists, and code
- Preserved existing structure while enhancing visual presentation
Result: FAQ dropdowns now have beautiful, consistent grey styling
that makes expanded content significantly easier to read and scan.
📱 New Access Method:
- Added Binder badge linking to mybinder.org launch
- Users can now run TinyTorch directly in browser without local setup
- Links to main branch: mybinder.org/v2/gh/MLSysBook/TinyTorch/main
🎯 User Experience Benefits:
- Zero-installation access for quick exploration
- Perfect for workshops, demos, and trying before installing
- Complements existing Jupyter Book documentation
- Positioned logically between Python and Jupyter Book badges
Result: Users now have multiple ways to engage with TinyTorch -
local installation, online documentation, and live interactive environment.
✨ Tone Improvements:
- Removed dismissive 'build toys' language about other tutorials
- Reframed as 'isolated components vs integrated systems' approach
- Much more respectful to other educators and learning resources
🏗️ Better Systems Engineering Analogy:
- Added compiler/OS analogy to explain systems thinking
- Helps readers understand why building integrated systems matters
- Concrete example: 'like understanding how every part of a compiler interacts'
📊 Enhanced Comparison:
- Updated comparison table to be more constructive
- Focus on 'Component vs Systems Approach' rather than dismissive contrasts
- Emphasizes integration and how everything connects
🎯 Educational Value:
- Explains WHY systems engineering matters without putting down alternatives
- Shows TinyTorch's unique value through positive comparison
- Maintains respectful tone while highlighting differentiating approach
Result: FAQ now educates about systems thinking benefits without
disrespecting other valuable learning resources. Much more professional
and constructive messaging.
🎨 Visual Design Improvements:
- Added proper spacing with <br> tags after each summary
- Used blockquotes (>) for key opening statements
- Added emoji section headers for better visual organization
- Added horizontal rules (---) to separate content sections
📖 Content Organization:
- Restructured answers with clear section headers
- Improved bullet point formatting and emphasis
- Added context headers like '🧪 Challenge Test', '🎯 Key Outcome'
- Made key phrases bold for easier scanning
🔍 Readability Enhancements:
- Eliminated wall-of-text appearance when expanded
- Created clear visual hierarchy within each answer
- Consistent formatting pattern across all FAQ entries
- Better information architecture for quick scanning
Result: FAQ dropdowns now transform from dense text blocks into
well-organized, scannable content that's actually pleasant to read
when expanded. Much better user experience
🔧 Module Structure Updates:
- Updated from 15 to 16 modules throughout documentation
- Fixed module names: 05_networks → 05_dense, 06_cnn → 06_spatial
- Added 07_attention module to documentation and flowchart
- Corrected module numbering in all sections (Deep Learning now 06-10, Production 11-15)
📊 Course Organization:
- Updated repository structure diagram with correct module names
- Fixed mermaid flowchart to show actual module dependencies
- Updated capstone references (15 core modules → 15 core modules + capstone = 16 total)
- Corrected learning path recommendations (core modules 01-10 for foundations)
📦 Package References:
- Added exports for dense.py, spatial.py, attention.py in tinytorch/core/
- Updated all module counts and difficulty progressions
- Fixed references to complete framework capabilities
Result: README now accurately reflects the actual 16-module structure with
correct naming, dependencies, and learning progression. No more confusion
between documentation and actual repository state.
📦 Dependency Management Fix:
- Added 'pip install -r requirements.txt' before 'pip install -e .'
- Explains that requirements.txt has all dependencies (numpy, jupyter, pytest, etc.)
- Clarifies that 'pip install -e .' installs TinyTorch package in editable mode
🐛 Problem Solved:
- Previously: 'pip install -e .' only installed numpy (from pyproject.toml)
- Students were missing matplotlib, PyYAML, pytest, rich, jupyter, nbdev, etc.
- Now: Proper two-step installation ensures all dependencies are available
Result: Students get working installation with all required dependencies
- Remove career projections and salary mentions (too sales-y)
- Add dropdown format for compact presentation
- Logical order: basic skepticism → advanced concerns → practical details
- Focus on learning benefits and technical substance
- More concise and scannable format
- Address Transformer dominance vs foundations learning
- Explain why not just use PyTorch/TensorFlow
- Differentiate from basic tutorials - emphasize systems thinking
- Show concrete ROI and career impact
- Bridge academic vs practical concerns
- Provide realistic time investment and career paths
- Address common objections with evidence-based responses
- Replace dry text description with engaging Mermaid flowchart
- Show clear progression through 4 educational layers: Foundation → Deep Learning → Production → Mastery
- Use color coding and visual flow arrows to demonstrate module dependencies
- Make it immediately clear how each module builds into the next
- Added detailed file hierarchy showing modules/source/, tinytorch/, book/, tito/ organization
- Included workflow explanation from development to testing to deployment
- Added difficulty progression visualization (⭐ to ⭐⭐⭐⭐⭐🥷)
- Enhanced module descriptions with clear learning objectives
- Improved onboarding experience for new contributors and students
- Changed from ambitious app development (computer vision, NLP, etc.) to realistic framework engineering
- New focus areas: performance optimization, algorithm extensions, systems engineering, benchmarking analysis, developer tools
- Projects now align with what students actually built: a complete ML framework
- Emphasizes systems engineering and optimization skills rather than application development
- Maintains 'no PyTorch imports' constraint to prove deep framework understanding
- Added 'Complete System Integration' section emphasizing how all 14 modules connect
- Highlighted that students build ONE cohesive ML framework, not isolated exercises
- Added capstone project section encouraging real applications using only TinyTorch
- Updated README.md 'What You'll Build' to emphasize system integration
- Added visual flow diagram showing module dependencies and connections
- Emphasized 'no PyTorch imports' constraint to prove framework completeness
Changed main tagline from:
'Most ML education teaches you to use frameworks. TinyTorch teaches you to understand them.'
To:
'Most ML education teaches you to use frameworks. TinyTorch teaches you to build them.'
Rationale:
- 'Understand' is vague and passive
- 'Build' is concrete and action-oriented
- Aligns perfectly with engineering focus we just established
- Reinforces the hands-on, construction-based learning approach
- More compelling for engineering-minded learners
Updated in both README.md and book/intro.md for consistency.
- Move 'The Big Picture: Why Build from Scratch?' to the top
- Add prominent 'What Makes TinyTorch Different' section highlighting unique value
- Emphasize build-first philosophy vs traditional 'use' frameworks approach
- Show concrete code comparison: traditional vs TinyTorch approach
- Better highlight real production skills, progressive mastery, instant feedback
- Reorganize content flow: vision → differentiators → practical details
- Updated all module references to start from 01 instead of 00
- Changed tagline to 'Build your own ML framework. Start small. Go deep.'
- Added educational foundation section linking to ML Systems book
- Updated README, documentation, CLI examples, and prerequisites
- Regenerated book content with consistent numbering throughout
- Maintains 14 modules total but with natural numbering (01-14)
