- Replace hardcoded module names array with dynamic reading from module.yaml files
- Add get_module_names() function to read actual module structure
- Fix IndexError in get_prev_module_name() and get_next_module_name() functions
- Update navigation logic to use actual module count instead of hardcoded assumptions
- Successfully converts all 16 modules to chapters with proper navigation
- Book build now completes without errors
- Remove generic learning communities section
- Remove vague 'next steps' career advice
- Remove fluffy usage instructions
- Keep focused: academic courses, books, alternative implementations, production internals
- Result: curated reference for students who built ML systems from scratch
✂️ Reduced MLOps Focus:
- Renamed 'MLOps & Production' → 'Development Tools'
- Removed redundant 'MLOps Community' link
- Focuses on practical development tools instead
🎯 Made Framework Differentiations Distinct:
- Micrograd: 'shows you the math, TinyTorch shows you the systems'
- Tinygrad: 'optimizes for speed, TinyTorch optimizes for learning'
- NNFS: 'focuses on algorithms, TinyTorch focuses on complete systems engineering'
💡 Benefits:
- Each differentiation now highlights specific strengths vs repetitive vehicle analogy
- Less MLOps emphasis (appears in course already)
- More concise and memorable comparisons
Result: Cleaner resource organization with unique, specific differentiations
that avoid repetition and over-emphasis on any single topic.
🔥 Major Improvements:
- Removed research papers section (belongs in specific labs as context)
- Added clear differentiation for alternative implementations with vehicle analogy
- Moved ML Systems book to books section with prominent positioning
- Added actual book links (O'Reilly, deeplearningbook.org) where available
- Focused on maintainable, stable resources
🎯 Key Differentiations Added:
- 'Micrograd teaches engine parts, TinyTorch teaches you to design the whole vehicle'
- 'NNFS teaches engine parts, TinyTorch teaches the whole vehicle and drive it'
- 'Tinygrad optimizes for speed, TinyTorch optimizes for learning systems thinking'
🏭 Production Focus:
- Added industrial tools: W&B, MLOps Community, Papers with Code
- Reorganized into: Courses, Books, Alternative Implementations, Production Tools
- Removed quickly-outdated content, kept stable educational resources
📖 ML Systems Book Positioning:
- Moved Vijay's book from courses to books section
- Positioned as 'the perfect companion to TinyTorch'
- Added proper book links for maintainability
Result: Much more focused, maintainable resource page that complements
TinyTorch without duplicating content that belongs in specific labs.
🎓 Course Additions:
- Added CS 249r: Tiny Machine Learning (Harvard) to course list
- Covers TinyML systems, edge AI, and resource-constrained machine learning
- Complements existing MIT TinyML course with Harvard perspective
📖 Section Naming Fix:
- Changed 'Essential Books' → 'Recommended Books'
- Avoids prescriptive language and duplication issues
- More inclusive and less hierarchical phrasing
🔄 Organization Benefits:
- Eliminates potential confusion with ML Systems book already in courses
- Creates clearer separation between course materials and supplementary books
- Better reflects that these are helpful additions, not requirements
Result: More thoughtful resource organization with key Harvard tinyML
course addition and improved section naming.
🔧 Title Configuration Fix:
- Changed book/_config.yml title from long form to simple 'Tiny🔥Torch'
- Eliminates duplicate title in browser tab (was showing 'Tiny🔥Torch — Tiny🔥Torch')
- Now Chrome tab displays clean 'Tiny🔥Torch' once
Result: Clean, professional browser tab title without duplication.
🔄 Chapter File Reorganization:
- Renamed 05-networks.md → 05-dense.md
- Renamed 06-cnn.md → 06-spatial.md
- Created 07-attention.md with transformer-focused content
- Renumbered all subsequent chapters (7→8, 8→9, 9→10, etc.)
- Updated final module: 15-capstone.md → 16-capstone.md
📚 Attention Chapter Content:
- Added comprehensive attention module introduction
- Covers self-attention, multi-head attention, transformer foundations
- Explains Query-Key-Value mechanism and scaled dot-product attention
- Connects to previous modules (tensors, activations, layers, dense)
- Positions attention as foundation for modern AI (GPT, BERT, ViTs)
✅ Build Verification:
- Jupyter Book builds successfully with no missing file errors
- All 16 chapters now properly indexed in table of contents
- New structure: Foundation (1-3), Building Blocks (4-7), Training (8-11),
Inference & Serving (12-16)
Result: Complete alignment between repository structure, book chapters,
and table of contents. Students can now navigate the full 16-module course
with proper attention coverage and updated section organization.
📖 New Resources Page:
- Created book/resources.md with curated external learning materials
- Academic courses: Stanford CS329S, Harvard ML Systems, MIT TinyML
- Essential books: Chip Huyen, Andriy Burkov, Deep Learning textbook
- Framework deep dives: PyTorch/TensorFlow internals and architecture
- Research papers: Autograd, Adam, Attention, TensorFlow/PyTorch papers
- Implementation guides: micrograd, tinygrad, Neural Networks from Scratch
- Communities: MLOps, r/MachineLearning, technical blogs
- Next steps: Post-TinyTorch learning paths and advanced specializations
🔄 Updated Table of Contents:
- Fixed module names: networks → dense, cnn → spatial
- Added 07_attention to Building Blocks section
- Updated all numbering to reflect 16-module structure
- Renamed 'Production & Performance' → 'Inference & Serving'
- Added new 'Additional Resources' section with 📚 Learning Resources
🎯 Educational Value:
- Provides context for TinyTorch implementations
- Bridges from educational framework to production systems
- Offers multiple learning paths for different interests
- Connects TinyTorch concepts to broader ML systems ecosystem
Result: Students now have comprehensive resources to deepen their
understanding and apply TinyTorch knowledge to real-world systems.
📖 Enhanced Visual Design:
- Wrapped entire FAQ content in blockquotes (>) for consistent grey background
- All bullet points, headers, and content now have improved readability
- Code blocks within blockquotes maintain proper formatting
- Consistent visual styling across all 8 FAQ entries
✨ User Experience Benefits:
- Grey background makes content much easier to read when expanded
- Better visual separation from surrounding text
- Professional appearance with improved contrast
- Reduces eye strain and improves content scanning
🎯 Technical Implementation:
- Added > prefix to all content lines within FAQ answers
- Maintained proper markdown formatting for headers, lists, and code
- Preserved existing structure while enhancing visual presentation
Result: FAQ dropdowns now have beautiful, consistent grey styling
that makes expanded content significantly easier to read and scan.
✨ Title Formatting:
- Split title into main header and subtitle for better readability
- Enhanced visual hierarchy in book introduction
🚀 Content Updates:
- Changed 'rocket ship' to 'AI rocket ship' for more specific branding
- Added '(Harvard)' to Prof. Vijay Janapa Reddi reference for clarity
- Maintains professional attribution while being more informative
Result: Cleaner book intro formatting with improved readability and attribution.
📱 New Access Method:
- Added Binder badge linking to mybinder.org launch
- Users can now run TinyTorch directly in browser without local setup
- Links to main branch: mybinder.org/v2/gh/MLSysBook/TinyTorch/main
🎯 User Experience Benefits:
- Zero-installation access for quick exploration
- Perfect for workshops, demos, and trying before installing
- Complements existing Jupyter Book documentation
- Positioned logically between Python and Jupyter Book badges
Result: Users now have multiple ways to engage with TinyTorch -
local installation, online documentation, and live interactive environment.
✨ Tone Improvements:
- Removed dismissive 'build toys' language about other tutorials
- Reframed as 'isolated components vs integrated systems' approach
- Much more respectful to other educators and learning resources
🏗️ Better Systems Engineering Analogy:
- Added compiler/OS analogy to explain systems thinking
- Helps readers understand why building integrated systems matters
- Concrete example: 'like understanding how every part of a compiler interacts'
📊 Enhanced Comparison:
- Updated comparison table to be more constructive
- Focus on 'Component vs Systems Approach' rather than dismissive contrasts
- Emphasizes integration and how everything connects
🎯 Educational Value:
- Explains WHY systems engineering matters without putting down alternatives
- Shows TinyTorch's unique value through positive comparison
- Maintains respectful tone while highlighting differentiating approach
Result: FAQ now educates about systems thinking benefits without
disrespecting other valuable learning resources. Much more professional
and constructive messaging.
🎨 Visual Design Improvements:
- Added proper spacing with <br> tags after each summary
- Used blockquotes (>) for key opening statements
- Added emoji section headers for better visual organization
- Added horizontal rules (---) to separate content sections
📖 Content Organization:
- Restructured answers with clear section headers
- Improved bullet point formatting and emphasis
- Added context headers like '🧪 Challenge Test', '🎯 Key Outcome'
- Made key phrases bold for easier scanning
🔍 Readability Enhancements:
- Eliminated wall-of-text appearance when expanded
- Created clear visual hierarchy within each answer
- Consistent formatting pattern across all FAQ entries
- Better information architecture for quick scanning
Result: FAQ dropdowns now transform from dense text blocks into
well-organized, scannable content that's actually pleasant to read
when expanded. Much better user experience
🔧 Module Structure Updates:
- Updated from 15 to 16 modules throughout documentation
- Fixed module names: 05_networks → 05_dense, 06_cnn → 06_spatial
- Added 07_attention module to documentation and flowchart
- Corrected module numbering in all sections (Deep Learning now 06-10, Production 11-15)
📊 Course Organization:
- Updated repository structure diagram with correct module names
- Fixed mermaid flowchart to show actual module dependencies
- Updated capstone references (15 core modules → 15 core modules + capstone = 16 total)
- Corrected learning path recommendations (core modules 01-10 for foundations)
📦 Package References:
- Added exports for dense.py, spatial.py, attention.py in tinytorch/core/
- Updated all module counts and difficulty progressions
- Fixed references to complete framework capabilities
Result: README now accurately reflects the actual 16-module structure with
correct naming, dependencies, and learning progression. No more confusion
between documentation and actual repository state.
📦 Dependency Management Fix:
- Added 'pip install -r requirements.txt' before 'pip install -e .'
- Explains that requirements.txt has all dependencies (numpy, jupyter, pytest, etc.)
- Clarifies that 'pip install -e .' installs TinyTorch package in editable mode
🐛 Problem Solved:
- Previously: 'pip install -e .' only installed numpy (from pyproject.toml)
- Students were missing matplotlib, PyYAML, pytest, rich, jupyter, nbdev, etc.
- Now: Proper two-step installation ensures all dependencies are available
Result: Students get working installation with all required dependencies
- Updated module.yaml files for 05_dense and 06_spatial to reference correct dev file names
- Fixed #| default_exp directives in dense_dev.py and spatial_dev.py to export to correct module names
- Fixed tensor assignment issues in 12_compression module by creating new Tensor objects instead of trying to assign to .data property
- Removed missing function imports from autograd integration test
- All individual module tests now pass (01_setup through 14_benchmarking)
- Generated correct module files: dense.py, spatial.py, attention.py
- Replace str | None with Optional[str] in export.py
- Replace list[str] with List[str] in config.py
- Add missing typing imports for compatibility
- Ensures CLI works on Python 3.8 systems
✅ Refactored test_tensor_activations_integration.py:
- Changed from re-testing activation math to testing Tensor-Activation interfaces
- Focus on: Tensor input → Activation → Tensor output compatibility
- Test dtype preservation, shape preservation, chaining, error handling
- Test activation outputs work with further Tensor operations
✅ Refactored test_layers_networks_integration.py:
- Changed from re-testing layer/network logic to testing Layer-Dense interfaces
- Focus on: Dense layer → Sequential network → MLP composition
- Test layer output as network input, network output as layer input
- Test multi-stage pipelines, parallel processing, modular replacement
Integration tests now properly focus on:
✅ Cross-module interface compatibility (not individual functionality)
✅ Data flow and pipeline integration between modules
✅ Shape/dtype preservation across module boundaries
✅ System-level workflows and architectural patterns
✅ Error handling when modules are incompatibly connected
✅ Component modularity and interchangeability
Establishes proper integration testing philosophy: test that modules work TOGETHER, not what individual modules do (that's for inline tests).
✅ Refactored test_tensor_attention_integration.py:
- Changed from re-testing attention functionality to testing interface compatibility
- Focus on: Tensor.data → Attention → numpy → Tensor roundtrip compatibility
- Test data type preservation across modules (float32, float64)
- Test shape preservation and error handling at interfaces
- Test that attention outputs can be converted back to Tensors
✅ Refactored test_attention_pipeline_integration.py:
- Changed from testing transformer algorithms to testing module pipelines
- Focus on: Attention → Dense → Activation integration workflows
- Test encoder-decoder patterns using multiple TinyTorch modules
- Test multi-layer workflows with residual connections
- Test data flow compatibility and modular component replacement
Integration tests now properly focus on:
✅ Interface compatibility (not functionality re-testing)
✅ Cross-module data flow and pipeline integration
✅ System-level workflows using multiple modules
✅ Shape/dtype preservation across module boundaries
✅ Error handling when modules are incompatibly connected
Follows integration testing best practices: test that modules work together, not what individual modules do.
✅ NBGrader solution/test structure: ### BEGIN/END SOLUTION blocks
✅ Educational TODO sections: STEP-BY-STEP, HINTS, EXAMPLES, LEARNING CONNECTIONS
✅ Immediate unit tests: proper assertions after each solution
✅ TinyTorch consistency: same patterns as tensor, layers, activations modules
✅ All tests passing: 100% success rate with comprehensive coverage
Module now follows established TinyTorch educational format:
- Detailed TODO instructions for student implementation
- Solution blocks wrapped in NBGrader tags
- Immediate feedback with unit tests after each piece
- Progress tracking with emojis and clear status messages
Ready for NBGrader processing and student use.
- Implement scaled dot-product attention with masking support
- Build multi-head attention with learnable projections
- Create sinusoidal positional encoding for sequence understanding
- Add layer normalization for training stability
- Complete transformer block with residual connections
- Include self-attention wrapper and utility functions
- Full inline testing with 100% pass rate
- Educational content explaining attention mechanisms
- Foundation for modern AI architectures (GPT, BERT, etc.)
This module bridges classical ML (tensors, layers, networks) with
modern transformer architectures that power ChatGPT and contemporary AI.
- Emphasize always working in dev branch
- Main branch for stable releases only
- Recommend feature branches for all changes
- Add YAML-style description for Cursor
- Clear workflow steps and quick reference
- Address math anxiety: explain math learning approach
- Address validation fears: highlight testing and feedback
- Address flexibility concerns: explain module dependencies
- Address toy project skepticism: emphasize real data and results
- Focus on actual questions students ask vs generic course info
- 4 key questions for students already interested in the course
- Focus on practical learning concerns vs skepticism
- Shorter than GitHub FAQ - appropriate for committed learners
- Covers time investment, skill level, support, modern relevance
- Remove career projections and salary mentions (too sales-y)
- Add dropdown format for compact presentation
- Logical order: basic skepticism → advanced concerns → practical details
- Focus on learning benefits and technical substance
- More concise and scannable format
- Address Transformer dominance vs foundations learning
- Explain why not just use PyTorch/TensorFlow
- Differentiate from basic tutorials - emphasize systems thinking
- Show concrete ROI and career impact
- Bridge academic vs practical concerns
- Provide realistic time investment and career paths
- Address common objections with evidence-based responses
- Replace dry text description with engaging Mermaid flowchart
- Show clear progression through 4 educational layers: Foundation → Deep Learning → Production → Mastery
- Use color coding and visual flow arrows to demonstrate module dependencies
- Make it immediately clear how each module builds into the next
- Regenerated all chapters with YAML-based difficulty ratings
- Updated book with improved navigation and fixed appendix links
- Applied copyright year 2025 across all pages
- Integrated inclusive language changes throughout generated content
- Book now reflects all UX and consistency improvements
- Added educational metadata (difficulty, time_estimate) to all module.yaml files
- Updated convert_readmes.py to read from YAML instead of hardcoded mappings
- Standardized difficulty progression: ⭐ → ⭐⭐ → ⭐⭐⭐ → ⭐⭐⭐⭐ → ⭐⭐⭐⭐⭐🥷
- Fixed path resolution for YAML reading in book build process
- Eliminated duplication: single source of truth for educational metadata
- Capstone gets special ninja treatment (⭐⭐⭐⭐⭐🥷) as beyond-expert level
