- CNN builds directly on layers/networks concepts while fresh
- Creates natural progression: layers → networks → cnn → dataloader
- 'Complete the layer toolkit first' before moving to data systems
- Move compression to 10 (right after training - model optimization)
- Move mlops to 11 (deployment comes next - production readiness)
- Move profiling to 12 (optimization tools - when needed)
- Move benchmarking to 13 (comparative analysis - with profiling)
- Move kernels to 14 (advanced optimization)
- Move transformer to 15 (final advanced topic)
Better Learning Flow:
- Training → 'I have a model, now what?' → Deployment
- Deployment → 'Now let me optimize' → Profiling/Benchmarking
- Mirrors real ML engineering: Deploy first, optimize second
- Profiling + Benchmarking work together naturally
Final progression (00-15):
00_setup → 01_tensor → 02_activations → 03_layers → 04_networks →
05_dataloader → 06_cnn → 07_autograd → 08_optimizers → 09_training →
10_compression → 11_mlops → 12_profiling → 13_benchmarking →
14_kernels → 15_transformer
- Add CNN as module 06 (after networks, before autograd)
- Shift all subsequent modules up by one number
- Add Transformer as module 15 (advanced/final module)
- Complete logical progression from foundations to production systems
Final progression:
00_setup → 01_tensor → 02_activations → 03_layers → 04_networks →
05_dataloader → 06_cnn → 07_autograd → 08_training → 09_config →
10_profiling → 11_compression → 12_kernels → 13_benchmarking →
14_mlops → 15_transformer
Clear student learning path: Foundation → Core ML → Advanced Architectures →
Training Systems → Production Systems → Advanced Topics
- Update all test files to import from tinytorch.core.* instead of relative paths
- Consistent with rock solid foundation principle
- Tests now use stable package imports, not local module imports
- Ensures tests validate the actual exported package functionality
- Aligns with production usage patterns
- Fix module imports to use tinytorch.core.* instead of local module imports
- Activations module now imports from tinytorch.core.tensor for stability
- Layers module imports from tinytorch.core.tensor and tinytorch.core.activations
- Test files updated to use main package imports for dependencies
- This ensures students can focus on current module without dependency issues
- Previous modules are 'locked in' and guaranteed to work
- Mirrors real-world usage patterns like PyTorch
- Maintains educational progression while ensuring system stability
- Reorganize hidden solutions to follow correct NBDev pattern (after each student implementation)
- Add comprehensive visualization system with _should_show_plots() control
- Implement smart test/development mode detection to suppress visualizations during pytest
- Add individual activation function plots and sample data visualizations
- Include comprehensive comparison plots for all activation functions
- Maintain educational inline tests with immediate feedback
- All 24 tests pass cleanly with proper visualization control
- Follows same structure as layers and tensor modules
Features:
- ReLU, Sigmoid, Tanh, Softmax visualizations with mathematical properties
- Real-time visual feedback during development
- Clean test execution without visual noise
- Educational context and use-case guidance for each activation
- Remove duplicate class definitions (was 800 lines → 517 lines)
- Follow consistent educational pattern like other modules
- Improve Build → Use → Reflect pedagogical framework
- Clean up TODO sections with proper implementation guidance
- Add comprehensive docstrings and examples
- Organize student and instructor implementations properly
- Maintain all functionality while improving readability
- All tests still pass (24/24 activations tests)
- Change --all flag meaning from 'clean both file types' to 'clean all modules'
- Make clean command consistent with test and export commands
- Require explicit module name or --all flag (no implicit behavior)
- Update help text and examples
- Now supports both:
- tito module clean tensor (specific module)
- tito module clean --all (all modules)
- Update test, export, and clean commands to use positional arguments
- Change from 'tito module test --module dataloader' to 'tito module test dataloader'
- Eliminates redundant --module flag within module command group
- Update help text and examples to reflect new syntax
- Maintains backward compatibility with --all flag
- More intuitive and consistent CLI design
- Move export functionality from 'tito package export' to 'tito module export'
- Require --all flag for exporting all modules (consistent with test command)
- Remove export from package command group to eliminate duplication
- Update help text and examples across all commands
- Fix tensor module arithmetic operators for complete functionality
- Clean up duplicate _quarto.yml and sidebar.yml files in modules/
This creates a consistent CLI pattern:
- tito module export --all (export all modules)
- tito module export --module <name> (export specific module)
- tito module test --all (test all modules)
- tito module test --module <name> (test specific module)
- Remove redundant fields from module.yaml files: exports_to, files, components
- Keep only essential system metadata: name, title, description, dependencies
- Export command now reads actual export targets from dev files (#| default_exp directive)
- Status command updated to use dev files as source of truth for export targets
- Export command shows detailed source → target mapping for better clarity
- Dependencies field retained as it's useful for CLI module ordering and prerequisites
- Eliminates duplication between YAML and dev files - dev files are the real truth
- Add Module Info sections with difficulty ratings to all README.md files
- Use consistent 4-star difficulty scale: ⭐ Beginner, ⭐⭐ Intermediate, ⭐⭐⭐ Advanced, ⭐⭐⭐⭐ Expert
- Include time estimates, prerequisites, and next steps for each module
- Maintain clear separation: README.md = student experience, module.yaml = system metadata
- Difficulty progression: Setup (⭐) → Tensor/Activations/Layers (⭐⭐) → Networks/CNN/DataLoader (⭐⭐⭐) → Transformer (⭐⭐⭐⭐)
- Help students plan their learning journey and set appropriate expectations
- Add new CleanCommand for cleaning up module directories
- Supports cleaning notebooks (*.ipynb) and cache files (__pycache__, *.pyc)
- Can clean specific modules or all modules
- Provides preview of files to be cleaned with confirmation
- Includes --force flag to skip confirmation
- Integrates with module command group as 'tito module clean'
- Preserves Python source files (*_dev.py) and other important files
- Fixes issue with duplicate file removal from __pycache__ directories
- Delete bin/py_to_notebook.py and tito/tools/py_to_notebook.py
- Update notebooks command to use Jupytext directly
- Jupytext is already configured in all *_dev.py files
- Simpler, more standard workflow using established tools
- Better integration with NBDev ecosystem
Benefits:
- Eliminates duplicate conversion tools
- Uses industry-standard Jupytext instead of custom tool
- Reduces maintenance burden
- Better error handling and compatibility
- Remove version field from all module.yaml files
- Update template generator to exclude version field
- Further simplify metadata to focus on system information only
- Status remains dynamically determined by test results
- Reduce module.yaml files from 100+ lines to ~25 lines focused on system needs
- Remove pedagogical details (learning objectives, difficulty, time estimates)
- Keep only essential fields: name, title, description, status, dependencies, exports, files, components
- Update status command to work with simplified metadata format
- Update metadata generation script to create simplified templates
- Focus on system metadata for CLI tools and build systems, not educational content
Before: Verbose pedagogical metadata with 20+ fields
After: Concise system metadata with 8 core fields
This aligns with the principle that module.yaml should be for systems, not pedagogy.
- Changed all module titles to be short and clean (e.g., 'Autograd' not 'Autograd - Automatic Differentiation')
- Updated metadata generation template to use concise titles by default
- Fixed CLI reference in metadata generator to use new hierarchical structure
- Titles are now consistent: just the module name capitalized
- Detailed descriptions remain in the description field where they belong
- Add module.yaml files for setup, tensor, activations, layers, and autograd modules
- Enhanced tito status command with --metadata flag for rich information display
- Created metadata schema with learning objectives, dependencies, components, and more
- Added metadata generation script (bin/generate_module_metadata.py)
- Comprehensive documentation in docs/development/module-metadata-system.md
- Status command now shows module status, difficulty, time estimates, and detailed metadata
- Supports dependency tracking, component-level status, and educational information
- Enables rich CLI experience with structured module information
- Rename modules/data/ → modules/dataloader/
- Rename data_dev.py → dataloader_dev.py
- Update NBDev export target: core.data → core.dataloader
- Rename test files: test_data.py → test_dataloader.py
- Update package exports to tinytorch.core.dataloader
- Update module imports and internal references
This makes the module name more descriptive and aligned with ML industry standards.
- Added pytest-timeout configuration with 5-minute timeout for all tests
- Added timeout handling to test command with proper error messages
- Created small local test dataset (50 train + 20 test samples) that mimics CIFAR-10 structure
- Updated data module tests to use local test data instead of downloading CIFAR-10
- Tests now run much faster (~0.1s vs ~30s) and don't require internet connection
- Added TestCIFAR10Dataset class that loads from local pickle files
- All test functionality preserved but using local data for speed and reliability
- Fixed plt.show() call in data module to respect _should_show_plots() check
- Protected test code that calls visualization functions in data module
- Protected test code that calls visualization functions in networks module
- All visualization functions now properly skip during testing
- Tests should no longer block waiting for user interaction
Introduces documentation for TinyTorch module development, including guides for developers and AI assistants.
Provides comprehensive resources for creating high-quality, educational modules, focusing on real-world applications and systems thinking.
Initializes the networks module, enabling the composition of layers into complete neural network architectures.
It introduces sequential networks, MLP creation, and network visualization tools to facilitate architecture understanding and analysis.
Adds practical classification and regression network implementations and network behavior analysis capabilities.
Adds ReLU, Sigmoid, and Tanh activation functions, enabling
non-linearity in neural networks.
Includes testing and visualization of each function to ensure
correct behavior and understanding of their properties.
Introduces a comprehensive module for 2D Convolutional Neural Networks.
This module provides a foundational understanding of CNNs through:
- Implementation of a naive Conv2D layer with sliding window convolution
- Visualization of kernel operations and feature map construction
- Composition of Conv2D layers with other layers to build a simple ConvNet
This structure provides a step-by-step guide to building and understanding CNNs, with clear examples and tests.
This commit introduces the core building blocks for neural networks,
including a naive matrix multiplication implementation and a Dense layer.
It provides a foundation for constructing and experimenting with
neural networks, emphasizing the concept of layers as tensor
transformations and function composition.
The module includes thorough testing and performance comparisons
to demonstrate the importance of optimized operations.
Implements the core Tensor class with data handling and properties, including initialization, shape, size, dtype, and string representation.
Adds element-wise addition and multiplication functions for tensors.
Implements tensor addition and multiplication as methods within the Tensor class.
Enhances the notebook by replacing some unicode characters with more standard and universally compatible symbols, improving the overall readability and user experience.
- Add template section to tensor, layers, activations, and cnn modules
- Create docs/development/module-template.md for future reference
- Clarify learning vs building structure consistently
- Show students where their code will live in the final package
- Decouple learning modules from production organization
- Add cnn_dev.py with NBDev educational pattern, Conv2D for-loop TODO, and all scaffolding
- Add README.md explaining learning goals, what is implemented vs provided, and rationale
- Add tests/test_cnn.py for basic correctness and shape tests
- Generate cnn_dev.ipynb for notebook workflow
- Add matmul_naive function with for-loop implementation for learning
- Update Dense layer to support both NumPy (@) and naive matrix multiplication
- Add comprehensive tests comparing both implementations (correctness & performance)
- Include step-by-step computation visualization for 2x2 matrices
- Fix missing imports in tensor.py and activations.py
- Export both tensor and activations modules to package
This provides students with immediate success using NumPy while allowing them to
understand the underlying computation through explicit for-loops. The scaffolding
includes performance comparisons and educational insights about why NumPy is faster.
- Add modules/networks/networks_dev.py and networks_dev.ipynb (Jupytext/nbdev educational pattern)
- Add comprehensive visualizations: architecture, data flow, layer analysis, network comparison
- Add modules/networks/README.md with learning goals, usage, and visualization docs
- Add modules/networks/tests/test_networks.py with thorough tests for composition, MLPs, and visualizations
- Register 'networks' in CLI info and test commands
- Update CLI info command to check layers/networks status
- This module focuses on forward pass only (no training yet)
- Restored tools/py_to_notebook.py as a focused, standalone tool
- Updated tito notebooks command to use subprocess to call the separate tool
- Maintains clean separation of concerns: tito.py for CLI orchestration, py_to_notebook.py for conversion logic
- Updated documentation to use 'tito notebooks' command instead of direct tool calls
- Benefits: easier debugging, better maintainability, focused single-responsibility modules
��️ Major architectural improvement implementing clean separation of concerns:
✨ NEW: Activations Module
- Complete activations module with ReLU, Sigmoid, Tanh implementations
- Educational NBDev structure with student TODOs + instructor solutions
- Comprehensive testing suite (24 tests) with mathematical correctness validation
- Visual learning features with matplotlib plotting (disabled during testing)
- Clean export to tinytorch.core.activations
🔧 REFACTOR: Layers Module
- Removed duplicate activation function implementations
- Clean import from activations module: 'from tinytorch.core.activations import ReLU, Sigmoid, Tanh'
- Updated documentation to reflect modular architecture
- Preserved all existing functionality while improving code organization
🧪 TESTING: Comprehensive Test Coverage
- All 24 activations tests passing ✅
- All 17 layers tests passing ✅
- Integration tests verify clean architecture works end-to-end
- CLI testing with 'tito test --module' works for both modules
📦 ARCHITECTURE: Clean Dependency Graph
- activations (math functions) → layers (building blocks) → networks (applications)
- Separation of concerns: pure math vs. neural network components
- Reusable components across future modules
- Single source of truth for activation implementations
�� PEDAGOGY: Enhanced Learning Experience
- Week-sized chunks: students master activations, then build layers
- Clear progression from mathematical foundations to applications
- Real-world software architecture patterns
- Modular design principles in practice
This establishes the foundation for scalable, maintainable ML systems education.
✨ Features:
- Dense layer with Xavier initialization (y = Wx + b)
- Activation functions: ReLU, Sigmoid, Tanh
- Layer composition for building neural networks
- Comprehensive test suite (17 passed, 5 skipped stretch goals)
- Package-level integration tests (14 passed)
- Complete documentation and examples
🎯 Educational Design:
- Follows 'Build → Use → Understand' pedagogical framework
- Immediate visual feedback with working examples
- Progressive complexity from simple layers to full networks
- Students see neural networks as function composition
🧪 Testing Architecture:
- Module tests: 17/17 core tests pass, 5 stretch goals available
- Package tests: 14/14 integration tests pass
- Dual testing supports both learning and validation
📚 Complete Implementation:
- Dense layer with proper weight initialization
- Numerically stable activation functions
- Batch processing support
- Real-world examples (image classification network)
- CLI integration: 'tito test --module layers'
This establishes the fundamental building blocks students need
to understand neural networks before diving into training.
- Fixed module-level tests to work with current implementation
- Added stretch goal testing pattern with pytest.skip()
- Created comprehensive testing documentation
- Both test levels now pass: 29 package tests + 22 module tests
- 11 stretch goals available for student implementation
Package tests (integration): ✅ 29/29 passed
Module tests (development): ✅ 22/22 passed, 11 skipped (stretch goals)
This provides immediate functionality for students while offering
clear implementation targets for advanced features.
- Restore complete setup module with system information and developer profiles
- Add NBDev educational patterns with #| export and #| hide directives
- Implement SystemInfo class for platform detection and compatibility checking
- Add DeveloperProfile class with ASCII art customization and personalization
- Include comprehensive educational content with step-by-step learning progression
- Expand test suite to 20 comprehensive tests covering all functionality:
- Function execution and output validation
- Arithmetic operations (basic, negative, floating-point)
- System information collection and compatibility
- Developer profile creation and customization
- ASCII art file loading and fallback behavior
- Error recovery and integration testing
- Update README with complete feature documentation and usage examples
- Convert to executed notebook showing rich output and system information
- Maintain file-based ASCII art system with tinytorch_flame.txt
- All tests passing with professional pytest structure
- Replace manual testing with pytest test classes and assertions
- Add comprehensive test coverage:
- Function execution without errors
- Correct output content validation
- ASCII art file existence and content checks
- Error handling and fallback behavior
- Missing file graceful handling
- Update README with pytest testing instructions
- Make testing consistent with other TinyTorch modules
- All 5 tests passing with proper pytest integration
- Replace complex setup module with simple hello_tinytorch() function
- Keep only the ASCII art file (tinytorch_flame.txt) for visual appeal
- Simplify tests to just verify function runs and file exists
- Update README to reflect simplified purpose and usage
- Remove complex developer profiles and system info classes
- Focus on minimal introduction to TinyTorch workflow
🔧 Notebook Compatibility:
- Fix __file__ NameError in notebook environment
- Add try/except for file path detection
- Handle both script and notebook execution contexts
- Use os.getcwd() fallback when __file__ not available
📓 Notebook Execution:
- Execute setup_dev.ipynb with all outputs generated
- Beautiful ASCII art displays properly in notebook
- All test cells show expected results
- Student-friendly interactive experience
🎨 ASCII Art Display:
- Stunning flame ASCII art renders perfectly in notebook
- Full profile display with Tiny🔥Torch branding
- Interactive testing cells with clear outputs
- Students can see the ASCII art immediately
🧪 Testing:
- All 17 tests still pass after notebook conversion
- File loading works in both environments
- Comprehensive coverage maintained
- Ready for student use
Students can now run the setup_dev.ipynb notebook and immediately see the beautiful ASCII art output, making their first TinyTorch experience memorable and engaging
