- Replace existing tests with comprehensive educational tests
- Add 10 comprehensive test cases covering matrix multiplication and Dense layers
- Include basic operations, different shapes, edge cases, and initialization
- Add layer composition and real neural network scenarios
- Test integration with activation functions and batch processing
- Provide detailed feedback, hints, and progress tracking
- Follow inline-first testing approach for immediate feedback
- Replace existing tests with comprehensive educational tests
- Add 12 comprehensive test cases covering all activation functions
- Include ReLU, Sigmoid, Tanh, and Softmax testing
- Add edge cases, numerical stability, and shape preservation tests
- Add function composition and real ML scenario testing
- Provide detailed feedback, hints, and progress tracking
- Follow inline-first testing approach for immediate feedback
- Add 17 intermediate test points across 6 modules for immediate student feedback
- Tensor module: Tests after creation, properties, arithmetic, and operators
- Activations module: Tests after each activation function (ReLU, Sigmoid, Tanh, Softmax)
- Layers module: Tests after matrix multiplication and Dense layer implementation
- Networks module: Tests after Sequential class and MLP creation
- CNN module: Tests after convolution, Conv2D layer, and flatten operations
- DataLoader module: Tests after Dataset interface and DataLoader class
- All tests include visual progress indicators and behavioral explanations
- Maintains NBGrader compliance with proper metadata and point allocation
- Enables steady forward progress and better debugging for students
- 100% test success rate across all modules and integration testing
✅ CONSOLIDATED ALL MODULES:
- tensor_dev.py: ✅ Already perfect (reference implementation)
- activations_dev.py: ✅ Already clean
- layers_dev.py: ✅ Consolidated duplicates, single matmul_naive + Dense
- networks_dev.py: ✅ Consolidated duplicates, single Sequential + create_mlp
- cnn_dev.py: ✅ Consolidated duplicates, single conv2d_naive + Conv2D + flatten
- dataloader_dev.py: ✅ Consolidated duplicates, single Dataset + DataLoader + SimpleDataset
🔧 STANDARDIZED PATTERN ACROSS ALL MODULES:
- One function/class per concept (no duplicates)
- Comprehensive educational comments with TODO, APPROACH, EXAMPLE, HINTS
- Complete solutions with ### BEGIN SOLUTION / ### END SOLUTION
- NBGrader metadata for all cells
- Comprehensive test cells with assertions
- Educational content explaining concepts and real-world applications
📊 VERIFICATION:
- All modules tested and working correctly
- All tests passing
- Clean educational structure maintained
- Production-ready implementations
🎉 RESULT: Complete TinyTorch educational framework with consistent,
clean, and comprehensive module structure following the tensor_dev.py pattern.
Ready for classroom use with professional-grade ML systems curriculum.
Enhanced all remaining modules with comprehensive educational content:
## Modules Updated
- ✅ 03_layers: Added NBGrader metadata, solution blocks for matmul_naive and Dense class
- ✅ 04_networks: Added NBGrader metadata, solution blocks for Sequential class and forward pass
- ✅ 05_cnn: Added NBGrader metadata, solution blocks for conv2d_naive function and Conv2D class
- ✅ 06_dataloader: Added NBGrader metadata, solution blocks for Dataset base class
## Key Features Added
- **NBGrader Metadata**: All cells properly tagged with grade, grade_id, locked, schema_version, solution, task flags
- **Solution Blocks**: All TODO sections now have ### BEGIN SOLUTION / ### END SOLUTION markers
- **Import Flexibility**: Robust import handling for development vs package usage
- **Educational Content**: Package structure documentation and mathematical foundations
- **Comprehensive Testing**: All modules run correctly as Python scripts
## Verification Results
- ✅ All modules execute without errors
- ✅ All solution blocks implemented correctly
- ✅ Export workflow works: tito export --all successfully exports all modules
- ✅ Package integration verified: all imports work correctly
- ✅ Educational content preserved and enhanced
## Ready for Production
- Complete NBGrader-compatible assignment system
- Streamlined tito export command with automatic .py → .ipynb conversion
- Comprehensive educational modules with real-world applications
- Robust testing infrastructure for all components
Total modules completed: 6/6 (setup, tensor, activations, layers, networks, cnn, dataloader)
- Added package structure documentation explaining modules/source/ vs tinytorch.core.
- Enhanced mathematical foundations with linear algebra refresher and Universal Approximation Theorem
- Added real-world applications for each activation function (ReLU, Sigmoid, Tanh, Softmax)
- Included mathematical properties, derivatives, ranges, and computational costs
- Added performance considerations and numerical stability explanations
- Connected to production ML systems (PyTorch, TensorFlow, JAX equivalents)
- Implemented streamlined 'tito export' command with automatic .py → .ipynb conversion
- All functionality preserved: scripts run correctly, tests pass, package integration works
- Ready to continue with remaining modules (layers, networks, cnn, dataloader)
- Remove unnecessary module_paths.txt file for cleaner architecture
- Update export command to discover modules dynamically from modules/source/
- Simplify nbdev command to support --all and module-specific exports
- Use single source of truth: nbdev settings.ini for module paths
- Clean up import structure in setup module for proper nbdev export
- Maintain clean separation between module discovery and export logic
This implements a proper software engineering approach with:
- Single source of truth (settings.ini)
- Dynamic discovery (no hardcoded paths)
- Clean CLI interface (tito package nbdev --export [--all|module])
- Robust error handling with helpful feedback
- Migrated all Python source files to assignments/source/ structure
- Updated nbdev configuration to use assignments/source as nbs_path
- Updated all tito commands (nbgrader, export, test) to use new structure
- Fixed hardcoded paths in Python files and documentation
- Updated config.py to use assignments/source instead of modules
- Fixed test command to use correct file naming (short names vs full module names)
- Regenerated all notebook files with clean metadata
- Verified complete workflow: Python source → NBGrader → nbdev export → testing
All systems now working: NBGrader (14 source assignments, 1 released), nbdev export (7 generated files), and pytest integration.
The modules/ directory has been retired and replaced with standard NBGrader structure.
✅ PYTHON-FIRST DEVELOPMENT:
- Always work in raw Python files (modules/XX/XX_dev.py)
- Generate Jupyter notebooks on demand using Jupytext
- NBGrader compliance through automated cell metadata
- nbdev for package building and exports
🔧 WORKFLOW IMPROVEMENTS:
- Fixed file priority: use XX_dev.py over XX_dev_enhanced.py
- Clean up enhanced files to use standard files as source of truth
- Updated documentation to highlight Python-first approach
📚 COMPLETE INSTRUCTOR WORKFLOW:
1. Edit modules/XX/XX_dev.py (Python source of truth)
2. Export to package: tito module export XX (nbdev)
3. Generate assignment: tito nbgrader generate XX (Python→Jupyter→NBGrader)
4. Release to students: tito nbgrader release XX
5. Auto-grade with pytest: tito nbgrader autograde XX
✅ VERIFIED WORKING:
- Python file editing ✅
- nbdev export to tinytorch package ✅
- Jupytext conversion to notebooks ✅
- NBGrader assignment generation ✅
- pytest integration for auto-grading ✅🎯 TOOLS INTEGRATION:
- Raw Python development (version control friendly)
- Jupytext (Python ↔ Jupyter conversion)
- nbdev (package building and exports)
- NBGrader (student assignments and auto-grading)
- pytest (testing within notebooks)
Perfect implementation of user's ideal workflow
- Move development artifacts to development/archived/ directory
- Remove NBGrader artifacts (assignments/, testing/, gradebook.db, logs)
- Update root README.md to match actual repository structure
- Provide clear navigation paths for instructors and students
- Remove outdated documentation references
- Clean root directory while preserving essential files
- Maintain all functionality while improving organization
Repository is now optimally structured for classroom use with clear entry points:
- Instructors: docs/INSTRUCTOR_GUIDE.md
- Students: docs/STUDENT_GUIDE.md
- Developers: docs/development/
✅ All functionality verified working after restructuring
- CNN builds directly on layers/networks concepts while fresh
- Creates natural progression: layers → networks → cnn → dataloader
- 'Complete the layer toolkit first' before moving to data systems
- Move compression to 10 (right after training - model optimization)
- Move mlops to 11 (deployment comes next - production readiness)
- Move profiling to 12 (optimization tools - when needed)
- Move benchmarking to 13 (comparative analysis - with profiling)
- Move kernels to 14 (advanced optimization)
- Move transformer to 15 (final advanced topic)
Better Learning Flow:
- Training → 'I have a model, now what?' → Deployment
- Deployment → 'Now let me optimize' → Profiling/Benchmarking
- Mirrors real ML engineering: Deploy first, optimize second
- Profiling + Benchmarking work together naturally
Final progression (00-15):
00_setup → 01_tensor → 02_activations → 03_layers → 04_networks →
05_dataloader → 06_cnn → 07_autograd → 08_optimizers → 09_training →
10_compression → 11_mlops → 12_profiling → 13_benchmarking →
14_kernels → 15_transformer
- Add CNN as module 06 (after networks, before autograd)
- Shift all subsequent modules up by one number
- Add Transformer as module 15 (advanced/final module)
- Complete logical progression from foundations to production systems
Final progression:
00_setup → 01_tensor → 02_activations → 03_layers → 04_networks →
05_dataloader → 06_cnn → 07_autograd → 08_training → 09_config →
10_profiling → 11_compression → 12_kernels → 13_benchmarking →
14_mlops → 15_transformer
Clear student learning path: Foundation → Core ML → Advanced Architectures →
Training Systems → Production Systems → Advanced Topics
- Update all test files to import from tinytorch.core.* instead of relative paths
- Consistent with rock solid foundation principle
- Tests now use stable package imports, not local module imports
- Ensures tests validate the actual exported package functionality
- Aligns with production usage patterns
- Fix module imports to use tinytorch.core.* instead of local module imports
- Activations module now imports from tinytorch.core.tensor for stability
- Layers module imports from tinytorch.core.tensor and tinytorch.core.activations
- Test files updated to use main package imports for dependencies
- This ensures students can focus on current module without dependency issues
- Previous modules are 'locked in' and guaranteed to work
- Mirrors real-world usage patterns like PyTorch
- Maintains educational progression while ensuring system stability
- Reorganize hidden solutions to follow correct NBDev pattern (after each student implementation)
- Add comprehensive visualization system with _should_show_plots() control
- Implement smart test/development mode detection to suppress visualizations during pytest
- Add individual activation function plots and sample data visualizations
- Include comprehensive comparison plots for all activation functions
- Maintain educational inline tests with immediate feedback
- All 24 tests pass cleanly with proper visualization control
- Follows same structure as layers and tensor modules
Features:
- ReLU, Sigmoid, Tanh, Softmax visualizations with mathematical properties
- Real-time visual feedback during development
- Clean test execution without visual noise
- Educational context and use-case guidance for each activation
- Remove duplicate class definitions (was 800 lines → 517 lines)
- Follow consistent educational pattern like other modules
- Improve Build → Use → Reflect pedagogical framework
- Clean up TODO sections with proper implementation guidance
- Add comprehensive docstrings and examples
- Organize student and instructor implementations properly
- Maintain all functionality while improving readability
- All tests still pass (24/24 activations tests)
- Change --all flag meaning from 'clean both file types' to 'clean all modules'
- Make clean command consistent with test and export commands
- Require explicit module name or --all flag (no implicit behavior)
- Update help text and examples
- Now supports both:
- tito module clean tensor (specific module)
- tito module clean --all (all modules)
- Update test, export, and clean commands to use positional arguments
- Change from 'tito module test --module dataloader' to 'tito module test dataloader'
- Eliminates redundant --module flag within module command group
- Update help text and examples to reflect new syntax
- Maintains backward compatibility with --all flag
- More intuitive and consistent CLI design
- Move export functionality from 'tito package export' to 'tito module export'
- Require --all flag for exporting all modules (consistent with test command)
- Remove export from package command group to eliminate duplication
- Update help text and examples across all commands
- Fix tensor module arithmetic operators for complete functionality
- Clean up duplicate _quarto.yml and sidebar.yml files in modules/
This creates a consistent CLI pattern:
- tito module export --all (export all modules)
- tito module export --module <name> (export specific module)
- tito module test --all (test all modules)
- tito module test --module <name> (test specific module)
- Remove redundant fields from module.yaml files: exports_to, files, components
- Keep only essential system metadata: name, title, description, dependencies
- Export command now reads actual export targets from dev files (#| default_exp directive)
- Status command updated to use dev files as source of truth for export targets
- Export command shows detailed source → target mapping for better clarity
- Dependencies field retained as it's useful for CLI module ordering and prerequisites
- Eliminates duplication between YAML and dev files - dev files are the real truth
- Add Module Info sections with difficulty ratings to all README.md files
- Use consistent 4-star difficulty scale: ⭐ Beginner, ⭐⭐ Intermediate, ⭐⭐⭐ Advanced, ⭐⭐⭐⭐ Expert
- Include time estimates, prerequisites, and next steps for each module
- Maintain clear separation: README.md = student experience, module.yaml = system metadata
- Difficulty progression: Setup (⭐) → Tensor/Activations/Layers (⭐⭐) → Networks/CNN/DataLoader (⭐⭐⭐) → Transformer (⭐⭐⭐⭐)
- Help students plan their learning journey and set appropriate expectations
- Add new CleanCommand for cleaning up module directories
- Supports cleaning notebooks (*.ipynb) and cache files (__pycache__, *.pyc)
- Can clean specific modules or all modules
- Provides preview of files to be cleaned with confirmation
- Includes --force flag to skip confirmation
- Integrates with module command group as 'tito module clean'
- Preserves Python source files (*_dev.py) and other important files
- Fixes issue with duplicate file removal from __pycache__ directories
- Delete bin/py_to_notebook.py and tito/tools/py_to_notebook.py
- Update notebooks command to use Jupytext directly
- Jupytext is already configured in all *_dev.py files
- Simpler, more standard workflow using established tools
- Better integration with NBDev ecosystem
Benefits:
- Eliminates duplicate conversion tools
- Uses industry-standard Jupytext instead of custom tool
- Reduces maintenance burden
- Better error handling and compatibility
- Remove version field from all module.yaml files
- Update template generator to exclude version field
- Further simplify metadata to focus on system information only
- Status remains dynamically determined by test results
- Reduce module.yaml files from 100+ lines to ~25 lines focused on system needs
- Remove pedagogical details (learning objectives, difficulty, time estimates)
- Keep only essential fields: name, title, description, status, dependencies, exports, files, components
- Update status command to work with simplified metadata format
- Update metadata generation script to create simplified templates
- Focus on system metadata for CLI tools and build systems, not educational content
Before: Verbose pedagogical metadata with 20+ fields
After: Concise system metadata with 8 core fields
This aligns with the principle that module.yaml should be for systems, not pedagogy.
- Changed all module titles to be short and clean (e.g., 'Autograd' not 'Autograd - Automatic Differentiation')
- Updated metadata generation template to use concise titles by default
- Fixed CLI reference in metadata generator to use new hierarchical structure
- Titles are now consistent: just the module name capitalized
- Detailed descriptions remain in the description field where they belong
- Add module.yaml files for setup, tensor, activations, layers, and autograd modules
- Enhanced tito status command with --metadata flag for rich information display
- Created metadata schema with learning objectives, dependencies, components, and more
- Added metadata generation script (bin/generate_module_metadata.py)
- Comprehensive documentation in docs/development/module-metadata-system.md
- Status command now shows module status, difficulty, time estimates, and detailed metadata
- Supports dependency tracking, component-level status, and educational information
- Enables rich CLI experience with structured module information
- Rename modules/data/ → modules/dataloader/
- Rename data_dev.py → dataloader_dev.py
- Update NBDev export target: core.data → core.dataloader
- Rename test files: test_data.py → test_dataloader.py
- Update package exports to tinytorch.core.dataloader
- Update module imports and internal references
This makes the module name more descriptive and aligned with ML industry standards.
- Added pytest-timeout configuration with 5-minute timeout for all tests
- Added timeout handling to test command with proper error messages
- Created small local test dataset (50 train + 20 test samples) that mimics CIFAR-10 structure
- Updated data module tests to use local test data instead of downloading CIFAR-10
- Tests now run much faster (~0.1s vs ~30s) and don't require internet connection
- Added TestCIFAR10Dataset class that loads from local pickle files
- All test functionality preserved but using local data for speed and reliability
- Fixed plt.show() call in data module to respect _should_show_plots() check
- Protected test code that calls visualization functions in data module
- Protected test code that calls visualization functions in networks module
- All visualization functions now properly skip during testing
- Tests should no longer block waiting for user interaction
Introduces documentation for TinyTorch module development, including guides for developers and AI assistants.
Provides comprehensive resources for creating high-quality, educational modules, focusing on real-world applications and systems thinking.
Initializes the networks module, enabling the composition of layers into complete neural network architectures.
It introduces sequential networks, MLP creation, and network visualization tools to facilitate architecture understanding and analysis.
Adds practical classification and regression network implementations and network behavior analysis capabilities.
Adds ReLU, Sigmoid, and Tanh activation functions, enabling
non-linearity in neural networks.
Includes testing and visualization of each function to ensure
correct behavior and understanding of their properties.
Introduces a comprehensive module for 2D Convolutional Neural Networks.
This module provides a foundational understanding of CNNs through:
- Implementation of a naive Conv2D layer with sliding window convolution
- Visualization of kernel operations and feature map construction
- Composition of Conv2D layers with other layers to build a simple ConvNet
This structure provides a step-by-step guide to building and understanding CNNs, with clear examples and tests.
This commit introduces the core building blocks for neural networks,
including a naive matrix multiplication implementation and a Dense layer.
It provides a foundation for constructing and experimenting with
neural networks, emphasizing the concept of layers as tensor
transformations and function composition.
The module includes thorough testing and performance comparisons
to demonstrate the importance of optimized operations.
Implements the core Tensor class with data handling and properties, including initialization, shape, size, dtype, and string representation.
Adds element-wise addition and multiplication functions for tensors.
Implements tensor addition and multiplication as methods within the Tensor class.
