Initializes the networks module, enabling the composition of layers into complete neural network architectures.
It introduces sequential networks, MLP creation, and network visualization tools to facilitate architecture understanding and analysis.
Adds practical classification and regression network implementations and network behavior analysis capabilities.
Adds ReLU, Sigmoid, and Tanh activation functions, enabling
non-linearity in neural networks.
Includes testing and visualization of each function to ensure
correct behavior and understanding of their properties.
Introduces a comprehensive module for 2D Convolutional Neural Networks.
This module provides a foundational understanding of CNNs through:
- Implementation of a naive Conv2D layer with sliding window convolution
- Visualization of kernel operations and feature map construction
- Composition of Conv2D layers with other layers to build a simple ConvNet
This structure provides a step-by-step guide to building and understanding CNNs, with clear examples and tests.
This commit introduces the core building blocks for neural networks,
including a naive matrix multiplication implementation and a Dense layer.
It provides a foundation for constructing and experimenting with
neural networks, emphasizing the concept of layers as tensor
transformations and function composition.
The module includes thorough testing and performance comparisons
to demonstrate the importance of optimized operations.
Implements the core Tensor class with data handling and properties, including initialization, shape, size, dtype, and string representation.
Adds element-wise addition and multiplication functions for tensors.
Implements tensor addition and multiplication as methods within the Tensor class.
Defines the structure and guidelines for creating educational modules in TinyTorch using NBDev.
This includes file format, key NBDev directives, educational structure, implementation guidance, testing, naming conventions, and educational principles to promote effective learning.
Enhances the notebook by replacing some unicode characters with more standard and universally compatible symbols, improving the overall readability and user experience.
- Add template section to tensor, layers, activations, and cnn modules
- Create docs/development/module-template.md for future reference
- Clarify learning vs building structure consistently
- Show students where their code will live in the final package
- Decouple learning modules from production organization
- Add cnn_dev.py with NBDev educational pattern, Conv2D for-loop TODO, and all scaffolding
- Add README.md explaining learning goals, what is implemented vs provided, and rationale
- Add tests/test_cnn.py for basic correctness and shape tests
- Generate cnn_dev.ipynb for notebook workflow
- Add matmul_naive function with for-loop implementation for learning
- Update Dense layer to support both NumPy (@) and naive matrix multiplication
- Add comprehensive tests comparing both implementations (correctness & performance)
- Include step-by-step computation visualization for 2x2 matrices
- Fix missing imports in tensor.py and activations.py
- Export both tensor and activations modules to package
This provides students with immediate success using NumPy while allowing them to
understand the underlying computation through explicit for-loops. The scaffolding
includes performance comparisons and educational insights about why NumPy is faster.
- Add modules/networks/networks_dev.py and networks_dev.ipynb (Jupytext/nbdev educational pattern)
- Add comprehensive visualizations: architecture, data flow, layer analysis, network comparison
- Add modules/networks/README.md with learning goals, usage, and visualization docs
- Add modules/networks/tests/test_networks.py with thorough tests for composition, MLPs, and visualizations
- Register 'networks' in CLI info and test commands
- Update CLI info command to check layers/networks status
- This module focuses on forward pass only (no training yet)
- Remove 14 empty/unused directories from tinytorch/ package
- Keep only essential directories: core/, datasets/, configs/
- All directories removed contained only empty __init__.py files or were completely empty
- CLI functionality preserved and tested working
- Cleaner package structure for development
- Delete _proc/ directory (duplicate of modules/ with processed files)
- Delete bin/tito.py (old CLI implementation, replaced by tito/ structure)
- Delete temporary log files (tito-cli.log, tinytorch-cli.log)
- Update bin/tito wrapper to use new CLI structure
- All CLI functionality preserved with new architecture
- Updated git-workflow.mdc with proper metadata and focused content
- Updated development-workflow.mdc with proper metadata
- Added cli-patterns.mdc for CLI development best practices
- Followed Cursor's best practices: focused, actionable, scoped
- Added proper globs for auto-attachment to relevant files
- Improved rule descriptions for better AI context
Key improvements:
- Proper metadata structure with description, globs, alwaysApply
- Focused content under 500 lines per rule
- Concrete examples and patterns
- Clear, actionable guidance for AI
- Better scoping for when rules should apply
- Created comprehensive Git workflow guidelines for ad-hoc development
- Focus on small, focused commits for easy reverts
- Added practical examples for different development scenarios
- Updated development workflow to reference new Git guidelines
- Updated CLI commands to use 'tito' instead of 'python bin/tito.py'
- Includes commit message format, branch strategies, and revert procedures
Key principles:
- Incremental commits for easy reverts
- Test before committing to avoid broken commits
- Use feature branches for larger changes
- Descriptive commit messages that explain what changed
- Ported all commands from bin/tito.py to new tito/ CLI architecture
- Added InfoCommand with system info and module status
- Added TestCommand with pytest integration
- Added DoctorCommand with environment diagnosis
- Added SyncCommand for nbdev export functionality
- Added ResetCommand for package cleanup
- Added JupyterCommand for notebook server
- Added NbdevCommand for nbdev development tools
- Added SubmitCommand and StatusCommand (placeholders)
- Fixed missing imports in tinytorch/core/tensor.py
- All commands now work with 'tito' command in shell
- Maintains professional architecture while restoring full functionality
Commands restored:
✅ info - System information and module status
✅ test - Run module tests with pytest
✅ doctor - Environment diagnosis
✅ sync - Export notebooks to package
✅ reset - Clean tinytorch package
✅ nbdev - nbdev development commands
✅ jupyter - Start Jupyter server
✅ submit - Module submission
✅ status - Module status
✅ notebooks - Build notebooks from Python files
The CLI now has both the professional architecture and all original functionality.
- Moved tools/py_to_notebook.py to bin/py_to_notebook.py
- Updated tito.py to reference the new location
- Made py_to_notebook.py executable for direct invocation
- Removed empty tools/ directory
- Updated documentation to reflect new location
- All tools now consolidated in bin/ directory for consistency
Benefits:
- Conventional organization (bin/ for executables)
- Can invoke tools directly: ./bin/py_to_notebook.py
- Cleaner project structure
- Consistent with other tools (tito.py, generate_student_notebooks.py)
- Restored tools/py_to_notebook.py as a focused, standalone tool
- Updated tito notebooks command to use subprocess to call the separate tool
- Maintains clean separation of concerns: tito.py for CLI orchestration, py_to_notebook.py for conversion logic
- Updated documentation to use 'tito notebooks' command instead of direct tool calls
- Benefits: easier debugging, better maintainability, focused single-responsibility modules
- Added 'tito notebooks' command to build all module notebooks from *_dev.py files
- Added 'tito notebooks --module <name>' for building specific module notebooks
- Uses existing tools/py_to_notebook.py for conversion
- Provides clear success/error feedback with Rich formatting
- Includes helpful next steps guidance after generation
��️ Major architectural improvement implementing clean separation of concerns:
✨ NEW: Activations Module
- Complete activations module with ReLU, Sigmoid, Tanh implementations
- Educational NBDev structure with student TODOs + instructor solutions
- Comprehensive testing suite (24 tests) with mathematical correctness validation
- Visual learning features with matplotlib plotting (disabled during testing)
- Clean export to tinytorch.core.activations
🔧 REFACTOR: Layers Module
- Removed duplicate activation function implementations
- Clean import from activations module: 'from tinytorch.core.activations import ReLU, Sigmoid, Tanh'
- Updated documentation to reflect modular architecture
- Preserved all existing functionality while improving code organization
🧪 TESTING: Comprehensive Test Coverage
- All 24 activations tests passing ✅
- All 17 layers tests passing ✅
- Integration tests verify clean architecture works end-to-end
- CLI testing with 'tito test --module' works for both modules
📦 ARCHITECTURE: Clean Dependency Graph
- activations (math functions) → layers (building blocks) → networks (applications)
- Separation of concerns: pure math vs. neural network components
- Reusable components across future modules
- Single source of truth for activation implementations
�� PEDAGOGY: Enhanced Learning Experience
- Week-sized chunks: students master activations, then build layers
- Clear progression from mathematical foundations to applications
- Real-world software architecture patterns
- Modular design principles in practice
This establishes the foundation for scalable, maintainable ML systems education.
✨ Features:
- Dense layer with Xavier initialization (y = Wx + b)
- Activation functions: ReLU, Sigmoid, Tanh
- Layer composition for building neural networks
- Comprehensive test suite (17 passed, 5 skipped stretch goals)
- Package-level integration tests (14 passed)
- Complete documentation and examples
🎯 Educational Design:
- Follows 'Build → Use → Understand' pedagogical framework
- Immediate visual feedback with working examples
- Progressive complexity from simple layers to full networks
- Students see neural networks as function composition
🧪 Testing Architecture:
- Module tests: 17/17 core tests pass, 5 stretch goals available
- Package tests: 14/14 integration tests pass
- Dual testing supports both learning and validation
📚 Complete Implementation:
- Dense layer with proper weight initialization
- Numerically stable activation functions
- Batch processing support
- Real-world examples (image classification network)
- CLI integration: 'tito test --module layers'
This establishes the fundamental building blocks students need
to understand neural networks before diving into training.
- Move all documentation to docs/ directory with clear organization
- Use lowercase-with-dashes naming convention (modern standard)
- Organize by audience: students/, development/, pedagogy/
- Create comprehensive docs/README.md index
- Clean up root directory (only README.md and quickstart.md remain)
Structure:
docs/
├── pedagogy/ # Educational philosophy
├── development/ # Module development guides
├── students/ # Student-facing documentation
└── README.md # Documentation index
This makes the project more professional and easier to navigate.
- Document core learning philosophy for TinyTorch modules
- Define building block progression: Tensors → Layers → Networks → Training → Production
- Establish immediate feedback and concrete results principles
- Guide for module developers and students
- Foundation for all future module development
This framework ensures students build understanding through hands-on
implementation with immediate positive feedback at each step.
- Fixed module-level tests to work with current implementation
- Added stretch goal testing pattern with pytest.skip()
- Created comprehensive testing documentation
- Both test levels now pass: 29 package tests + 22 module tests
- 11 stretch goals available for student implementation
Package tests (integration): ✅ 29/29 passed
Module tests (development): ✅ 22/22 passed, 11 skipped (stretch goals)
This provides immediate functionality for students while offering
clear implementation targets for advanced features.
- Add explicit pytest requirement to testing-patterns.mdc
- Update user-preferences.mdc to include pytest preference
- Modify development-workflow.mdc to reference pytest usage
- Specify that all tests must use pytest framework
- Document pytest features to use (classes, fixtures, assertions)
- Add comprehensive examples of proper pytest structure
- Include DO NOT USE section for prohibited testing approaches
- Ensure CLI integration expects pytest-compatible test files
- Make it clear that manual testing and other frameworks are not allowed
- Restore complete setup module with system information and developer profiles
- Add NBDev educational patterns with #| export and #| hide directives
- Implement SystemInfo class for platform detection and compatibility checking
- Add DeveloperProfile class with ASCII art customization and personalization
- Include comprehensive educational content with step-by-step learning progression
- Expand test suite to 20 comprehensive tests covering all functionality:
- Function execution and output validation
- Arithmetic operations (basic, negative, floating-point)
- System information collection and compatibility
- Developer profile creation and customization
- ASCII art file loading and fallback behavior
- Error recovery and integration testing
- Update README with complete feature documentation and usage examples
- Convert to executed notebook showing rich output and system information
- Maintain file-based ASCII art system with tinytorch_flame.txt
- All tests passing with professional pytest structure
- Replace manual testing with pytest test classes and assertions
- Add comprehensive test coverage:
- Function execution without errors
- Correct output content validation
- ASCII art file existence and content checks
- Error handling and fallback behavior
- Missing file graceful handling
- Update README with pytest testing instructions
- Make testing consistent with other TinyTorch modules
- All 5 tests passing with proper pytest integration
- Replace complex setup module with simple hello_tinytorch() function
- Keep only the ASCII art file (tinytorch_flame.txt) for visual appeal
- Simplify tests to just verify function runs and file exists
- Update README to reflect simplified purpose and usage
- Remove complex developer profiles and system info classes
- Focus on minimal introduction to TinyTorch workflow
🔧 Notebook Compatibility:
- Fix __file__ NameError in notebook environment
- Add try/except for file path detection
- Handle both script and notebook execution contexts
- Use os.getcwd() fallback when __file__ not available
📓 Notebook Execution:
- Execute setup_dev.ipynb with all outputs generated
- Beautiful ASCII art displays properly in notebook
- All test cells show expected results
- Student-friendly interactive experience
🎨 ASCII Art Display:
- Stunning flame ASCII art renders perfectly in notebook
- Full profile display with Tiny🔥Torch branding
- Interactive testing cells with clear outputs
- Students can see the ASCII art immediately
🧪 Testing:
- All 17 tests still pass after notebook conversion
- File loading works in both environments
- Comprehensive coverage maintained
- Ready for student use
Students can now run the setup_dev.ipynb notebook and immediately see the beautiful ASCII art output, making their first TinyTorch experience memorable and engaging
🎨 File-Based ASCII Art System:
- Add beautiful tinytorch_flame.txt with sophisticated flame design
- Dynamic file loading with fallback to simple flame
- Easy customization - students can edit the file directly
- Proper error handling for missing files
🔥 Enhanced Visual Experience:
- Stunning detailed flame ASCII art from file
- 'Tiny🔥Torch' branding with 'Build ML Systems from Scratch!' tagline
- Much more sophisticated design than hardcoded version
- Students can easily customize by editing tinytorch_flame.txt
📚 Educational Benefits:
- Two customization options: parameter override or file editing
- Teaches file I/O and error handling concepts
- Encourages creative expression and ownership
- Professional development practices
🧪 Testing:
- Updated all tests to work with file-based system
- All 17 tests pass successfully
- Comprehensive coverage of file loading and fallback
- Verified both custom and default ASCII art functionality
Students can now easily create their own ASCII art by editing the tinytorch_flame.txt file or providing custom art via parameters
🎨 ASCII Art Features:
- Add beautiful TinyTorch flame ASCII art as default
- Support custom ASCII art for student personalization
- New get_ascii_art() and get_full_profile() methods
- Enhanced visual appeal and student engagement
👨💻 Profile Updates:
- Update instructor info: vj@eecs.harvard.edu, @profvjreddi
- Add comprehensive ASCII art examples and ideas
- Encourage creative student customization
- Professional signature generation with visual flair
🧪 Testing:
- Add 2 new ASCII art tests (17 total tests pass)
- Test default flame art and custom art functionality
- Verify full profile display with proper formatting
- Comprehensive coverage of all new features
Students can now create truly personalized TinyTorch experiences with custom ASCII art, making their learning journey unique and memorable
- Add DeveloperProfile class with customizable developer information
- Default to course instructor (Vijay Janapa Reddi) but allow student customization
- Include professional signature generation for code attribution
- Add comprehensive test coverage (6 new tests, 15 total tests pass)
- Encourage student ownership and personalization of their TinyTorch journey
- Maintain educational structure with TODO sections and hidden solutions
- Fix syntax errors in setup_dev.py test cells (proper indentation)
- Add comprehensive test suite for setup module (test_setup.py)
- Test coverage: functions, SystemInfo class, integration tests
- Custom test runner (no pytest dependency required)
- All 9 tests pass successfully
- Update .gitignore to allow modules/ directory for educational structure
- Fix CLI tool to look for tests in modules/{module}/tests/ instead of tests/
- Update test imports to use parent directory module imports
- Update Cursor rules to reflect new test structure:
* Project structure shows tests/ subdirectory
* Testing patterns show correct paths and import patterns
* Development workflow shows updated test locations
- Test imports now work: from tensor_dev import Tensor
- CLI commands now find tests in correct locations
Tests are now properly organized and discoverable
- Move test_tensor.py to modules/tensor/tests/test_tensor.py
- Create tests/ directories in setup and tensor modules
- Clean up setup module: remove setup_dev.html, setup_dev_files/, __init__.py
- Establish clean module structure pattern:
* {module}_dev.py - Main development file
* {module}_dev.ipynb - Generated notebook
* README.md - Module documentation
* tests/ - Test directory with test_{module}.py
This makes it crystal clear to students where tests are located.
- Fix frontmatter for all rules to include proper descriptions
- Update rule types according to Cursor guidelines:
* alwaysApply: project-structure, user-preferences
* globs: nbdev-educational-pattern, testing-patterns
* description only: development-workflow, ml-systems-course-context
- All rules under 500 lines, focused and actionable
- Follow MDC format with proper metadata
- Ready for proper Cursor rule functionality
- Project structure guide (always applied)
- NBDev educational pattern for module development files
- Development workflow and CLI usage
- Testing patterns and conventions
- ML Systems course context and learning approach
- User preferences and conventions
These rules will help with codebase navigation, understanding the educational approach,
and maintaining consistency across the TinyTorch framework.
🎯 Student Version (Visible):
- All functions raise NotImplementedError('Student implementation required')
- Clear TODO instructions with specific guidance
- Test cells handle NotImplementedError gracefully
- Students must implement everything from scratch
🔧 Instructor Version (Hidden with #|hide):
- Complete working implementations
- Same function signatures as student version
- Production-ready code for package export
📚 Added MODULE_GENERATION_GUIDE.md:
- Clear instructions for creating student/instructor modules
- NBDev workflow documentation
- Module structure patterns and templates
- Generation commands and checklist
🔄 Generation Flow:
1. Write .py with student stubs + hidden instructor solutions
2. Convert to notebook: jupytext --to notebook module_dev.py
3. Export package: python bin/tito.py sync --module name
4. Package gets instructor solutions, students see exercises
Perfect foundation for all TinyTorch modules
- Removed modules/setup/setup_nbdev_educational.* files
- Keep only the clean setup_dev.* files
- Setup module now focused and clean
- NBDev features work behind scenes without clutter
- Focused on original purpose: just setting up development environment
- Students don't see NBDev educational features explanations
- Simple workflow: hello world, basic class, export, test, progress
- NBDev #|hide directive works behind scenes for instructor solutions
- Clean and simple, just like the original but with hidden solutions
Back to basics: setup is about setup, not teaching NBDev features
SINGLE SOURCE approach: One notebook serves both instructors and students
🎯 Key Features:
- #|hide directive hides complete solutions from students
- #|code-fold creates collapsible sections for details
- #|export with educational metadata
- Progressive learning: simple → ML-relevant complexity
- Vector operations (add, dot product) as ML foundations
- ML-aware SystemInfo class with library checking
- Comprehensive testing for both student/instructor versions
🔄 Workflow:
- Students see exercises with TODOs and hints
- Instructors see complete solutions (hidden by default)
- Package exports get instructor (complete) implementations
- Single notebook maintains both audiences
This establishes the pattern for all TinyTorch modules
- Create tensor_nbdev_educational.py with NBDev directives
- Demonstrate #|hide, #|code-fold, #|filter_stream features
- Convert using standard Jupytext (# %% markers)
- Add comprehensive guides and comparisons
- Show superior approach vs custom generator
Key insight: NBDev already has mature educational capabilities!
No need to build custom tools when industry standards exist.
