17 Commits

Author SHA1 Message Date
Vijay Janapa Reddi
4264699b5f Update test files with progressive integration and checkpoint improvements 2025-12-04 11:08:17 -08:00
Vijay Janapa Reddi
4aeb3c9c69 Merge main into dev, resolving conflicts with dev's version 2025-12-03 07:26:43 -08:00
Vijay Janapa Reddi
42e07151d5 Add subscribe modal popup with MLSysBook integration
- Add subscribe-modal.js with elegant popup form
- Update top bar: fire-themed dark design (56px), orange accent
- Subscribe button triggers modal instead of navigating away
- Modal shows MLSysBook + TinyTorch branding connection
- Form submits to mlsysbook newsletter with tinytorch-website tag
- Orange Subscribe button matches TinyTorch fire theme
- Responsive design with dark mode support
2025-12-03 05:56:38 -08:00
Vijay Janapa Reddi
bd7fcb2177 Release preparation: fix package exports, tests, and documentation
Package exports:
- Fix tinytorch/__init__.py to export all required components for milestones
- Add Dense as alias for Linear for compatibility
- Add loss functions (MSELoss, CrossEntropyLoss, BinaryCrossEntropyLoss)
- Export spatial operations, data loaders, and transformer components

Test infrastructure:
- Create tests/conftest.py to handle path setup
- Create tests/test_utils.py with shared test utilities
- Rename test_progressive_integration.py files to include module number
- Fix syntax errors in test files (spaces in class names)
- Remove stale test file referencing non-existent modules

Documentation:
- Update README.md with correct milestone file names
- Fix milestone requirements to match actual module dependencies

Export system:
- Run tito export --all to regenerate package from source modules
- Ensure all 20 modules are properly exported
2025-12-02 14:19:56 -05:00
Vijay Janapa Reddi
d3a126235c Restructure: Separate developer source (src/) from learner notebooks (modules/)
Major directory restructure to support both developer and learner workflows:

Structure Changes:
- NEW: src/ directory for Python source files (version controlled)
  - Files renamed: tensor.py → 01_tensor.py (matches directory naming)
  - All 20 modules moved from modules/ to src/
- CHANGED: modules/ now holds generated notebooks (gitignored)
  - Generated from src/*.py using jupytext
  - Learners work in notebooks, developers work in Python source
- UNCHANGED: tinytorch/ package (still auto-generated from notebooks)

Workflow: src/*.py → modules/*.ipynb → tinytorch/*.py

Command Updates:
- Updated export command to read from src/ and generate to modules/
- Export flow: discovers modules in src/, converts to notebooks in modules/, exports to tinytorch/
- All 20 modules tested and working

Configuration:
- Updated .gitignore to ignore modules/ directory
- Updated README.md with new three-layer architecture explanation
- Updated export.py source mappings and paths

Benefits:
- Clean separation: developers edit Python, learners use notebooks
- Better version control: only Python source committed, notebooks generated
- Flexible learning: can work in notebooks OR Python source
- Maintains backward compatibility: tinytorch package unchanged

Tested:
- Single module export: tito export 01_tensor 
- All modules export: tito export --all 
- Package imports: from tinytorch.core.tensor import Tensor 
- 20/20 modules successfully converted and exported
2025-11-25 00:02:21 -05:00
Vijay Janapa Reddi
0e135f1aea Implement Tensor slicing with progressive disclosure and fix embedding gradient flow
WHAT: Added Tensor.__getitem__ (slicing) following progressive disclosure principles

MODULE 01 (Tensor):
- Added __getitem__ method for basic slicing operations
- Clean implementation with NO gradient mentions (progressive disclosure)
- Supports all NumPy-style indexing: x[0], x[:3], x[1:4], x[:, 1]
- Ensures scalar results are wrapped in arrays

MODULE 05 (Autograd):
- Added SliceBackward function for gradient computation
- Implements proper gradient scatter: zeros everywhere except sliced positions
- Added monkey-patching in enable_autograd() for __getitem__
- Follows same pattern as existing operations (add, mul, matmul)

MODULE 11 (Embeddings):
- Updated PositionalEncoding to use Tensor slicing instead of .data
- Fixed multiple .data accesses that broke computation graphs
- Removed Tensor() wrapping that created gradient-disconnected leafs
- Uses proper Tensor operations to preserve gradient flow

TESTING:
- All 6 component tests PASS (Embedding, Attention, FFN, Residual, Forward, Training)
- 19/19 parameters get gradients (was 18/19 before)
- Loss dropping better: 1.54→1.08 (vs 1.62→1.24 before)
- Model still not learning (0% accuracy) - needs fresh session to test monkey-patching

WHY THIS MATTERS:
- Tensor slicing is FUNDAMENTAL - needed by transformers for position embeddings
- Progressive disclosure maintains educational integrity
- Follows existing TinyTorch architecture patterns
- Enables position embeddings to potentially learn (pending verification)

DOCUMENTS CREATED:
- milestones/05_2017_transformer/TENSOR_SLICING_IMPLEMENTATION.md
- milestones/05_2017_transformer/STATUS.md
- milestones/05_2017_transformer/FIXES_SUMMARY.md
- milestones/05_2017_transformer/DEBUG_REVERSAL.md
- tests/milestones/test_reversal_debug.py (component tests)

ARCHITECTURAL PRINCIPLE:
Progressive disclosure is not just nice-to-have, it's CRITICAL for educational systems.
Don't expose Module 05 concepts (gradients) in Module 01 (basic operations).
Monkey-patch when features are needed, not before.
2025-11-22 18:26:12 -05:00
Vijay Janapa Reddi
763cdd2bf2 Implement Tensor slicing with progressive disclosure and fix embedding gradient flow
WHAT: Added Tensor.__getitem__ (slicing) following progressive disclosure principles

MODULE 01 (Tensor):
- Added __getitem__ method for basic slicing operations
- Clean implementation with NO gradient mentions (progressive disclosure)
- Supports all NumPy-style indexing: x[0], x[:3], x[1:4], x[:, 1]
- Ensures scalar results are wrapped in arrays

MODULE 05 (Autograd):
- Added SliceBackward function for gradient computation
- Implements proper gradient scatter: zeros everywhere except sliced positions
- Added monkey-patching in enable_autograd() for __getitem__
- Follows same pattern as existing operations (add, mul, matmul)

MODULE 11 (Embeddings):
- Updated PositionalEncoding to use Tensor slicing instead of .data
- Fixed multiple .data accesses that broke computation graphs
- Removed Tensor() wrapping that created gradient-disconnected leafs
- Uses proper Tensor operations to preserve gradient flow

TESTING:
- All 6 component tests PASS (Embedding, Attention, FFN, Residual, Forward, Training)
- 19/19 parameters get gradients (was 18/19 before)
- Loss dropping better: 1.54→1.08 (vs 1.62→1.24 before)
- Model still not learning (0% accuracy) - needs fresh session to test monkey-patching

WHY THIS MATTERS:
- Tensor slicing is FUNDAMENTAL - needed by transformers for position embeddings
- Progressive disclosure maintains educational integrity
- Follows existing TinyTorch architecture patterns
- Enables position embeddings to potentially learn (pending verification)

DOCUMENTS CREATED:
- milestones/05_2017_transformer/TENSOR_SLICING_IMPLEMENTATION.md
- milestones/05_2017_transformer/STATUS.md
- milestones/05_2017_transformer/FIXES_SUMMARY.md
- milestones/05_2017_transformer/DEBUG_REVERSAL.md
- tests/milestones/test_reversal_debug.py (component tests)

ARCHITECTURAL PRINCIPLE:
Progressive disclosure is not just nice-to-have, it's CRITICAL for educational systems.
Don't expose Module 05 concepts (gradients) in Module 01 (basic operations).
Monkey-patch when features are needed, not before.
2025-11-22 18:26:12 -05:00
Vijay Janapa Reddi
41b132f55f Update tinytorch and tito with module exports
Re-exported all modules after restructuring:
- Updated _modidx.py with new module locations
- Removed outdated autogeneration headers
- Updated all core modules (tensor, autograd, layers, etc.)
- Updated optimization modules (quantization, compression, etc.)
- Updated TITO commands for new structure

Changes include:
- 24 tinytorch/ module files
- 24 tito/ command and core files
- Updated references from modules/source/ to modules/

All modules re-exported via nbdev from their new locations.
2025-11-10 19:42:03 -05:00
Vijay Janapa Reddi
96880b3133 Update tinytorch and tito with module exports
Re-exported all modules after restructuring:
- Updated _modidx.py with new module locations
- Removed outdated autogeneration headers
- Updated all core modules (tensor, autograd, layers, etc.)
- Updated optimization modules (quantization, compression, etc.)
- Updated TITO commands for new structure

Changes include:
- 24 tinytorch/ module files
- 24 tito/ command and core files
- Updated references from modules/source/ to modules/

All modules re-exported via nbdev from their new locations.
2025-11-10 19:42:03 -05:00
Vijay Janapa Reddi
ae330dd477 Regenerate tinytorch package from all module exports
- Run tito export --all to update all exported code
- Fix file permissions (chmod u+w) to allow export writes
- Update 12 modified files with latest module code
- Add 3 new files (tinygpt, acceleration, compression)
- All 21 modules successfully exported
2025-11-10 06:23:47 -05:00
Vijay Janapa Reddi
c188ccc1d3 Regenerate tinytorch package from all module exports
- Run tito export --all to update all exported code
- Fix file permissions (chmod u+w) to allow export writes
- Update 12 modified files with latest module code
- Add 3 new files (tinygpt, acceleration, compression)
- All 21 modules successfully exported
2025-11-10 06:23:47 -05:00
Vijay Janapa Reddi
4e1b536fac Module 17: Export QuantizationComplete for INT8 quantization
- Added QuantizationComplete class with quantize/dequantize methods
- Exported quantization functions to tinytorch/optimization/quantization.py
- Provides 4x memory reduction with minimal accuracy loss
- Removed pedagogical QuantizedLinear export to avoid conflicts
- Added proper imports to export block
2025-11-06 15:50:48 -05:00
Vijay Janapa Reddi
6259f91be9 Module 17: Export QuantizationComplete for INT8 quantization
- Added QuantizationComplete class with quantize/dequantize methods
- Exported quantization functions to tinytorch/optimization/quantization.py
- Provides 4x memory reduction with minimal accuracy loss
- Removed pedagogical QuantizedLinear export to avoid conflicts
- Added proper imports to export block
2025-11-06 15:50:48 -05:00
Vijay Janapa Reddi
caff73a75b Reset package and export modules 01-07 only (skip broken spatial module) 2025-09-30 13:41:00 -04:00
Vijay Janapa Reddi
ba6bd79a67 Reset package and export modules 01-07 only (skip broken spatial module) 2025-09-30 13:41:00 -04:00
Vijay Janapa Reddi
da6e4374e0 Add exported package files and cleanup
This commit includes:
- Exported tinytorch package files from nbdev (autograd, losses, optimizers, training, etc.)
- Updated activations.py and layers.py with __call__ methods
- New module exports: attention, spatial, tokenization, transformer, etc.
- Removed old _modidx.py file
- Cleanup of duplicate milestone directories

These are the generated package files that correspond to the source modules
we've been developing. Students will import from these when using TinyTorch.
2025-09-30 12:38:56 -04:00
Vijay Janapa Reddi
1f23035a1e Add exported package files and cleanup
This commit includes:
- Exported tinytorch package files from nbdev (autograd, losses, optimizers, training, etc.)
- Updated activations.py and layers.py with __call__ methods
- New module exports: attention, spatial, tokenization, transformer, etc.
- Removed old _modidx.py file
- Cleanup of duplicate milestone directories

These are the generated package files that correspond to the source modules
we've been developing. Students will import from these when using TinyTorch.
2025-09-30 12:38:56 -04:00