Major directory restructure to support both developer and learner workflows:
Structure Changes:
- NEW: src/ directory for Python source files (version controlled)
- Files renamed: tensor.py → 01_tensor.py (matches directory naming)
- All 20 modules moved from modules/ to src/
- CHANGED: modules/ now holds generated notebooks (gitignored)
- Generated from src/*.py using jupytext
- Learners work in notebooks, developers work in Python source
- UNCHANGED: tinytorch/ package (still auto-generated from notebooks)
Workflow: src/*.py → modules/*.ipynb → tinytorch/*.py
Command Updates:
- Updated export command to read from src/ and generate to modules/
- Export flow: discovers modules in src/, converts to notebooks in modules/, exports to tinytorch/
- All 20 modules tested and working
Configuration:
- Updated .gitignore to ignore modules/ directory
- Updated README.md with new three-layer architecture explanation
- Updated export.py source mappings and paths
Benefits:
- Clean separation: developers edit Python, learners use notebooks
- Better version control: only Python source committed, notebooks generated
- Flexible learning: can work in notebooks OR Python source
- Maintains backward compatibility: tinytorch package unchanged
Tested:
- Single module export: tito export 01_tensor ✅
- All modules export: tito export --all ✅
- Package imports: from tinytorch.core.tensor import Tensor ✅
- 20/20 modules successfully converted and exported
Re-exported all modules after restructuring:
- Updated _modidx.py with new module locations
- Removed outdated autogeneration headers
- Updated all core modules (tensor, autograd, layers, etc.)
- Updated optimization modules (quantization, compression, etc.)
- Updated TITO commands for new structure
Changes include:
- 24 tinytorch/ module files
- 24 tito/ command and core files
- Updated references from modules/source/ to modules/
All modules re-exported via nbdev from their new locations.
- Added typing imports (List, Dict, Tuple, Optional, Set) to export section
- Fixed NameError: name 'List' is not defined
- Fixed milestone copilot references from SimpleTokenizer to CharTokenizer
- Verified transformer learning: 99.1% loss decrease in 500 steps
Training results:
- Initial loss: 3.555
- Final loss: 0.031
- Training time: 52.1s for 500 steps
- Gradient flow: All 21 parameters receiving gradients
- Model: 1-layer GPT with 32d embeddings, 4 heads
Issue: CharTokenizer was failing with NameError: name 'List' is not defined
Root cause: typing imports were not marked with #| export
Fix:
✅ Added #| export directive to import block in tokenization_dev.py
✅ Re-exported module using 'tito export 10_tokenization'
✅ typing.List, Dict, Tuple, Optional, Set now properly exported
Verification:
- CharTokenizer.build_vocab() works ✅
- encode() and decode() work ✅
- Tested on Shakespeare sample text ✅
This fixes the integration with vaswani_shakespeare.py which now properly
uses CharTokenizer from Module 10 instead of manual tokenization.
This commit includes:
- Exported tinytorch package files from nbdev (autograd, losses, optimizers, training, etc.)
- Updated activations.py and layers.py with __call__ methods
- New module exports: attention, spatial, tokenization, transformer, etc.
- Removed old _modidx.py file
- Cleanup of duplicate milestone directories
These are the generated package files that correspond to the source modules
we've been developing. Students will import from these when using TinyTorch.
