3 Commits

Author	SHA1	Message	Date
Vijay Janapa Reddi	fe07e2b7a5	fix(tokenization): Add missing imports to tokenization module ... - Added typing imports (List, Dict, Tuple, Optional, Set) to export section - Fixed NameError: name 'List' is not defined - Fixed milestone copilot references from SimpleTokenizer to CharTokenizer - Verified transformer learning: 99.1% loss decrease in 500 steps Training results: - Initial loss: 3.555 - Final loss: 0.031 - Training time: 52.1s for 500 steps - Gradient flow: All 21 parameters receiving gradients - Model: 1-layer GPT with 32d embeddings, 4 heads	2025-10-30 11:09:38 -04:00
Vijay Janapa Reddi	0e84216700	feat: Complete transformer integration with milestones ... - Add tokenization module (tinytorch/text/tokenization.py) - Update Milestone 05 transformer demos (validation, TinyCoder, Shakespeare) - Update book chapters with milestones overview - Update README and integration plan - Sync module notebooks and metadata	2025-10-19 12:46:58 -04:00
Vijay Janapa Reddi	8be87d0add	Fix nbdev export system across all 20 modules ... PROBLEM: - nbdev requires #| export directive on EACH cell to export when using # %% markers - Cell markers inside class definitions split classes across multiple cells - Only partial classes were being exported to tinytorch package - Missing matmul, arithmetic operations, and activation classes in exports SOLUTION: 1. Removed # %% cell markers INSIDE class definitions (kept classes as single units) 2. Added #| export to imports cell at top of each module 3. Added #| export before each exportable class definition in all 20 modules 4. Added __call__ method to Sigmoid for functional usage 5. Fixed numpy import (moved to module level from __init__) MODULES FIXED: - 01_tensor: Tensor class with all operations (matmul, arithmetic, shape ops) - 02_activations: Sigmoid, ReLU, Tanh, GELU, Softmax classes - 03_layers: Linear, Dropout classes - 04_losses: MSELoss, CrossEntropyLoss, BinaryCrossEntropyLoss classes - 05_autograd: Function, AddBackward, MulBackward, MatmulBackward, SumBackward - 06_optimizers: Optimizer, SGD, Adam, AdamW classes - 07_training: CosineSchedule, Trainer classes - 08_dataloader: Dataset, TensorDataset, DataLoader classes - 09_spatial: Conv2d, MaxPool2d, AvgPool2d, SimpleCNN classes - 10-20: All exportable classes in remaining modules TESTING: - Test functions use 'if __name__ == "__main__"' guards - Tests run in notebooks but NOT on import - Rosenblatt Perceptron milestone working perfectly RESULT: ✅ All 20 modules export correctly ✅ Perceptron (1957) milestone functional ✅ Clean separation: development (modules/source) vs package (tinytorch)	2025-09-30 11:21:04 -04:00