6 Commits

Author	SHA1	Message	Date
Vijay Janapa Reddi	3dfaca0f19	Add Profiler demo to Module 18 Compression ... - Added Section 8.5: Measuring Compression Impact with Profiler - Demonstrates 70% magnitude pruning parameter reduction - Shows sparsity measurements and active parameter counts - Uses Profiler from Module 15 for measurements - Educates students on compression workflow: measure prune validate deploy	2025-11-06 20:38:50 -05:00
Vijay Janapa Reddi	a64d636256	Streamline Module 18 Compression (Option 2: Moderate cleanup) ... - Removed Section 9: Systems Analysis (118 lines) - Removed analyze_compression_accuracy_tradeoff function (56 lines) - Replaced minimal Tensor/Linear implementations with proper imports (57 lines saved) - Added CompressionComplete export class with all core methods (120 lines) - Net reduction: 111 lines (7%) Result: 1564 → 1453 lines Focus: Core compression techniques (pruning, distillation, low-rank) Imports: Now uses tinytorch.core.tensor and tinytorch.core.layers	2025-11-06 20:13:51 -05:00
Vijay Janapa Reddi	dfc8577cad	feat: Add PyTorch-style __call__ methods and update milestone syntax ... This commit implements comprehensive PyTorch compatibility improvements: **Core Changes:** - Add __call__ methods to all neural network components in modules 11-18 - Enable PyTorch-standard calling syntax: model(input) vs model.forward(input) - Maintain backward compatibility - forward() methods still work **Modules Updated:** - Module 11 (Embeddings): Embedding, PositionalEncoding, EmbeddingLayer - Module 12 (Attention): MultiHeadAttention - Module 13 (Transformers): LayerNorm, MLP, TransformerBlock, GPT - Module 17 (Quantization): QuantizedLinear - Module 18 (Compression): Linear, Sequential classes **Milestone Updates:** - Replace all .forward() calls with direct () calls in milestone examples - Update transformer milestones (vaswani_shakespeare, tinystories_gpt, tinytalks_gpt) - Update CNN and MLP milestone examples - Update MILESTONE_TEMPLATE.py for consistency **Educational Benefits:** - Students now write identical syntax to production PyTorch code - Seamless transition from TinyTorch to PyTorch development - Industry-standard calling conventions from day one **Implementation Pattern:** ```python def __call__(self, *args, **kwargs): """Allows the component to be called like a function.""" return self.forward(*args, **kwargs) ``` All changes maintain full backward compatibility while enabling PyTorch-style usage.(https://claude.ai/code)	2025-10-28 13:46:05 -04:00
Vijay Janapa Reddi	8be87d0add	Fix nbdev export system across all 20 modules ... PROBLEM: - nbdev requires #| export directive on EACH cell to export when using # %% markers - Cell markers inside class definitions split classes across multiple cells - Only partial classes were being exported to tinytorch package - Missing matmul, arithmetic operations, and activation classes in exports SOLUTION: 1. Removed # %% cell markers INSIDE class definitions (kept classes as single units) 2. Added #| export to imports cell at top of each module 3. Added #| export before each exportable class definition in all 20 modules 4. Added __call__ method to Sigmoid for functional usage 5. Fixed numpy import (moved to module level from __init__) MODULES FIXED: - 01_tensor: Tensor class with all operations (matmul, arithmetic, shape ops) - 02_activations: Sigmoid, ReLU, Tanh, GELU, Softmax classes - 03_layers: Linear, Dropout classes - 04_losses: MSELoss, CrossEntropyLoss, BinaryCrossEntropyLoss classes - 05_autograd: Function, AddBackward, MulBackward, MatmulBackward, SumBackward - 06_optimizers: Optimizer, SGD, Adam, AdamW classes - 07_training: CosineSchedule, Trainer classes - 08_dataloader: Dataset, TensorDataset, DataLoader classes - 09_spatial: Conv2d, MaxPool2d, AvgPool2d, SimpleCNN classes - 10-20: All exportable classes in remaining modules TESTING: - Test functions use 'if __name__ == "__main__"' guards - Tests run in notebooks but NOT on import - Rosenblatt Perceptron milestone working perfectly RESULT: ✅ All 20 modules export correctly ✅ Perceptron (1957) milestone functional ✅ Clean separation: development (modules/source) vs package (tinytorch)	2025-09-30 11:21:04 -04:00
Vijay Janapa Reddi	9cfc7673cf	feat: update advanced modules (09-20) with latest improvements ... - Update spatial, tokenization, embeddings, attention modules - Update transformers, kv-caching, profiling modules - Update acceleration, quantization, compression modules - Update benchmarking and capstone modules - Align with current TinyTorch standards and patterns	2025-09-30 09:45:00 -04:00
Vijay Janapa Reddi	e1a9541c4b	Clean up module imports: convert tinytorch.core to sys.path style ... - Remove circular imports where modules imported from themselves - Convert tinytorch.core imports to sys.path relative imports - Only import dependencies that are actually used in each module - Preserve documentation imports in markdown cells - Use consistent relative path pattern across all modules - Remove hardcoded absolute paths in favor of relative imports Affected modules: 02_activations, 03_layers, 04_losses, 06_optimizers, 07_training, 09_spatial, 12_attention, 17_quantization	2025-09-30 08:58:58 -04:00