Vijay Janapa Reddi 5094c611bd Fix gradient propagation: enable autograd and patch activations/losses ...

CRITICAL FIX: Gradients now flow through entire training stack!

Changes:
1. Enable autograd in __init__.py - patches Tensor operations on import
2. Extend enable_autograd() to patch Sigmoid and BCE forward methods
3. Fix gradient accumulation to handle broadcasting (bias gradients)
4. Fix optimizer.step() - param.grad is numpy array, not Tensor.data
5. Add debug_gradients.py for systematic gradient flow testing

Architecture:
- Clean patching pattern - all gradient tracking in enable_autograd()
- Activations/losses remain simple (Module 02/04)
- Autograd (Module 05) upgrades them with gradient tracking
- Pedagogically sound: separation of concerns

Results:
✅ All 6 debug tests pass
✅ Perceptron learns: 50% → 93% accuracy
✅ Loss decreases: 0.79 → 0.36
✅ Weights update correctly through SGD

2025-09-30 13:51:30 -04:00

..

optimizers_dev.ipynb

Fix gradient propagation: enable autograd and patch activations/losses

2025-09-30 13:51:30 -04:00

optimizers_dev.py

Fix gradient propagation: enable autograd and patch activations/losses

2025-09-30 13:51:30 -04:00