mirror of
https://github.com/MLSysBook/TinyTorch.git
synced 2026-04-28 04:38:04 -05:00
ad5404cb2e
New Features: - Add MSEBackward gradient computation for regression tasks - Patch MSELoss in enable_autograd() for gradient tracking - All 3 loss functions now support autograd: MSE, BCE, CrossEntropy Test Suite Organization: - Reorganize tests/ into focused directories - Create tests/integration/ for cross-module tests - Create tests/05_autograd/ for autograd edge cases - Create tests/debugging/ for common student pitfalls - Add comprehensive tests/README.md explaining test philosophy Integration Tests: - Move test_gradient_flow.py to integration/ - 20 comprehensive gradient flow tests - Tests cover: tensors, layers, activations, losses, optimizers - Tests validate: basic ops, chain rule, broadcasting, training loops - 19/20 tests passing (MSE now fixed!) Results: ✅ Perceptron learns: 50% → 93% accuracy ✅ Clean test organization guides future development ✅ Tests catch the exact bugs that broke training Pedagogical Value: - Test organization teaches testing best practices - Gradient flow tests show what integration testing catches - Sets foundation for debugging/diagnostic tests
TinyTorch Test Suite
Comprehensive testing organized by purpose and scope.
Test Organization
📦 Module Tests (XX_modulename/)
Purpose: Test individual module functionality
Scope: Single module, isolated behavior
Example: 01_tensor/test_progressive_integration.py
These tests validate that each module works correctly in isolation.
🔗 Integration Tests (integration/)
Purpose: Test cross-module interactions
Scope: Multiple modules working together
Files:
test_gradient_flow.py- CRITICAL: Validates gradients flow through entire training stacktest_end_to_end_training.py- Full training loops (TODO)test_module_compatibility.py- Module interfaces (TODO)
Why this matters:
- Catches bugs that unit tests miss
- Validates the "seams" between modules
- Ensures training actually works end-to-end
🐛 Debugging Tests (debugging/)
Purpose: Catch common student pitfalls
Scope: Pedagogical - teaches debugging
Files:
test_gradient_vanishing.py- Detect/diagnose vanishing gradients (TODO)test_gradient_explosion.py- Detect/diagnose exploding gradients (TODO)test_common_mistakes.py- "Did you forget backward()?" style tests (TODO)
Philosophy: When these tests fail, the error message should teach the student what went wrong and how to fix it.
⚡ Autograd Edge Cases (05_autograd/)
Purpose: Stress-test autograd system
Scope: Autograd internals and edge cases
Files:
test_broadcasting.py- Broadcasting gradient bugs (TODO)test_computation_graph.py- Graph construction edge cases (TODO)test_backward_edge_cases.py- Numerical stability, etc. (TODO)
Running Tests
All tests
pytest tests/ -v
Integration tests only (recommended for debugging training issues)
pytest tests/integration/ -v
Specific test
pytest tests/integration/test_gradient_flow.py -v
Run without pytest
python tests/integration/test_gradient_flow.py
Test Philosophy
- Integration tests catch real bugs: The gradient flow test caught the exact bugs that prevented training
- Descriptive names: Test names should explain what they test
- Good error messages: When tests fail, students should understand why
- Pedagogical value: Tests teach correct usage patterns
Adding New Tests
When adding a test, ask:
- Is it testing one module? → Put in
XX_modulename/ - Is it testing modules working together? → Put in
integration/ - Is it teaching debugging? → Put in
debugging/ - Is it an autograd edge case? → Put in
05_autograd/
Most Important Tests
🔥 Must pass before merging:
integration/test_gradient_flow.py- If this fails, training is broken
📚 Module validation:
- Each module's inline tests (in
modules/source/) - Module-specific tests in
tests/XX_modulename/
Test Coverage Goals
- ✅ All tensor operations have gradient tests
- ✅ All layers compute gradients correctly
- ✅ All activations integrate with autograd
- ✅ All loss functions compute gradients
- ✅ All optimizers update parameters
- ⏳ End-to-end training converges (TODO)
- ⏳ Common pitfalls are detected (TODO)