Vijay Janapa Reddi 87d5a7e381 fix(module-05): Add TransposeBackward and fix MatmulBackward for batched ops ...

TransposeBackward:
- New backward function for transpose operation
- Patch Tensor.transpose() to track gradients
- Critical for attention (Q @ K.T) gradient flow

MatmulBackward batched fix:
- Change np.dot to np.matmul for batched 3D+ tensors
- Use np.swapaxes instead of .T for proper batched transpose
- Fixes gradient shapes in attention mechanisms

Tests added:
- tests/05_autograd/test_batched_matmul_backward.py (3 tests)
- Updated tests/regression/test_gradient_flow_fixes.py (9 tests total)

All gradient flow issues for transformer training are now resolved!

2025-10-27 20:35:06 -04:00

..

applications

Reset package and export modules 01-07 only (skip broken spatial module)

2025-09-30 13:41:00 -04:00

backends

FOUNDATION: Establish AI Engineering as a discipline through TinyTorch

2025-09-25 11:16:28 -04:00

benchmarking

Reset package and export modules 01-07 only (skip broken spatial module)

2025-09-30 13:41:00 -04:00

configs

Adds initial TinyTorch CLI and core structure

2025-07-09 00:23:19 -04:00

core

fix(module-05): Add TransposeBackward and fix MatmulBackward for batched ops

2025-10-27 20:35:06 -04:00

datasets

Adds initial TinyTorch CLI and core structure

2025-07-09 00:23:19 -04:00

experimental

FOUNDATION: Establish AI Engineering as a discipline through TinyTorch

2025-09-25 11:16:28 -04:00

generation

Reset package and export modules 01-07 only (skip broken spatial module)

2025-09-30 13:41:00 -04:00

models

fix(module-13): Rewrite LayerNorm to use Tensor operations

2025-10-27 20:30:21 -04:00

nn

refactor: Keep explicit module imports + optimize CNN milestone

2025-09-30 17:15:40 -04:00

optim

Organize package with nn and optim modules

2025-09-23 08:10:47 -04:00

optimization

Reset package and export modules 01-07 only (skip broken spatial module)

2025-09-30 13:41:00 -04:00

profiler

FOUNDATION: Establish AI Engineering as a discipline through TinyTorch

2025-09-25 11:16:28 -04:00

profiling

Reset package and export modules 01-07 only (skip broken spatial module)

2025-09-30 13:41:00 -04:00

quantization

FOUNDATION: Establish AI Engineering as a discipline through TinyTorch

2025-09-25 11:16:28 -04:00

text

fix(module-11): Fix Embedding and PositionalEncoding gradient flow

2025-10-27 20:30:03 -04:00

utils

Clean up module imports: convert tinytorch.core to sys.path style

2025-09-30 08:58:58 -04:00

__init__.py

Fix gradient propagation: enable autograd and patch activations/losses

2025-09-30 13:51:30 -04:00

_modidx.py

🤖 Fix transformer module exports and milestone 05 imports

2025-10-27 16:17:55 -04:00