Vijay Janapa Reddi b5079bba40 fix(autograd): Add SoftmaxBackward and patch Softmax.forward() ...

- Implemented SoftmaxBackward with proper gradient formula
- Patched Softmax.forward() in enable_autograd()
- Fixed LayerNorm gamma/beta to have requires_grad=True

Progress:
- Softmax now correctly computes gradients
- LayerNorm parameters initialized with requires_grad
- Still debugging: Q/K/V projections, LayerNorms in blocks, MLP first layer

Current: 9/21 parameters receive gradients (was 0/21)

2025-10-28 08:04:19 -04:00

..

applications

Reset package and export modules 01-07 only (skip broken spatial module)

2025-09-30 13:41:00 -04:00

backends

FOUNDATION: Establish AI Engineering as a discipline through TinyTorch

2025-09-25 11:16:28 -04:00

benchmarking

Reset package and export modules 01-07 only (skip broken spatial module)

2025-09-30 13:41:00 -04:00

configs

Adds initial TinyTorch CLI and core structure

2025-07-09 00:23:19 -04:00

core

fix(autograd): Add SoftmaxBackward and patch Softmax.forward()

2025-10-28 08:04:19 -04:00

datasets

Adds initial TinyTorch CLI and core structure

2025-07-09 00:23:19 -04:00

experimental

FOUNDATION: Establish AI Engineering as a discipline through TinyTorch

2025-09-25 11:16:28 -04:00

generation

Reset package and export modules 01-07 only (skip broken spatial module)

2025-09-30 13:41:00 -04:00

models

fix(autograd): Add SoftmaxBackward and patch Softmax.forward()

2025-10-28 08:04:19 -04:00

nn

refactor: Keep explicit module imports + optimize CNN milestone

2025-09-30 17:15:40 -04:00

optim

Organize package with nn and optim modules

2025-09-23 08:10:47 -04:00

optimization

Reset package and export modules 01-07 only (skip broken spatial module)

2025-09-30 13:41:00 -04:00

profiler

FOUNDATION: Establish AI Engineering as a discipline through TinyTorch

2025-09-25 11:16:28 -04:00

profiling

Reset package and export modules 01-07 only (skip broken spatial module)

2025-09-30 13:41:00 -04:00

quantization

FOUNDATION: Establish AI Engineering as a discipline through TinyTorch

2025-09-25 11:16:28 -04:00

text

fix(autograd): Add EmbeddingBackward and ReshapeBackward

2025-10-28 07:56:20 -04:00

utils

Clean up module imports: convert tinytorch.core to sys.path style

2025-09-30 08:58:58 -04:00

__init__.py

Fix gradient propagation: enable autograd and patch activations/losses

2025-09-30 13:51:30 -04:00

_modidx.py

🤖 Fix transformer module exports and milestone 05 imports

2025-10-27 16:17:55 -04:00