[GH-ISSUE #1341] tanh missing from enable_autograd() wiring #5720

New Issue

GiteaMirror · 2026-04-21T21:45:12-05:00

GiteaMirror commented

2026-04-21 21:45:12 -05:00

Originally created by @profvjreddi on GitHub (Apr 16, 2026).
Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1341

just noticed tanh doesnt track gradients. if i do Tanh()(x) with x.requires_grad=True, x.grad is None after backward().

looked at enable_autograd() in src/06_autograd/06_autograd.py ... sigmoid, relu, gelu, softmax, mse, bce and crossentropy are all in the patch list but tanh is missing. also no TanhBackward class anywhere.

came up while looking at #1336.

fix is pretty mechanical:

add TanhBackward following the same shape as ReLUBackward. math is d/dx tanh(x) = 1 - tanh(x)^2
wrap Tanh.forward in enable_autograd() same way as tracked_sigmoid_forward
add Tanh to the import line in the try block next to the other activations

Originally created by @profvjreddi on GitHub (Apr 16, 2026). Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1341 just noticed tanh doesnt track gradients. if i do Tanh()(x) with x.requires_grad=True, x.grad is None after backward(). looked at enable_autograd() in src/06_autograd/06_autograd.py ... sigmoid, relu, gelu, softmax, mse, bce and crossentropy are all in the patch list but tanh is missing. also no TanhBackward class anywhere. came up while looking at #1336. fix is pretty mechanical: - add TanhBackward following the same shape as ReLUBackward. math is d/dx tanh(x) = 1 - tanh(x)^2 - wrap Tanh.forward in enable_autograd() same way as tracked_sigmoid_forward - add Tanh to the import line in the try block next to the other activations

GiteaMirror added the type: bug area: book area: tinytorch labels 2026-04-21 21:45:12 -05:00

GiteaMirror closed this issue

2026-04-21 21:45:14 -05:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#5720