[PR #1343] [MERGED] fix(tinytorch): wire Tanh into enable_autograd() #8132

New Issue

GiteaMirror · 2026-04-27T17:26:11-05:00

GiteaMirror commented

2026-04-27 17:26:11 -05:00

📋 Pull Request Information

Original PR: https://github.com/harvard-edge/cs249r_book/pull/1343
Author: @profvjreddi
Created: 4/16/2026
Status: ✅ Merged
Merged: 4/16/2026
Merged by: @profvjreddi

Base: dev ← Head: fix/tanh-autograd-wiring

📝 Commits (1)

242093e fix(tinytorch): wire Tanh into enable_autograd()

📊 Changes

2 files changed (+94 additions, -1 deletions)

View changed files

📝 tinytorch/src/06_autograd/06_autograd.py (+73 -1)
📝 tinytorch/tests/06_autograd/test_06_autograd_progressive.py (+21 -0)

📄 Description

closes #1341

what

Tanh was the only activation not patched by enable_autograd(). calling Tanh()(x).backward() silently left x.grad as None because the forward never attached a computation graph.

changes

follows the existing tracked_sigmoid_forward pattern precisely (same "save output for backward" shape since d/dx tanh(x) = 1 - tanh(x)^2 can be derived from the forward result):

change	location
new `TanhBackward(Function)` class	`src/06_autograd/06_autograd.py` (between SigmoidBackward and SoftmaxBackward)
new `tracked_tanh_forward` wrapper	inside `enable_autograd()`
Tanh added to activations import + patch install	inside `enable_autograd()`
regression test	`tests/06_autograd/test_06_autograd_progressive.py`

test

the new test_tanh_activation_gradient asserts:

x.grad is not None after calling tanh(x).sum().backward()
values match the analytical derivative at 0 and ±1 within rtol=1e-5

context

came out of #1336 where @Shashank-Tripathi-07's finite-diff tests exposed the bypass. see #1341 for the diagnosis and #1342 for the followup backward-math audit across other ops.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/harvard-edge/cs249r_book/pull/1343 **Author:** [@profvjreddi](https://github.com/profvjreddi) **Created:** 4/16/2026 **Status:** ✅ Merged **Merged:** 4/16/2026 **Merged by:** [@profvjreddi](https://github.com/profvjreddi) **Base:** `dev` ← **Head:** `fix/tanh-autograd-wiring` --- ### 📝 Commits (1) - [`242093e`](https://github.com/harvard-edge/cs249r_book/commit/242093e407a1fe047ac846c154a06ef55299e315) fix(tinytorch): wire Tanh into enable_autograd() ### 📊 Changes **2 files changed** (+94 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `tinytorch/src/06_autograd/06_autograd.py` (+73 -1) 📝 `tinytorch/tests/06_autograd/test_06_autograd_progressive.py` (+21 -0) </details> ### 📄 Description closes #1341 ## what Tanh was the only activation not patched by `enable_autograd()`. calling `Tanh()(x).backward()` silently left `x.grad` as None because the forward never attached a computation graph. ## changes follows the existing `tracked_sigmoid_forward` pattern precisely (same "save output for backward" shape since `d/dx tanh(x) = 1 - tanh(x)^2` can be derived from the forward result): | change | location | |---|---| | new `TanhBackward(Function)` class | `src/06_autograd/06_autograd.py` (between SigmoidBackward and SoftmaxBackward) | | new `tracked_tanh_forward` wrapper | inside `enable_autograd()` | | Tanh added to activations import + patch install | inside `enable_autograd()` | | regression test | `tests/06_autograd/test_06_autograd_progressive.py` | ## test the new `test_tanh_activation_gradient` asserts: - `x.grad is not None` after calling `tanh(x).sum().backward()` - values match the analytical derivative at 0 and ±1 within rtol=1e-5 ## context came out of #1336 where @Shashank-Tripathi-07's finite-diff tests exposed the bypass. see #1341 for the diagnosis and #1342 for the followup backward-math audit across other ops. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-04-27 17:26:12 -05:00

GiteaMirror closed this issue

2026-04-27 17:26:16 -05:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#8132