github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-05-21 13:31:55 -05:00
Code Issues 114 Packages Projects Releases 24 Wiki Activity

[PR #1781] fix(tinytorch): correct MatmulBackward gradients for 1D vector inputs #15732

New Issue
Open
opened 2026-05-20 14:04:53 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-20 14:04:53 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/harvard-edge/cs249r_book/pull/1781
Author: @Shashank-Tripathi-07
Created: 5/18/2026
Status: 🔄 Open

Base: dev ← Head: fix/tinytorch-audit-20260518

📝 Commits (1)

  • b4fae46 fix(tinytorch): correct MatmulBackward gradients for 1D vector inputs

📊 Changes

1 file changed (+10 additions, -6 deletions)

View changed files

📝 tinytorch/src/06_autograd/06_autograd.py (+10 -6)

📄 Description

Summary

  • Fixes a correctness bug in MatmulBackward.apply() in tinytorch/src/06_autograd/06_autograd.py where gradient computation was wrong when input B is a 1D vector.

Bug Details

Root cause: When B is 1D with shape (k,), NumPy's .T is a no-op (1D arrays have no second axis to swap). The old code then called np.matmul(grad_output, b.data) which either raises a shape error (when m != k) or silently computes a dot product returning a scalar (when m == k) instead of the correct (m, k) gradient matrix.

Correct math: For A(m,k) @ b(k,) -> out(m,):

  • grad_A = np.outer(grad_output, b) -- shape (m, k) (was broken)
  • grad_b = a.T @ grad_output -- shape (k,) (was broken for 1D a)

The symmetric case (1D a with 2D B) is also fixed using np.outer(a, grad_output).

The 2D+ batched path using np.swapaxes is preserved unchanged and correct.

Test plan

  • Verify MatmulBackward with A(m,k) @ b(k,) produces grad_A of shape (m, k) and grad_b of shape (k,)
  • Verify existing 2D matmul test (A(2,2) @ B(2,2)) still passes
  • Verify batched matmul (3D inputs) still passes
  • Run tinytorch test suite: pytest tinytorch/tests/

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/harvard-edge/cs249r_book/pull/1781 **Author:** [@Shashank-Tripathi-07](https://github.com/Shashank-Tripathi-07) **Created:** 5/18/2026 **Status:** 🔄 Open **Base:** `dev` ← **Head:** `fix/tinytorch-audit-20260518` --- ### 📝 Commits (1) - [`b4fae46`](https://github.com/harvard-edge/cs249r_book/commit/b4fae46e183a981384d63458370c2246a7d805fe) fix(tinytorch): correct MatmulBackward gradients for 1D vector inputs ### 📊 Changes **1 file changed** (+10 additions, -6 deletions) <details> <summary>View changed files</summary> 📝 `tinytorch/src/06_autograd/06_autograd.py` (+10 -6) </details> ### 📄 Description ## Summary - Fixes a correctness bug in `MatmulBackward.apply()` in `tinytorch/src/06_autograd/06_autograd.py` where gradient computation was wrong when input `B` is a 1D vector. ## Bug Details **Root cause**: When `B` is 1D with shape `(k,)`, NumPy's `.T` is a no-op (1D arrays have no second axis to swap). The old code then called `np.matmul(grad_output, b.data)` which either raises a shape error (when `m != k`) or silently computes a dot product returning a scalar (when `m == k`) instead of the correct `(m, k)` gradient matrix. **Correct math**: For `A(m,k) @ b(k,)` -> `out(m,)`: - `grad_A = np.outer(grad_output, b)` -- shape `(m, k)` (was broken) - `grad_b = a.T @ grad_output` -- shape `(k,)` (was broken for 1D `a`) The symmetric case (1D `a` with 2D `B`) is also fixed using `np.outer(a, grad_output)`. The 2D+ batched path using `np.swapaxes` is preserved unchanged and correct. ## Test plan - [ ] Verify `MatmulBackward` with `A(m,k) @ b(k,)` produces `grad_A` of shape `(m, k)` and `grad_b` of shape `(k,)` - [ ] Verify existing 2D matmul test (`A(2,2) @ B(2,2)`) still passes - [ ] Verify batched matmul (3D inputs) still passes - [ ] Run `tinytorch` test suite: `pytest tinytorch/tests/` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-05-20 14:04:53 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area: book
area: collabs
area: kits
area: labs
area: socratiq
area: tinytorch
area: tools
area: website
bug
dependencies
format: epub
format: pdf
javascript
link-health
link-rot
priority-high
pull-request
Mirrored from GitHub Pull Request
 staffml
type: bug
type: citation
type: code
type: errata
type: improvement
type: new
type: question
vault-sli
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#15732
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?