github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-05-21 13:31:55 -05:00
Code Issues 114 Packages Projects Releases 24 Wiki Activity

[PR #1785] fix(tinytorch): fix GPT causal mask convention in module 13 #15736

New Issue
Open
opened 2026-05-20 14:05:03 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-20 14:05:03 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/harvard-edge/cs249r_book/pull/1785
Author: @Shashank-Tripathi-07
Created: 5/18/2026
Status: 🔄 Open

Base: dev ← Head: fix/tinytorch-modules-10-20-audit

📝 Commits (2)

  • fbb3f3d fix(site): add nav-footer and dropdown-menu dark mode selectors
  • b9ce5b2 fix(tinytorch): fix GPT causal mask convention in module 13

📊 Changes

3 files changed (+41 additions, -6 deletions)

View changed files

📝 shared/config/footer-site.yml (+1 -1)
📝 shared/styles/_site-dark.scss (+37 -3)
📝 tinytorch/src/13_transformers/13_transformers.py (+3 -2)

📄 Description

Summary

  • GPT._create_causal_mask in 13_transformers.py was generating an additive -inf mask (upper-triangular with -inf, 0 for allowed positions), but _apply_mask in 12_attention.py expects a binary lower-triangular mask (1=allow, 0=block).
  • With the old mask, every allowed position (value 0) received (1-0)*MASK_VALUE = -1e9 from _apply_mask, blocking all attention including valid past/present positions -- only future positions coincidentally received a correct -inf via (1-(-inf))*(-1e9).
  • Fixed by switching to np.tril(np.ones((seq_len, seq_len))) which matches the binary convention _apply_mask requires.

Test plan

  • Run test_unit_transformer_block in 13_transformers.py -- shape checks pass
  • Manually verify that GPT.generate() produces non-uniform attention over past tokens (all tokens no longer receive near-zero weight due to incorrect masking)
  • Run test_unit_scaled_dot_product_attention in 12_attention.py to confirm the mask format contract is respected end-to-end

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/harvard-edge/cs249r_book/pull/1785 **Author:** [@Shashank-Tripathi-07](https://github.com/Shashank-Tripathi-07) **Created:** 5/18/2026 **Status:** 🔄 Open **Base:** `dev` ← **Head:** `fix/tinytorch-modules-10-20-audit` --- ### 📝 Commits (2) - [`fbb3f3d`](https://github.com/harvard-edge/cs249r_book/commit/fbb3f3d1638fd7b123d885c9cbcd1cbd7e8257cc) fix(site): add nav-footer and dropdown-menu dark mode selectors - [`b9ce5b2`](https://github.com/harvard-edge/cs249r_book/commit/b9ce5b25224dbf81ac442fcbf503a742a0783efd) fix(tinytorch): fix GPT causal mask convention in module 13 ### 📊 Changes **3 files changed** (+41 additions, -6 deletions) <details> <summary>View changed files</summary> 📝 `shared/config/footer-site.yml` (+1 -1) 📝 `shared/styles/_site-dark.scss` (+37 -3) 📝 `tinytorch/src/13_transformers/13_transformers.py` (+3 -2) </details> ### 📄 Description ## Summary - `GPT._create_causal_mask` in `13_transformers.py` was generating an additive -inf mask (upper-triangular with -inf, 0 for allowed positions), but `_apply_mask` in `12_attention.py` expects a binary lower-triangular mask (1=allow, 0=block). - With the old mask, every allowed position (value 0) received `(1-0)*MASK_VALUE = -1e9` from `_apply_mask`, blocking all attention including valid past/present positions -- only future positions coincidentally received a correct -inf via `(1-(-inf))*(-1e9)`. - Fixed by switching to `np.tril(np.ones((seq_len, seq_len)))` which matches the binary convention `_apply_mask` requires. ## Test plan - [x] Run `test_unit_transformer_block` in `13_transformers.py` -- shape checks pass - [x] Manually verify that `GPT.generate()` produces non-uniform attention over past tokens (all tokens no longer receive near-zero weight due to incorrect masking) - [x] Run `test_unit_scaled_dot_product_attention` in `12_attention.py` to confirm the mask format contract is respected end-to-end --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-05-20 14:05:03 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area: book
area: collabs
area: kits
area: labs
area: socratiq
area: tinytorch
area: tools
area: website
bug
dependencies
format: epub
format: pdf
javascript
link-health
link-rot
priority-high
pull-request
Mirrored from GitHub Pull Request
 staffml
type: bug
type: citation
type: code
type: errata
type: improvement
type: new
type: question
vault-sli
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#15736
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?