github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-05-11 08:59:07 -05:00
Code Issues 81 Packages Projects Releases 24 Wiki Activity

[PR #1156] [MERGED] fix(activations): correct GELU hint about 1.702 constant #1146

New Issue
Closed
opened 2026-03-22 16:01:58 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-03-22 16:01:58 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/harvard-edge/cs249r_book/pull/1156
Author: @profvjreddi
Created: 2/4/2026
Status: ✅ Merged
Merged: 2/4/2026
Merged by: @profvjreddi

Base: dev ← Head: feature/tinytorch-core

📝 Commits (1)

  • c3a9230 fix(activations): correct misleading GELU hint about 1.702 constant

📊 Changes

1 file changed (+1 additions, -1 deletions)

View changed files

📝 tinytorch/src/02_activations/02_activations.py (+1 -1)

📄 Description

Summary

  • Fixes misleading hint in Module 2 GELU implementation that claimed the 1.702 constant comes from √(2/π) ≈ 0.798
  • The 1.702 is actually an empirically fitted constant so that sigmoid(1.702x) ≈ Φ(x) (the Gaussian CDF)
  • The √(2/π) constant appears in the separate tanh-based GELU approximation, not the sigmoid approximation used here

Fixes #1154

Test plan

  • Verified the hint now accurately describes the origin of the 1.702 constant
  • Confirmed no other references to this incorrect claim exist in the module

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/harvard-edge/cs249r_book/pull/1156 **Author:** [@profvjreddi](https://github.com/profvjreddi) **Created:** 2/4/2026 **Status:** ✅ Merged **Merged:** 2/4/2026 **Merged by:** [@profvjreddi](https://github.com/profvjreddi) **Base:** `dev` ← **Head:** `feature/tinytorch-core` --- ### 📝 Commits (1) - [`c3a9230`](https://github.com/harvard-edge/cs249r_book/commit/c3a9230ea87c9a42454606b1dfee12c0c3b6b53f) fix(activations): correct misleading GELU hint about 1.702 constant ### 📊 Changes **1 file changed** (+1 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `tinytorch/src/02_activations/02_activations.py` (+1 -1) </details> ### 📄 Description ## Summary - Fixes misleading hint in Module 2 GELU implementation that claimed the 1.702 constant comes from √(2/π) ≈ 0.798 - The 1.702 is actually an empirically fitted constant so that `sigmoid(1.702x) ≈ Φ(x)` (the Gaussian CDF) - The √(2/π) constant appears in the separate **tanh-based** GELU approximation, not the sigmoid approximation used here Fixes #1154 ## Test plan - [x] Verified the hint now accurately describes the origin of the 1.702 constant - [x] Confirmed no other references to this incorrect claim exist in the module --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-03-22 16:01:58 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area: book
area: collabs
area: kits
area: socratiq
area: tinytorch
area: tools
area: website
bug
dependencies
format: epub
format: pdf
link-health
link-rot
priority-high
pull-request
Mirrored from GitHub Pull Request
 staffml
type: bug
type: citation
type: code
type: errata
type: improvement
type: new
type: question
vault-sli
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#1146
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?