github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-07-16 23:24:55 -05:00
Code Issues 276 Packages Projects Releases 26 Wiki Activity

[GH-ISSUE #1689] [TinyTorch] GELU implementation missing from module 2 docs #19815

New Issue
Closed
opened 2026-06-01 18:22:44 -05:00 by GiteaMirror · 3 comments
GiteaMirror commented 2026-06-01 18:22:44 -05:00
Owner
Copy Link

Originally created by @bdub-1 on GitHub (May 6, 2026).
Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1689

Module

02 Activations

Type of Improvement

Better explanation or documentation

Description

The GELU implementation is missing for the core concepts section.

Proposed Solution

Include this code as implementation along with necessary GELU supporting info. Logically this section seems to belong after #sigmoid-and-tanh and before #softmax-and-numerical-stability.

class GELU:
    def forward(self, x: Tensor) -> Tensor:
        """Apply GELU activation using the fast sigmoid approximation."""
        return Sigmoid()(x * 1.702) * x
Originally created by @bdub-1 on GitHub (May 6, 2026). Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1689 ### Module 02 Activations ### Type of Improvement Better explanation or documentation ### Description The GELU implementation is missing for the [core concepts](https://mlsysbook.ai/tinytorch/modules/02_activations.html#core-concepts) section. ### Proposed Solution Include this code as implementation along with necessary GELU supporting info. Logically this section seems to belong after `#sigmoid-and-tanh` and before `#softmax-and-numerical-stability`. ```py class GELU: def forward(self, x: Tensor) -> Tensor: """Apply GELU activation using the fast sigmoid approximation.""" return Sigmoid()(x * 1.702) * x ```
GiteaMirror added the area: booktype: bugtype: improvementarea: tinytorch labels 2026-06-01 18:22:44 -05:00
GiteaMirror commented 2026-06-01 18:22:47 -05:00
Author
Owner
Copy Link

@Shashank-Tripathi-07 commented on GitHub (May 6, 2026):

Thanks !

I'll make a PR on this in a few hours and submit it.

<!-- gh-comment-id:4389435550 --> @Shashank-Tripathi-07 commented on GitHub (May 6, 2026): Thanks ! I'll make a PR on this in a few hours and submit it.
GiteaMirror commented 2026-06-01 18:22:48 -05:00
Author
Owner
Copy Link

@bdub-1 commented on GitHub (May 6, 2026):

Please see the linked PR!

<!-- gh-comment-id:4389518791 --> @bdub-1 commented on GitHub (May 6, 2026): Please see the linked PR!
GiteaMirror commented 2026-06-01 18:22:51 -05:00
Author
Owner
Copy Link

@Shashank-Tripathi-07 commented on GitHub (May 6, 2026):

Saw it, done correctly. Very good !!

PR will be merged by @profvjreddi in his own time 🤝🏻

<!-- gh-comment-id:4389614928 --> @Shashank-Tripathi-07 commented on GitHub (May 6, 2026): Saw it, done correctly. Very good !! PR will be merged by @profvjreddi in his own time 🤝🏻
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area: book
area: collabs
area: kits
area: labs
area: socratiq
area: staffml
area: tinytorch
area: tools
area: website
bug
dependencies
format: epub
format: pdf
javascript
link-health
link-rot
priority-high
pull-request
Mirrored from GitHub Pull Request
 staffml
type: bug
type: citation
type: code
type: errata
type: improvement
type: new
type: question
vault-sli
No Label area: book area: tinytorch type: bug type: improvement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#19815
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?