The GELU implementation is missing for the core concepts section.
Proposed Solution
Include this code as implementation along with necessary GELU supporting info. Logically this section seems to belong after #sigmoid-and-tanh and before #softmax-and-numerical-stability.
classGELU:defforward(self,x:Tensor)->Tensor:"""Apply GELU activation using the fast sigmoid approximation."""returnSigmoid()(x*1.702)*x
Originally created by @bdub-1 on GitHub (May 6, 2026).
Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1689
### Module
02 Activations
### Type of Improvement
Better explanation or documentation
### Description
The GELU implementation is missing for the [core concepts](https://mlsysbook.ai/tinytorch/modules/02_activations.html#core-concepts) section.
### Proposed Solution
Include this code as implementation along with necessary GELU supporting info. Logically this section seems to belong after `#sigmoid-and-tanh` and before `#softmax-and-numerical-stability`.
```py
class GELU:
def forward(self, x: Tensor) -> Tensor:
"""Apply GELU activation using the fast sigmoid approximation."""
return Sigmoid()(x * 1.702) * x
```
@Shashank-Tripathi-07 commented on GitHub (May 6, 2026):
Thanks !
I'll make a PR on this in a few hours and submit it.
<!-- gh-comment-id:4389435550 -->
@Shashank-Tripathi-07 commented on GitHub (May 6, 2026):
Thanks !
I'll make a PR on this in a few hours and submit it.
@Shashank-Tripathi-07 commented on GitHub (May 6, 2026):
Saw it, done correctly. Very good !!
PR will be merged by @profvjreddi in his own time 🤝🏻
<!-- gh-comment-id:4389614928 -->
@Shashank-Tripathi-07 commented on GitHub (May 6, 2026):
Saw it, done correctly. Very good !!
PR will be merged by @profvjreddi in his own time 🤝🏻
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @bdub-1 on GitHub (May 6, 2026).
Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1689
Module
02 Activations
Type of Improvement
Better explanation or documentation
Description
The GELU implementation is missing for the core concepts section.
Proposed Solution
Include this code as implementation along with necessary GELU supporting info. Logically this section seems to belong after
#sigmoid-and-tanhand before#softmax-and-numerical-stability.@Shashank-Tripathi-07 commented on GitHub (May 6, 2026):
Thanks !
I'll make a PR on this in a few hours and submit it.
@bdub-1 commented on GitHub (May 6, 2026):
Please see the linked PR!
@Shashank-Tripathi-07 commented on GitHub (May 6, 2026):
Saw it, done correctly. Very good !!
PR will be merged by @profvjreddi in his own time 🤝🏻