[GH-ISSUE #1161] [Module 3] Typo on Xavier/Glorot definitions throughout module #5673

New Issue

GiteaMirror · 2026-04-21T21:40:36-05:00

GiteaMirror commented

2026-04-21 21:40:36 -05:00

Originally created by @oscarf189 on GitHub (Feb 5, 2026).
Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1161

1a80e57fa0/tinytorch/src/03_layers/03_layers.py (L138)

Minor typo, unclear if solution is to rename or update formula.

There are multiple references to Xavier/Glorot having a sigma of \sqrt(1/fan_in). However, this corresponds to LeCun --> Equation 15 in "Efficient BackProp". The Xavier/Glorot is sqrt(2/(fan_in+fan_out)), this one is defined in Equation 12 in "Understanding the difficulty of training deep feedforward neural networks"

Originally created by @oscarf189 on GitHub (Feb 5, 2026). Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1161 https://github.com/harvard-edge/cs249r_book/blob/1a80e57fa07a5e551641df8db4dd2131524cb31f/tinytorch/src/03_layers/03_layers.py#L138 Minor typo, unclear if solution is to rename or update formula. There are multiple references to Xavier/Glorot having a sigma of `\sqrt(1/fan_in)`. However, this corresponds to LeCun --> Equation 15 in ["Efficient BackProp"](http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf). The Xavier/Glorot is `sqrt(2/(fan_in+fan_out))`, this one is defined in Equation 12 in ["Understanding the difficulty of training deep feedforward neural networks"](https://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf)

GiteaMirror added the type: errata area: tinytorch labels 2026-04-21 21:40:36 -05:00

GiteaMirror closed this issue

2026-04-21 21:40:37 -05:00

GiteaMirror commented

2026-04-21 21:40:40 -05:00

@profvjreddi commented on GitHub (Feb 6, 2026):

Hi @oscarf189! Thanks for catching this 🙏 You're right on! The formula sqrt(1/fan_in) is LeCun initialization. I've updated the terminology in c1c8c11ee good sir! Thank you again, love it that you folks are catching all the mistakes 🤗 Thank you!

@all-contributors please add @oscarf189 as a contributor for ✍️ Doc in TinyTorch

@profvjreddi commented on GitHub (Feb 6, 2026): Hi @oscarf189! Thanks for catching this 🙏 You're right on! The formula sqrt(1/fan_in) is LeCun initialization. I've updated the terminology in c1c8c11ee good sir! Thank you again, love it that you folks are catching all the mistakes 🤗 Thank you! @all-contributors please add @oscarf189 as a contributor for ✍️ Doc in TinyTorch

GiteaMirror commented

2026-04-21 21:40:42 -05:00

@github-actions[bot] commented on GitHub (Feb 6, 2026):

I've added @oscarf189 as a contributor to tinytorch! 🎉

Recognized for: doc
Project: tinytorch (explicitly mentioned in comment)
Based on: @all-contributors please add @oscarf189 as a contributor for ✍️ Doc in TinyTorch

The contributor list has been updated in:

tinytorch/.all-contributorsrc
tinytorch/README.md
Main README.md

We love recognizing our contributors! ❤️

@github-actions[bot] commented on GitHub (Feb 6, 2026): I've added @oscarf189 as a contributor to **tinytorch**! :tada: **Recognized for:** doc **Project:** tinytorch (explicitly mentioned in comment) **Based on:** @all-contributors please add @oscarf189 as a contributor for ✍️ Doc in TinyTorch The contributor list has been updated in: - `tinytorch/.all-contributorsrc` - `tinytorch/README.md` - Main `README.md` We love recognizing our contributors! :heart:

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#5673