mirror of
https://github.com/harvard-edge/cs249r_book.git
synced 2026-07-16 23:24:55 -05:00
[GH-ISSUE #1925] [TinyTorch] #35285
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/cs249r_book#35285
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Cohegen on GitHub (Jul 2, 2026).
Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1925
Module
07 Optimizers
Type of Improvement
New example or demonstration
Description
Add Muon optimizer
Proposed Solution
I would like to suggest adding Muon optimizer. Muon has shown promising results for training neural networks and could provide tinytorch users and learners with an additional optimization option alongside the currently supported optimizers.
@emailtosalvi commented on GitHub (Jul 2, 2026):
...i disagree, because of my lack in knowledge.
@nijinomichi commented on GitHub (Jul 3, 2026):
Interesting proposal.
Optimizers improve how models learn from gradients.
I'm curious whether there is a complementary layer above optimization itself:
how humans and AI learn to collaborate over long periods through trust, reflection, and dialogue.
Perhaps future AI systems will need not only better optimization algorithms,
but also better interaction frameworks that help sustain meaningful co-creation.
These seem like different layers of the same ecosystem.
@profvjreddi commented on GitHub (Jul 3, 2026):
Thanks for the suggestion @Cohegen, and Muon is genuinely interesting. The Newton-Schulz orthogonalization idea is clever and it has been showing strong results, so I appreciate you bringing it up.
For now we want to keep module 07 focused on the classic ladder of SGD, momentum, and Adam. Those are the optimizers a learner has to understand first, they are everywhere in production, and the mechanisms they teach are stable enough that they will still matter years from now. TinyTorch's job in these core modules is to build that durable foundation from scratch, so I try to be deliberate about what earns a slot.
Muon is newer and still proving itself, and it is not in the mainstream frameworks yet, so I would rather let it settle before it goes into a foundations course. That said, this is exactly the kind of modern research worth revisiting later, most likely as an optional advanced example that teaches the orthogonalization mechanism rather than as another core optimizer. Please keep suggestions like this coming.