github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-05-07 10:08:50 -05:00
Code Issues 85 Packages Projects Releases 24 Wiki Activity

[GH-ISSUE #1180] Module 18 - Memoization #5684

New Issue
Closed
opened 2026-04-21 21:41:50 -05:00 by GiteaMirror · 6 comments
GiteaMirror commented 2026-04-21 21:41:50 -05:00
Owner
Copy Link

Originally created by @ngbolin on GitHub (Feb 19, 2026).
Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1180

Hello,

When computing the number of redundant operations relative to caching for 100 tokens, there is an additional 4,950 computations (instead of 5050). This is because under caching, one will still need to compute the K and V computations for each token.

Image
Originally created by @ngbolin on GitHub (Feb 19, 2026). Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1180 Hello, When computing the number of redundant operations relative to caching for 100 tokens, there is an additional 4,950 computations (instead of 5050). This is because under caching, one will still need to compute the K and V computations for each token. <img width="562" height="243" alt="Image" src="https://github.com/user-attachments/assets/934657c7-eb0f-4abf-878a-b746156f1dea" />
GiteaMirror added the area: tinytorchtype: improvement labels 2026-04-21 21:41:50 -05:00
GiteaMirror commented 2026-04-21 21:41:55 -05:00
Author
Owner
Copy Link

@profvjreddi commented on GitHub (Feb 19, 2026):

Thanks for catching this, @ngbolin! You're absolutely right.

The sum 1 + 2 + ... + 100 = 5,050 is the total number of K,V computations in the naive approach, but not all of them are redundant — each token still requires its own K,V computation even with caching (100 necessary computations). So the correct number of redundant computations is 5,050 − 100 = 4,950.

This has been fixed in both the module source and the paper to clarify the distinction:

  • "5,050 total K,V computations, of which 4,950 are redundant"

Thanks for the careful reading! @all-contributors please add @ngbolin as a contributor for ✍️ Doc in TinyTorch

<!-- gh-comment-id:3927612800 --> @profvjreddi commented on GitHub (Feb 19, 2026): Thanks for catching this, @ngbolin! You're absolutely right. The sum 1 + 2 + ... + 100 = 5,050 is the **total** number of K,V computations in the naive approach, but not all of them are redundant — each token still requires its own K,V computation even with caching (100 necessary computations). So the correct number of **redundant** computations is 5,050 − 100 = **4,950**. This has been fixed in both the module source and the paper to clarify the distinction: - "5,050 total K,V computations, of which 4,950 are redundant" Thanks for the careful reading! @all-contributors please add @ngbolin as a contributor for ✍️ Doc in TinyTorch
GiteaMirror commented 2026-04-21 21:41:56 -05:00
Author
Owner
Copy Link

@github-actions[bot] commented on GitHub (Feb 19, 2026):

I've added @ngbolin as a contributor to tinytorch! 🎉

Recognized for: doc
Project: tinytorch (explicitly mentioned in comment)
Based on: Thanks for the careful reading! @all-contributors please add @ngbolin as a contributor for ✍️ Doc in TinyTorch

The contributor list has been updated in:

  • tinytorch/.all-contributorsrc
  • tinytorch/README.md
  • Main README.md

We love recognizing our contributors! ❤️

<!-- gh-comment-id:3927635971 --> @github-actions[bot] commented on GitHub (Feb 19, 2026): I've added @ngbolin as a contributor to **tinytorch**! :tada: **Recognized for:** doc **Project:** tinytorch (explicitly mentioned in comment) **Based on:** Thanks for the careful reading! @all-contributors please add @ngbolin as a contributor for ✍️ Doc in TinyTorch The contributor list has been updated in: - `tinytorch/.all-contributorsrc` - `tinytorch/README.md` - Main `README.md` We love recognizing our contributors! :heart:
GiteaMirror commented 2026-04-21 21:41:58 -05:00
Author
Owner
Copy Link

@profvjreddi commented on GitHub (Feb 19, 2026):

Hi @ngbolin, I wanted to reach out because you've been one of our most dedicated contributors, working through the modules from Autograd (#1131) all the way up to Memoization here. That kind of systematic engagement is exactly what we hoped TinyTorch would inspire.

We're having a TinyTorch community meetup on Tuesday, March 3rd at 9am and would love to have you share a few minutes about your experience. What drew you to TinyTorch, what you've learned working through the modules, anything you'd like to share. No pressure on format, just a casual conversation.

As one of our first active users, we'd also like to send you a small thank you. Keep an eye on our Discord for more details!

<!-- gh-comment-id:3929785140 --> @profvjreddi commented on GitHub (Feb 19, 2026): Hi @ngbolin, I wanted to reach out because you've been one of our most dedicated contributors, working through the modules from Autograd (#1131) all the way up to Memoization here. That kind of systematic engagement is exactly what we hoped TinyTorch would inspire. We're having a TinyTorch community meetup on **Tuesday, March 3rd at 9am** and would love to have you share a few minutes about your experience. What drew you to TinyTorch, what you've learned working through the modules, anything you'd like to share. No pressure on format, just a casual conversation. As one of our first active users, we'd also like to send you a small thank you. Keep an eye on our Discord for more details!
GiteaMirror commented 2026-04-21 21:41:59 -05:00
Author
Owner
Copy Link

@ngbolin commented on GitHub (Feb 20, 2026):

Hi Prof Reddi,

Thank you so much for the kind message, I would love to share my experiences while working through the modules, and what I've learnt. See you on 3rd March, 9am!

<!-- gh-comment-id:3931980283 --> @ngbolin commented on GitHub (Feb 20, 2026): Hi Prof Reddi, Thank you so much for the kind message, I would love to share my experiences while working through the modules, and what I've learnt. See you on 3rd March, 9am!
GiteaMirror commented 2026-04-21 21:42:00 -05:00
Author
Owner
Copy Link

@ngbolin commented on GitHub (Feb 20, 2026):

Sorry, I couldn't use the discord invite from your earlier email - it appears that the invite has expired. Will you or your team be able to send another discord invite, please? Thank you!

My email is ngbolin91@gmail.com.

<!-- gh-comment-id:3932483162 --> @ngbolin commented on GitHub (Feb 20, 2026): Sorry, I couldn't use the discord invite from your earlier email - it appears that the invite has expired. Will you or your team be able to send another discord invite, please? Thank you! My email is ngbolin91@gmail.com.
GiteaMirror commented 2026-04-21 21:42:01 -05:00
Author
Owner
Copy Link

@profvjreddi commented on GitHub (Feb 24, 2026):

Hi @ngbolin,

Thanks for your interest in the TinyTorch community meetup on Tuesday, March 3rd at 9am. To get you set up as a speaker and share logistics, we need a few details from you.

Please:

  1. Reply to: karijanapareddi@gmail.com
    CC: vj@eecs.harvard.edu and andreamattia.garavagno@santannapisa.it

  2. Use the email address you want us to use for Zoom registration and for promoting you to panelist so you can speak on the call (it's a webinar-style meeting).

  3. Slide We'd love it if you could share a slide or two about your experience, you can add them here—there's a template to follow:
    https://docs.google.com/presentation/d/1h0bQZJVtmXYuO4jgakE3x3xGt0e8fNsvW7QfWK1_5D4/edit?slide=id.g3cac3cc65ae_0_19#slide=id.g3cac3cc65ae_0_19

We'll send logistical details (Zoom link, agenda, etc.) to your registered email.

Looking forward to having you on the call.

<!-- gh-comment-id:3952968928 --> @profvjreddi commented on GitHub (Feb 24, 2026): Hi @ngbolin, Thanks for your interest in the TinyTorch community meetup on **Tuesday, March 3rd at 9am**. To get you set up as a speaker and share logistics, we need a few details from you. **Please:** 1. **Reply to:** **karijanapareddi@gmail.com** **CC:** vj@eecs.harvard.edu and andreamattia.garavagno@santannapisa.it 2. **Use the email address** you want us to use for Zoom registration and for promoting you to **panelist** so you can speak on the call (it's a webinar-style meeting). 3. **Slide** We'd love it if you could share a slide or two about your experience, you can add them here—there's a template to follow: https://docs.google.com/presentation/d/1h0bQZJVtmXYuO4jgakE3x3xGt0e8fNsvW7QfWK1_5D4/edit?slide=id.g3cac3cc65ae_0_19#slide=id.g3cac3cc65ae_0_19 We'll send logistical details (Zoom link, agenda, etc.) to your registered email. Looking forward to having you on the call.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area: book
area: collabs
area: kits
area: socratiq
area: tinytorch
area: tools
area: website
bug
dependencies
format: epub
format: pdf
link-health
link-rot
priority-high
pull-request
Mirrored from GitHub Pull Request
 staffml
type: bug
type: citation
type: code
type: errata
type: improvement
type: new
type: question
vault-sli
No Label area: tinytorch type: improvement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#5684
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?