[GH-ISSUE #1180] Module 18 - Memoization #4342

New Issue

GiteaMirror · 2026-04-19T12:21:21-05:00

GiteaMirror commented

2026-04-19 12:21:21 -05:00

Originally created by @ngbolin on GitHub (Feb 19, 2026).
Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1180

Hello,

When computing the number of redundant operations relative to caching for 100 tokens, there is an additional 4,950 computations (instead of 5050). This is because under caching, one will still need to compute the K and V computations for each token.

Originally created by @ngbolin on GitHub (Feb 19, 2026). Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1180 Hello, When computing the number of redundant operations relative to caching for 100 tokens, there is an additional 4,950 computations (instead of 5050). This is because under caching, one will still need to compute the K and V computations for each token. <img width="562" height="243" alt="Image" src="https://github.com/user-attachments/assets/934657c7-eb0f-4abf-878a-b746156f1dea" />

GiteaMirror added the area: tinytorch type: improvement labels 2026-04-19 12:21:22 -05:00

GiteaMirror closed this issue

2026-04-19 12:21:22 -05:00

GiteaMirror commented

2026-04-19 12:21:24 -05:00

@profvjreddi commented on GitHub (Feb 19, 2026):

Thanks for catching this, @ngbolin! You're absolutely right.

The sum 1 + 2 + ... + 100 = 5,050 is the total number of K,V computations in the naive approach, but not all of them are redundant — each token still requires its own K,V computation even with caching (100 necessary computations). So the correct number of redundant computations is 5,050 − 100 = 4,950.

This has been fixed in both the module source and the paper to clarify the distinction:

"5,050 total K,V computations, of which 4,950 are redundant"

Thanks for the careful reading! @all-contributors please add @ngbolin as a contributor for ✍️ Doc in TinyTorch

@profvjreddi commented on GitHub (Feb 19, 2026): Thanks for catching this, @ngbolin! You're absolutely right. The sum 1 + 2 + ... + 100 = 5,050 is the **total** number of K,V computations in the naive approach, but not all of them are redundant — each token still requires its own K,V computation even with caching (100 necessary computations). So the correct number of **redundant** computations is 5,050 − 100 = **4,950**. This has been fixed in both the module source and the paper to clarify the distinction: - "5,050 total K,V computations, of which 4,950 are redundant" Thanks for the careful reading! @all-contributors please add @ngbolin as a contributor for ✍️ Doc in TinyTorch

GiteaMirror commented

2026-04-19 12:21:24 -05:00

@github-actions[bot] commented on GitHub (Feb 19, 2026):

I've added @ngbolin as a contributor to tinytorch! 🎉

Recognized for: doc
Project: tinytorch (explicitly mentioned in comment)
Based on: Thanks for the careful reading! @all-contributors please add @ngbolin as a contributor for ✍️ Doc in TinyTorch

The contributor list has been updated in:

tinytorch/.all-contributorsrc
tinytorch/README.md
Main README.md

We love recognizing our contributors! ❤️

@github-actions[bot] commented on GitHub (Feb 19, 2026): I've added @ngbolin as a contributor to **tinytorch**! :tada: **Recognized for:** doc **Project:** tinytorch (explicitly mentioned in comment) **Based on:** Thanks for the careful reading! @all-contributors please add @ngbolin as a contributor for ✍️ Doc in TinyTorch The contributor list has been updated in: - `tinytorch/.all-contributorsrc` - `tinytorch/README.md` - Main `README.md` We love recognizing our contributors! :heart:

GiteaMirror commented

2026-04-19 12:21:25 -05:00

@profvjreddi commented on GitHub (Feb 19, 2026):

Hi @ngbolin, I wanted to reach out because you've been one of our most dedicated contributors, working through the modules from Autograd (#1131) all the way up to Memoization here. That kind of systematic engagement is exactly what we hoped TinyTorch would inspire.

We're having a TinyTorch community meetup on Tuesday, March 3rd at 9am and would love to have you share a few minutes about your experience. What drew you to TinyTorch, what you've learned working through the modules, anything you'd like to share. No pressure on format, just a casual conversation.

As one of our first active users, we'd also like to send you a small thank you. Keep an eye on our Discord for more details!

@profvjreddi commented on GitHub (Feb 19, 2026): Hi @ngbolin, I wanted to reach out because you've been one of our most dedicated contributors, working through the modules from Autograd (#1131) all the way up to Memoization here. That kind of systematic engagement is exactly what we hoped TinyTorch would inspire. We're having a TinyTorch community meetup on **Tuesday, March 3rd at 9am** and would love to have you share a few minutes about your experience. What drew you to TinyTorch, what you've learned working through the modules, anything you'd like to share. No pressure on format, just a casual conversation. As one of our first active users, we'd also like to send you a small thank you. Keep an eye on our Discord for more details!

GiteaMirror commented

2026-04-19 12:21:25 -05:00

@ngbolin commented on GitHub (Feb 20, 2026):

Hi Prof Reddi,

Thank you so much for the kind message, I would love to share my experiences while working through the modules, and what I've learnt. See you on 3rd March, 9am!

@ngbolin commented on GitHub (Feb 20, 2026): Hi Prof Reddi, Thank you so much for the kind message, I would love to share my experiences while working through the modules, and what I've learnt. See you on 3rd March, 9am!

GiteaMirror commented

2026-04-19 12:21:26 -05:00

@ngbolin commented on GitHub (Feb 20, 2026):

Sorry, I couldn't use the discord invite from your earlier email - it appears that the invite has expired. Will you or your team be able to send another discord invite, please? Thank you!

My email is ngbolin91@gmail.com.

@ngbolin commented on GitHub (Feb 20, 2026): Sorry, I couldn't use the discord invite from your earlier email - it appears that the invite has expired. Will you or your team be able to send another discord invite, please? Thank you! My email is ngbolin91@gmail.com.

GiteaMirror commented

2026-04-19 12:21:27 -05:00

@profvjreddi commented on GitHub (Feb 24, 2026):

Hi @ngbolin,

Thanks for your interest in the TinyTorch community meetup on Tuesday, March 3rd at 9am. To get you set up as a speaker and share logistics, we need a few details from you.

Please:

Reply to: karijanapareddi@gmail.com
CC: vj@eecs.harvard.edu and andreamattia.garavagno@santannapisa.it
Use the email address you want us to use for Zoom registration and for promoting you to panelist so you can speak on the call (it's a webinar-style meeting).
Slide We'd love it if you could share a slide or two about your experience, you can add them here—there's a template to follow:
https://docs.google.com/presentation/d/1h0bQZJVtmXYuO4jgakE3x3xGt0e8fNsvW7QfWK1_5D4/edit?slide=id.g3cac3cc65ae_0_19#slide=id.g3cac3cc65ae_0_19

We'll send logistical details (Zoom link, agenda, etc.) to your registered email.

Looking forward to having you on the call.

@profvjreddi commented on GitHub (Feb 24, 2026): Hi @ngbolin, Thanks for your interest in the TinyTorch community meetup on **Tuesday, March 3rd at 9am**. To get you set up as a speaker and share logistics, we need a few details from you. **Please:** 1. **Reply to:** **karijanapareddi@gmail.com** **CC:** vj@eecs.harvard.edu and andreamattia.garavagno@santannapisa.it 2. **Use the email address** you want us to use for Zoom registration and for promoting you to **panelist** so you can speak on the call (it's a webinar-style meeting). 3. **Slide** We'd love it if you could share a slide or two about your experience, you can add them here—there's a template to follow: https://docs.google.com/presentation/d/1h0bQZJVtmXYuO4jgakE3x3xGt0e8fNsvW7QfWK1_5D4/edit?slide=id.g3cac3cc65ae_0_19#slide=id.g3cac3cc65ae_0_19 We'll send logistical details (Zoom link, agenda, etc.) to your registered email. Looking forward to having you on the call.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#4342