github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-07-16 23:24:55 -05:00
Code Issues 276 Packages Projects Releases 26 Wiki Activity

[GH-ISSUE #1502] [StaffML] Improve: The Real-Time KWS Deadline #13531

New Issue
Closed
opened 2026-05-17 17:41:05 -05:00 by GiteaMirror · 1 comment
GiteaMirror commented 2026-05-17 17:41:05 -05:00
Owner
Copy Link

Originally created by @asgalon on GitHub (Apr 23, 2026).
Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1502

Question ID: tinyml-0384
Title: The Real-Time KWS Deadline
Level: L2 | Track: tinyml | Area: latency

Suggested improvement:
The answer check should perhaps not only look for numbers, but also for units. I just did a simple calculation that it would need 4 inferences per seconds, ergo 80 x 4 MFLOPS, which leaves 16 MFLOPS margin capacity (5%) for everything else. So the detected answer was "16" while the model answer was given as "2". I am still not quite clear about why it expected "2" as an answer. My napkin calculation was probably a bit overnapkinnish, but not that wrong.

Why this is better:
It is a bit unclear how the answer is extracted from the input, there may be different more or less valid approaches to solving a problem involving different numerical representations of the same result.

Originally created by @asgalon on GitHub (Apr 23, 2026). Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1502 **Question ID:** `tinyml-0384` **Title:** The Real-Time KWS Deadline **Level:** L2 | **Track:** tinyml | **Area:** latency **Suggested improvement:** The answer check should perhaps not only look for numbers, but also for units. I just did a simple calculation that it would need 4 inferences per seconds, ergo 80 x 4 MFLOPS, which leaves 16 MFLOPS margin capacity (5%) for everything else. So the detected answer was "16" while the model answer was given as "2". I am still not quite clear about why it expected "2" as an answer. My napkin calculation was probably a bit overnapkinnish, but not that wrong. **Why this is better:** It is a bit unclear how the answer is extracted from the input, there may be different more or less valid approaches to solving a problem involving different numerical representations of the same result.
GiteaMirror added the area: bookstaffmlpriority-hightype: question labels 2026-05-17 17:41:05 -05:00
GiteaMirror commented 2026-05-17 17:41:09 -05:00
Author
Owner
Copy Link

@Shashank-Tripathi-07 commented on GitHub (Apr 23, 2026):

oh okay I think I see what's going on

so option 3 is actually using the same approach you did, 4 windows per second times 80 MFLOPs = 320 MFLOPS needed, MCU has 336, so 16 MFLOPS headroom. that math is fine honestly. but then it says "the MCU only runs at 168 MHz" like MHz and MFLOPS are the same thing, which they're not cortex-m4 does about 2 flops per cycle so you get ~336 MFLOPS out of 168 MHz. so the distractor ends up using correct reasoning and then breaking it with a bad unit assumption at the end

your 16 MFLOPS answer wasn't wrong, it's literally the same result as the 12ms slack in option 0, just expressed differently

I think someone needs to fix option 3 so it's wrong for an actually wrong reason, and probably also update the napkin math to show both ways of solving it. will see if I can put a patch together

P.S. It was nice meeting you in today's meeting 😄

<!-- gh-comment-id:4306716545 --> @Shashank-Tripathi-07 commented on GitHub (Apr 23, 2026): oh okay I think I see what's going on so option 3 is actually using the same approach you did, 4 windows per second times 80 MFLOPs = 320 MFLOPS needed, MCU has 336, so 16 MFLOPS headroom. that math is fine honestly. but then it says "the MCU only runs at 168 MHz" like MHz and MFLOPS are the same thing, which they're not cortex-m4 does about 2 flops per cycle so you get ~336 MFLOPS out of 168 MHz. so the distractor ends up using correct reasoning and then breaking it with a bad unit assumption at the end your 16 MFLOPS answer wasn't wrong, it's literally the same result as the 12ms slack in option 0, just expressed differently I think someone needs to fix option 3 so it's wrong for an actually wrong reason, and probably also update the napkin math to show both ways of solving it. will see if I can put a patch together P.S. It was nice meeting you in today's meeting 😄
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
area: book
area: collabs
area: kits
area: labs
area: socratiq
area: staffml
area: tinytorch
area: tools
area: website
bug
dependencies
format: epub
format: pdf
javascript
link-health
link-rot
priority-high
pull-request
Mirrored from GitHub Pull Request
 staffml
type: bug
type: citation
type: code
type: errata
type: improvement
type: new
type: question
vault-sli
No Label area: book priority-high staffml type: question
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/cs249r_book#13531
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?