Suggested improvement:
The answer check should perhaps not only look for numbers, but also for units. I just did a simple calculation that it would need 4 inferences per seconds, ergo 80 x 4 MFLOPS, which leaves 16 MFLOPS margin capacity (5%) for everything else. So the detected answer was "16" while the model answer was given as "2". I am still not quite clear about why it expected "2" as an answer. My napkin calculation was probably a bit overnapkinnish, but not that wrong.
Why this is better:
It is a bit unclear how the answer is extracted from the input, there may be different more or less valid approaches to solving a problem involving different numerical representations of the same result.
Originally created by @asgalon on GitHub (Apr 23, 2026).
Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1502
**Question ID:** `tinyml-0384`
**Title:** The Real-Time KWS Deadline
**Level:** L2 | **Track:** tinyml | **Area:** latency
**Suggested improvement:**
The answer check should perhaps not only look for numbers, but also for units. I just did a simple calculation that it would need 4 inferences per seconds, ergo 80 x 4 MFLOPS, which leaves 16 MFLOPS margin capacity (5%) for everything else. So the detected answer was "16" while the model answer was given as "2". I am still not quite clear about why it expected "2" as an answer. My napkin calculation was probably a bit overnapkinnish, but not that wrong.
**Why this is better:**
It is a bit unclear how the answer is extracted from the input, there may be different more or less valid approaches to solving a problem involving different numerical representations of the same result.
@Shashank-Tripathi-07 commented on GitHub (Apr 23, 2026):
oh okay I think I see what's going on
so option 3 is actually using the same approach you did, 4 windows per second times 80 MFLOPs = 320 MFLOPS needed, MCU has 336, so 16 MFLOPS headroom. that math is fine honestly. but then it says "the MCU only runs at 168 MHz" like MHz and MFLOPS are the same thing, which they're not cortex-m4 does about 2 flops per cycle so you get ~336 MFLOPS out of 168 MHz. so the distractor ends up using correct reasoning and then breaking it with a bad unit assumption at the end
your 16 MFLOPS answer wasn't wrong, it's literally the same result as the 12ms slack in option 0, just expressed differently
I think someone needs to fix option 3 so it's wrong for an actually wrong reason, and probably also update the napkin math to show both ways of solving it. will see if I can put a patch together
P.S. It was nice meeting you in today's meeting 😄
<!-- gh-comment-id:4306716545 -->
@Shashank-Tripathi-07 commented on GitHub (Apr 23, 2026):
oh okay I think I see what's going on
so option 3 is actually using the same approach you did, 4 windows per second times 80 MFLOPs = 320 MFLOPS needed, MCU has 336, so 16 MFLOPS headroom. that math is fine honestly. but then it says "the MCU only runs at 168 MHz" like MHz and MFLOPS are the same thing, which they're not cortex-m4 does about 2 flops per cycle so you get ~336 MFLOPS out of 168 MHz. so the distractor ends up using correct reasoning and then breaking it with a bad unit assumption at the end
your 16 MFLOPS answer wasn't wrong, it's literally the same result as the 12ms slack in option 0, just expressed differently
I think someone needs to fix option 3 so it's wrong for an actually wrong reason, and probably also update the napkin math to show both ways of solving it. will see if I can put a patch together
P.S. It was nice meeting you in today's meeting 😄
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @asgalon on GitHub (Apr 23, 2026).
Original GitHub issue: https://github.com/harvard-edge/cs249r_book/issues/1502
Question ID:
tinyml-0384Title: The Real-Time KWS Deadline
Level: L2 | Track: tinyml | Area: latency
Suggested improvement:
The answer check should perhaps not only look for numbers, but also for units. I just did a simple calculation that it would need 4 inferences per seconds, ergo 80 x 4 MFLOPS, which leaves 16 MFLOPS margin capacity (5%) for everything else. So the detected answer was "16" while the model answer was given as "2". I am still not quite clear about why it expected "2" as an answer. My napkin calculation was probably a bit overnapkinnish, but not that wrong.
Why this is better:
It is a bit unclear how the answer is extracted from the input, there may be different more or less valid approaches to solving a problem involving different numerical representations of the same result.
@Shashank-Tripathi-07 commented on GitHub (Apr 23, 2026):
oh okay I think I see what's going on
so option 3 is actually using the same approach you did, 4 windows per second times 80 MFLOPs = 320 MFLOPS needed, MCU has 336, so 16 MFLOPS headroom. that math is fine honestly. but then it says "the MCU only runs at 168 MHz" like MHz and MFLOPS are the same thing, which they're not cortex-m4 does about 2 flops per cycle so you get ~336 MFLOPS out of 168 MHz. so the distractor ends up using correct reasoning and then breaking it with a bad unit assumption at the end
your 16 MFLOPS answer wasn't wrong, it's literally the same result as the 12ms slack in option 0, just expressed differently
I think someone needs to fix option 3 so it's wrong for an actually wrong reason, and probably also update the napkin math to show both ways of solving it. will see if I can put a patch together
P.S. It was nice meeting you in today's meeting 😄