Follow-up to #2180 for the questions listed in the closing comment. The
mlsysim registry (hardware/data/mobile/iPhone15Pro.yaml) gives the A17 Pro
51.2 GB/s (LPDDR5); every figure derived from the old value is recomputed.
- mobile-0109 (68 GB/s): weight load 0.05 -> 0.07 ms; the ~3.8 ms
overhead-bound total is unchanged.
- mobile-0998 (68 GB/s): A17 Pro TPOT bound 7.4 -> 9.8 ms. Snapdragon still
wins both concurrency and TPOT, and Tensor G3 (10.0 ms) is still slower.
- mobile-1001 (68 GB/s): the scenario said iPhone 16 Pro but the title and
analysis are A17 Pro, so it now says iPhone 15 Pro. The 7.4 ms baseline is
faster than 500 MB / 51.2 GB/s allows, so TPOT is rescaled 7.4 -> 9.8 ms
and 14 -> 18.6 ms; the implied sustained bandwidth is 26.9 GB/s, keeping
the 52.5% ratio and the "roughly halved" conclusion.
- mobile-1021 (68 GB/s): the A17 Pro is an LPDDR5 design, so it now
illustrates the LPDDR5 case that misses 30 ms. The LPDDR5 TPOT is
1.5 GB / (51.2 - 2.6) GB/s = 30.9 ms; the old 51.2/50*30 = 30.7 ms had the
ratio inverted. Tensor G3 (registry 60 GB/s, not 51.2) is dropped from that
sentence.
- mobile-1060 (~100 GB/s): A17 Pro decode 66.7 -> 34.1 tok/s. Tensor G3
uses the registry's 60 GB/s instead of 68 (45.3 -> 40.0 tok/s). The
ranking becomes Snapdragon > Tensor G3 > A17 Pro; the lesson (decode
follows bandwidth, not TOPS) is kept: Tensor G3 has the lowest TOPS but
beats the A17 Pro.
Only text fields change. Refs #2180.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SxFzWDUc3UeWQ1QBGjw6UM
The mlsysim registry (hardware/data/mobile/iPhone15Pro.yaml) gives the
A17 Pro 51.2 GB/s (LPDDR5-6400, 64-bit), as do 29 other A17 Pro questions.
These 28 used 60, 68, 85, 100 or 120 GB/s; 68.25 GB/s is the M1's LPDDR4X
figure. Every number derived from the bandwidth is recomputed:
- 68 GB/s (23 questions): decode, ridge-point and load-time figures
rescaled by 68/51.2; e.g. mobile-0978 INT8 TPOT 14.7 -> 19.5 ms,
mobile-1010 ridge 514 -> 684 ops/byte, mobile-1783 contended decode
10 -> 5.2 tok/s.
- Where a scenario's measured number exceeded the hardware limit, the
scenario is rescaled so the question still teaches the same point:
mobile-1005 TPOT 7.4 -> 9.8 ms baseline (was faster than 500 MB /
51.2 GB/s allows), mobile-1013 TPOT 88 -> 117 ms with the target scaled
30 -> 40 ms, mobile-1786 measured 15/12 -> 11/9 tok/s (15 tok/s of a
4.3 GB model needs 65 GB/s).
- mobile-0724 (120 GB/s): a 7B INT4 model cannot reach the scenario's
25 tok/s at 51.2 GB/s (ceiling 14.6 tok/s), so the model is now 3B INT4
(1.5 GB), which keeps the 25 -> 5 tok/s contention story.
- mobile-1422 (100 GB/s effective): the layout-copy traffic is now 80 MB
per transpose at ~40 GB/s effective, keeping the 16 ms copy estimate.
- mobile-1027, mobile-1036, mobile-1597: LPDDR5X -> LPDDR5.
mobile-1775 from the issue is already status: deleted and is left alone.
Only text fields change. Refs #2180.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SxFzWDUc3UeWQ1QBGjw6UM
- edge-1138: the stem used 54.6M FLOPs "under the benchmark's convention",
but every other layer counts 2 ops per MAC. On that basis the stem is
2 * 112*112*32 * 27 = 21.7M FLOPs, AI = 39 FLOPs/byte over the question's
552 KB, below the 55 FLOPs/byte ridge: memory-bound at ~71% roofline MFU,
not compute-bound. Conclusion and realistic_solution updated to match.
- edge-2138: Orin NX ships as 8 GB or 16 GB at 102.4 GB/s (mlsysim
JetsonOrinNX.yaml: 16 GiB, 102 GB/s); "32GB at 68 GB/s" matches no Orin.
Spill traffic is now ~4% of the bandwidth ceiling; the conclusion holds.
- edge-0586: the unfused itemization now includes the conv's 12 MB
input+weight read, so it sums to the stated 62 MB.
Refs #2180.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SxFzWDUc3UeWQ1QBGjw6UM
At batch 1 every decode step streams all weights from DRAM/HBM once;
"resident" means resident in DRAM, not on-chip. These six questions
counted only KV traffic, which flips three answers and understates two
latencies by about 20x:
- edge-1186: INT8 takes 3.78 s and misses the 2 s SLO; INT4 takes 1.93 s
and meets it only at peak bandwidth (was: both meet it in 72 ms).
- edge-0918: (1 GB + 0.128 GB) per token at 25 GB/s gives 22.2 tok/s, the
measured rate; weights are 89% of the bytes, so speculative decoding or
lower-bit weights beat GQA (which caps out near 25 tok/s).
- mobile-1054: the 1.9 GB weight stream alone takes 24.7 ms > 20 ms TPOT,
so no context length meets the SLO (was: ~4K context fits).
- edge-1430: TPOT 105 ms, decode 10.5 s, E2E ~10.65 s (was 575 ms);
prefill is floored at 100 ms by its weight read.
- edge-1194: decode ~7.3 s, 96% of it weight traffic (was ~320 ms); the
64 GB/s figure is now labelled as effective (Orin NX peak 102.4 GB/s).
- cloud-3015: the 4.3 ms weight-plus-KV figure is the batch-1 floor, not a
pessimistic bound; the conclusion (KV cache wins) holds at 3-11x.
Only text fields change; ids, taxonomy tags and audit stamps are untouched.
Refs #2178.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SxFzWDUc3UeWQ1QBGjw6UM
10 of 23 questions tagged with a moved concept change subject to the new
home; 44 of 185 subject-only questions gain the concepts two independent
taggers both chose (80.5% tag-or-not agreement). vault check --strict passes.
878 concepts (+71 net), 2,020 relationships (+326, -25, 25 reworded), 27
redefinitions, 67 alias sets, 33 background tags, 6 moves; guides 16 roles,
24 symptoms, 7 paths; gaps: attention-FFN disaggregation closed, verifiable
ML computation and reasoning-trace monitoring declared. Coverage judgments
updated for the new concepts and a blind held-out checklist added
(OSDI/NSDI/EuroSys/ASPLOS 2026, 202 papers, 22.3% covered, 96.5% homed).
Symptoms now declare the quantity they are measured by (`quantities`), so a
symptom's own metric no longer poses as a cause; the compiler, the StaffML
loader, and the explorer accept a check or lever that touches a cause or the
quantity and reject a quantity listed in another bucket.
README and CHANGELOG record the method, the numbers, and the open author
decisions.
Both independent taggers chose expert offloading for the same six
questions and none of the other new concepts for any subject-only
question. vault check --strict passes.
Two independent taggers (gpt-6.1-sol, claude-opus-5-5) offered each of
the 445 subject-only questions its subject's concepts plus the 63 new
ones; they agreed on tag-or-not for 76.6%. A concept is written only
where both chose it. 339 questions remain subject-only.
Of 25 questions, 7 had taken their subject from the concept's old home:
three PII questions move to privacy-dp, four FLOP-accounting questions to
workload-characterization. The other 18 are tagged by what they test and
keep their subject. vault check --strict passes.
Regenerated with emit_taxonomy_labels: 63 new concepts, the renamed
diffusion or flow-matching policy, and the two moved concepts. vault
check --strict passes on all 10,716 questions.
Latency comparisons such as "4.5 minutes" wrapped onto two lines in the
narrow column, raw latencies showed noise digits (272.82 ns), and the
ESP32-S3's 80 MB/s bandwidth rounded to "0 GB/s".
StaffML is a static site and cannot run MLSysIM, so its numbers lived in a
hand-copied table that cited a deleted mlsysim/core/constants.py. Five
devices had drifted: Jetson Orin NX showed its FP16 rate under an INT8
label, the A17 Pro listed 100 GB/s against 51.2 GB/s, B200 memory was 192
against 180, TPU v5p had the wrong year, and every ESP32-S3 field was off.
scripts/export_mlsysim_data.py now reads the MLSysIM registries straight
from the repo and writes src/data/mlsysim.json, which hardware.ts serves.
Each row records the registry entry it came from. Interconnect and latency
values move to MLSysIM's measured or sourced figures (NVLink B200 0.5 us,
InfiniBand XDR 5 us, 100GbE 50 us, Hopper HBM 273 ns). The cross-US fiber
row stays local because MLSysIM has no route distance. The simulator's
flops_per_token is 6 x parameters, now labeled as training cost.
Latency comparisons such as "4.5 minutes" wrapped onto two lines in the
narrow column, raw latencies showed noise digits (272.82 ns), and the
ESP32-S3's 80 MB/s bandwidth rounded to "0 GB/s".
Covers the scenario-plus-question context, the Assumptions-only cut used
in interview mode, and the site and worker copies of the delimiter
pattern and the cut, which must stay identical.
Groq is the production provider with Workers AI as the floor, the /ask
body has mode, canonicalAnswer, and napkinMath, /interview is documented,
the default models match the code, and the Groq free tier's per-minute
limit is explained.
claude-3-5-haiku-latest and gemini-1.5-flash no longer exist, so adding
either key would have failed every request into the fallback. Haiku 5.5
rejects a non-default temperature and thinks by default, so the Anthropic
call drops temperature and turns thinking off to keep the short reply
budget for text. Neither key is set in production, so live traffic is
unchanged.
The vulnerable undici is pinned inside wrangler's miniflare, so wrangler
moves to 4.153 in both workers. These are build and test tools only; the
interviewer worker's deploy bundle is unchanged. Typecheck, the vault
worker's tests, and both dry-run deploys pass.
Resolves the open Dependabot alerts on staffml/app, including the critical
next advisory (fixed in 16.4.1). Every bump stays inside the declared major
version. Typecheck, unit tests, the production build, and the headless
E2E smoke all pass.
An uploaded test version stays publicly reachable through its preview URL
with whatever vars it was uploaded with, spending the production provider
keys.
The interviewer only received the scenario, which often leaves out the
actual question. It now gets the scenario plus the question, and the
question's napkin math so the worker can ground both modes. Copy-as-prompt
carries the same blocks.
Vijay Janapa Reddi
github-actions[bot]
Claude
Nishitha RandClaude Opus 5.5