Add reproducible Epoch AI frontier training compute (@fig-ai-compute-growth) and single-run pretraining energy (@fig-frontier-training-energy) figures to Volume II, Chapter 15, paired with builder-reported anchors at declared measurement boundaries, deterministic CSV extraction and manifest receipts, and structured provenance enforcement across book CSV series and figure cells.
Vol I: Ch2 cloud compute floor (200 TFLOP/s) so the TPU v4 row's 275
TFLOP/s sits in its tier; Ch8 utilization anchored in PaLM Table 3 MFU
(MT-NLG 30.2, PaLM 46.2) with Megatron-LM's 52 percent explained as
counting recomputed activations; Ch9 cost table from the registry's
1,281,167 images and 148 GB; Ch11 about 2.1x between the two ratios; Ch12
names the MLPerf Training v3.1 system behind 5,000 images/s; Ch13 prints
7.5x; Ch14 margin figure at about three days and the Knight Capital war
story rewritten from the SEC order.
Vol II: Ch1 separates MFU from Megatron-LM's percent of peak; Ch7 prose
accounts for all 419 interruptions and matches the figure legend.
Vol IV: Ch10 Equation 9 evaluated, with a measured Franka stop time (552
ms, handbook p. 85) and its derived deceleration; Ch13 recovery 2.18 to
2.55 s; Ch14 counts the dispatch-to-admission interval in the takeover
budget, as MLSysIM already did (spare 48.7 mm, in-loop overrun 737.8 mm);
Ch14 names Waymo as Ch7 and Ch16 do.
MLSysIM: new provenance entries and tests for the added values.
howard2017mobilenet duplicated howard2017mobilenets with a truncated
author list; every book citation uses the latter. Its curated provenance
moves to the surviving key. The MLPerf-EDU paper cited it for
MobileNetV2, which is Sandler et al. (2018); it now cites sandler2018.
The 8.5 billion searches a day was uncited and traced to a third-party
web counter. Google's blog (Srinivasan, 3 March 2025) states more than
5 trillion searches a year; ReferenceStats.Workloads.GoogleSearchesPerDay
now holds that divided by 365 (13.7 billion, a lower bound) with the
post as provenance, ch1 says "more than" and cites it, and the System
Assumptions table updates from the same entry. Bib entry added through
bib add and re-sealed after unbracing the author. Paper provenance
count 245 -> 246.
A new chapter 17, after Release, teaches who answers for the decisions
behind an admitted action, oversight a person can actually perform,
permission in stages, how a detection gap between groups of people
becomes a gap in impact energy, who bears risk without choosing it, data
about people, and the work the machine moves onto people. It opens on the
2023 Cruise collision. The approval record names who approved every
decision and blocks release when a name is missing. Privacy and incident
disclosure move in from Release, and worker burden from Frontier. The
fourth law is now named permission must be earned. The part opener,
roadmap, spine figure, dependency map, appendices, glossary, labs, and
concept maps follow the new order.
iea2025energyai through bib add --url (published 10 April 2025).
shehabi2024united through bib add --doi, then @techreport with the full
author list, 2024 date, and LBNL-2001637 from LBNL's BibTeX export,
re-sealed against that export (DataCite carries one author and 2025).
villalobos2024will (checked against the PMLR page: title, authors, year,
pages), elsworth2025measuring, schneider2025life, schneider2024carbon,
chung2025energy, and luccioni2025misinformation (checked against arXiv:
title, authors, year). Replaces the hand-written manifest records from
the earlier house-style edits.
`bib add` refuses existing keys, so an entry edited after its gateway
seal (sentence-case title, @misc -> @inproceedings, venue, pages) had no
binder path back to a verified manifest record; records were re-written
by hand. `bib reseal --key K --note "..."` re-resolves the source the
entry was sealed from (or a --doi/--arxiv/--url given), compares title,
author surnames in order, year, DOI, and pages with the edited entry,
and rewrites the manifest record only if they all agree. It never edits
references.bib.
`fetch_url` now reads the page head and records Highwire citation_*
meta tags (title, authors, year, pages, venue, DOI) as evidence, so
publisher pages such as PMLR are checked on authors and pages rather
than on caller overrides. Author comparison folds LaTeX accent escapes
(M\k{a}dry), Unicode diacritics, and name particles.
Only bibs/ conflicted (adjacent additions on both sides). references.bib
keeps both sides; the Waymo data hub title takes dev's sentence case.
manifest.json is the union, dev winning on shared keys; the offline
ledger check passes.
Elsworth et al. 2025 (Gemini per-prompt energy, emissions, water),
Schneider et al. 2025 (AI hardware life-cycle emissions), Schneider and
Mattia 2024 (cloud emissions allocation), Chung et al. 2025 (ML.ENERGY
benchmark, NeurIPS 2025 D&B), and Luccioni et al. 2025 (environmental
transparency). All added through bib add --arxiv, then house-style title
case and venue, with manifest records re-recorded.
The entry cited the arXiv preprint with year 2024 while its manifest
record still said 2022 from the original DataCite seal. Re-sealed through
the reference gateway from the PMLR page (proceedings.mlr.press, HTTP 200),
then converted to house-style @inproceedings from the page's citation
metadata: PMLR 235, pages 49523--49544, published title with "Position:".
Manifest record now carries year 2024, the canonical venue, and the
superseded preprint DOI. The book's cited claims (2026-2032 window,
~3e14-token stock) match the published version.
Branch commits skipped hooks; the full suite at the merge tip found these.
- Bib: drop duplicate epochai2026benchmarks from the merge; rename
villalobos2022will to villalobos2024will to match its year; re-record
ledger hashes for 24 entries whose content is unchanged (tidy reformat)
- Checks: scope cross-chapter-callout-ref to vol1 (vol2-4 carry 113 legacy
refs); add Whetstone to heading proper nouns; lego-dead-code skips
render output
- Units: computed sizes in decimal units, hardware capacities through
fmt_memory_capacity (ch6, ch11, ch14, ch16)
- Prose: abbreviation first-use expansions after content moves, footnote
marker placement, two footnote fixes, margin caption, bracketed source
citations, index main, orphan and duplicate equation labels, dead
LEGO exports
- Auto-fixers: Black on hidden cells, whitespace, blank lines, tables,
bibtex tidy, canonical spellings
Conflict resolution:
- ComputeTrend: dev and this branch re-pinned it to Epoch AI in parallel;
kept this branch's rows and the vol1/vol2 cells built on them
- literature/series.py and registry: kept both series APIs
(Literature.Series.<Name> for vol1/vol2, Literature.<Name> for vol3/vol4)
- bibs/manifest.json: union of both sides' verified entries
- ch16: Waymo paragraph drops a stale "over twelve years" sentence (now
computed from the measured 12x), cites the hub for June 2026 rates, and
says vehicles were "involved in" crashes; Gender Shades re-audit reports
an association, not a cause, and no longer claims unmeasured systems
"persisted".
- ch14: RobustBench described as a standardized ensemble, not
defense-adaptive; ImageNet-V2 wording no longer overclaims.
- ch3/ch6: captions cite the IEEE, IBTA, PCI-SIG, and NVIDIA sources
instead of a nonexistent table; ch3 drops the "lower bound" overclaim.
- ch1, ch2, ch4, ch8, ch9, ch10, ch11, ch12, ch15: claims scoped to what
the data shows (delivered rate vs. hardware, TDP vs. TGP conventions,
ResNet-50 bridging, mature-benchmark wording, interactivity vs.
hardware, peak-ratio bound, spread order, NNAPI vs vendor runtimes);
three announcer sentences folded into content.
- ch11: Apple Neural Engine ratings cited to six Apple announcements; the
precision change is hedged consistently.
- ch15: the 33x one-year reduction now loads from GoogleServingFactors
with a guard.
- New Literature.Series.GeminiPromptEnergy: Table 1 of Elsworth et al.
(2025), energy of the median Gemini text prompt by serving component.
- New fig-prompt-energy-boundary in inference energy costs: 0.24 Wh full
stack (accelerators 58%, host 25%, idle and overhead 17%) against
0.10 Wh for the accelerator-only boundary most estimates use (2.4x);
the prose relates it to the worked example's 1 Wh and the paper's 33x
one-year reduction.
- Bib entry elsworth2025measuring via the gateway (sentence-case title).
- New Literature.Series.NCCLAllReduceH100Node: NVIDIA exemplar-performance
all_reduce_perf output (MIT), one node of eight H100s over NVLink,
8 B to 16 GB.
- New fig-nccl-allreduce-measured after the modeled crossover figure:
~17 us floor, 274 GB/s asymptotic algorithm bandwidth (480 GB/s bus),
model knee ~5 MB, and measured time up to 1.7x the model between 8 and
128 MB, so the model is a lower bound in the gradient-bucket range.
- Bib entry nvidia2026exemplarnccl via the gateway.
- New Literature.Series.WaymoRiderOnlyMiles and WaymoVsHumanBenchmark
from the Waymo Safety Impact Data Hub (published data cited as facts;
no redistribution license stated by the source).
- fig-autonomous-miles-scaling rebuilt: (a) rider-only miles 22M (Jun
2024) to 271M (Jun 2026); (b) incidents per million miles against the
human benchmark with 95% CIs. The fleet-safety paragraph now quotes the
measured exposure and the 183 injury and 2 fatal crashes it implies.
- The illustrative data/autonomous_miles_scaling.csv is removed.
- Bib entries waymo2026safetyimpact and kusano2024waymo via the gateway.
- New Literature.Series.AIBenchmarkSoCs: ETH AI Benchmark mobile SoC
ranking (published scores cited as facts; no redistribution license is
stated by the source).
- New fig-mobile-ai-spread after the fleet-heterogeneity paragraph:
processors from 2020 on span ~400x in measured AI score, and
same-generation flagships differ ~14x (Tensor G4 vs Dimensity 9600 M).
- Bib entries ignatov2019aibenchmark and ethz2026aibenchmark via the
gateway.
- New Literature.Series.RobustBenchCIFAR10Linf: RobustBench (MIT)
CIFAR-10 l-infinity 8/255 entries with clean and AutoAttack accuracy.
- New fig-robustbench-progress after the adaptive-evaluation paragraph:
best accuracy under attack rose from 44% (2018) to 75% (2024) at 94%
clean, and four published defenses fall below 5% under the
standardized attack.
- Bib entries croce2021robustbench and croce2020autoattack via the
gateway.
- New Literature.Series.ImageNetTestbedShift: Taori et al. (2020) testbed
accuracies (MIT) on ImageNet, ImageNet-V2, ObjectNet, ImageNet-R, and
ImageNet-Sketch.
- New fig-imagenet-v2-shift after the covariate-shift paragraph: all 212
models at >= 55% ImageNet accuracy lose 7-14 points (median 12) on
ImageNet-V2, along a near-linear trend.
- Bib entry taori2020measuring via the gateway.
- New Literature.Series.MLPerfInferenceDGXH100: NVIDIA's same eight-H100
DGX system across MLPerf Inference v3.1-v5.0 (Apache-2.0).
- New fig-mlperf-software-gains closing the TensorRT section: with the
silicon unchanged, GPT-J rose 2.7x, Llama 2 70B 1.4x, SDXL 1.2x, while
ResNet-50 and BERT did not move.
- Bib entry mlcommons2025inference via the gateway.
- New Literature.Series.AcmeKalosJobSizes: AcmeTrace Kalos cluster
(Hu et al., NSDI 24; CC BY 4.0) jobs and GPU time by requested GPUs.
- New fig-acme-job-size after the two-team scheduling scenario: 86% of
19,907 GPU jobs use one GPU and 0.1% of GPU time, while jobs of 256+
GPUs are 4% of jobs and 96% of GPU time.
- Bib entry hu2024characterization via the gateway.
- New Literature.Series.MLPerfTrainingLLMScaling: MLPerf Training
v3.1-v5.1 LLM results (Apache-2.0), time-to-train recomputed from the
run logs, with global batch size.
- New fig-mlperf-training-scaling after fig-scaling-tax as its measured
counterpart: five same-round series (GPT-3 on H100, TPU v5p, Trillium;
Llama 3.1 405B on H100 and GB200) keep 71-84% of ideal at their
largest size; Eos GPT-3 512 -> 10,752 H100 cut time 14.9x.
- Prose names the batch-size confound and points to the critical batch
size section.
- Bib entry mlcommons2025training via the gateway.
- New Literature.Series.MLPerfStorageV1Training: MLPerf Storage v1.0
results (Apache-2.0), one row per system x workload x emulated
accelerator.
- New fig-mlperf-storage-demand after the workload bandwidth-profile
paragraph: throughput scales with emulated accelerator count (the N
term of eq-pipeline-throughput); per-accelerator medians differ ~15x
across 3D U-Net, CosmoFlow, and ResNet-50; H100 emulation raises
demand 1.6-2x over A100 (the T_compute term).
- Bib entries mlcommons2024storage and devarajan2021dlio via the gateway.
Seven matplotlib figures across six chapters, each with lead-in, caption,
alt text, and payoff prose; every plotted and quoted value comes from a
provenance-tracked MLSysIM Literature series via a guarded LEGO cell.
- mlsysim: series for Epoch AI models by domain, Open X-Embodiment
composition, LeRobot Hub growth and episode histogram, MLPerf Inference
Edge (Orin), KITTI odometry, California DMV disengagements, and Waymo
safety-impact headline rows
- bib: KITTI, Epoch AI models data, Waymo safety hub, California DMV
disengagement reports via the reference gateway
- vol4: figures in the boundary, brain, data (2), memory, intervention,
and release chapters
- New Literature.Series.AcceleratorPowerRatings: 14 published package
power ratings (TDP, board power, or TGP as each vendor states them),
2012-2025, each row tied to its primary source; values match the
hardware registry wherever it carries the part, except MI250X
(series 560 W per the CDNA 2 white paper, registry 500 W).
- New Literature.Series.AISupercomputers: Epoch AI supercomputer snapshot
(CC BY 4.0), 475 systems with size, chips, and facility power.
- New fig-chip-vs-cluster-power in power and thermal envelopes: the
largest system held at 20-30 MW through 2021, then rose 9x from
Frontier to xAI Colossus while one package's rating doubled.
- Ten source bib entries added through the gateway.
Thirteen matplotlib figures across ten chapters, each with lead-in,
caption, alt text, and payoff prose; every plotted and quoted value comes
from a provenance-tracked MLSysIM Literature series via a guarded LEGO cell.
- mlsysim: literature/series.py adds CSV-backed series (METR time horizon,
Epoch inference prices, Terminal-Bench 2.0 scaffolds, Fiction.LiveBench,
Epoch SWE-bench Verified runs, Cybench, OSWorld step budgets, FrontierSWE,
SWE-bench trajectories, Large Language Monkeys, AgentDojo)
- mlsysim: Mixtral-8x7B gains kv_heads 8 from its published config; KV
sizing had fallen back to 32 heads and overstated the cache 4x
- bib: METR, Cybench, Terminal-Bench, Epoch hub and price insight,
FrontierSWE, Fiction.LiveBench via the reference gateway
- vol3: figures in introduction, test-time compute, context engineering,
KV cache, sandboxes, security, harness (2), evaluation (2), fine-tuning,
and agent economics (2)
- New Literature.Series.InferenceXDeepSeekR1: 325 measured serving
configurations (DeepSeek-R1, FP8, 1k/1k) from SemiAnalysis InferenceX,
Apache-2.0, each row linked to its benchmark run.
- New fig-throughput-interactivity in the batching efficiency-curve
section: per-GPU output against per-user interactivity, one frontier
per accelerator, with the 30x B200 span and the 44x GB200/H100 gap at
50 tok/s/user set against the 2.3x peak FP8 ratio.
- Bib entry semianalysis2026inferencex added through the gateway.
- Literature.ComputeTrend re-pinned to Epoch AI Notable AI Models; the
pre-2020 rows were 1.5-67x too high and ResNet-50 had no source.
- New Literature.AlgorithmicEfficiency from Hernandez and Brown (2020),
Table 3 (compute to AlexNet-level accuracy).
- New literature/series.py: Literature.Series, provenance-carrying CSV
snapshots behind the trend figures (Rupp microprocessor data, OWID
storage prices, Northcutt label errors, Epoch models/hardware/MMLU,
pre-training tokens, public-text stock, Wild-Time, PyTorch, TensorFlow,
XLA, ONNX, and JAX operator counts).
- Hardware: K80 (per GPU) and MI325X added; B200 FP32 and TF32 rates.
- Models: optional release_year field; VGG-16 added; years for the
memory-wall models.
- Bib entries for the new sources added through the reference gateway.
- Ranking-service drift single-sourced in RankingDriftScenario (94 ->
88 percent over four weeks); the retraining-interval example now uses
the decay that slide implies (0.24 percent/day, not 2 percent/day,
which would have reached 54 percent by week four). The optimum moves
from one day to three; prose says so and notes that fourfold traffic
brings it to the table's daily cadence. Node-day figure and its cell
no longer assume a daily run. Quiz item recomputed in chapter macros.
- Tool catalog compacted to the property plus one example (DVC,
Airflow, GitHub Actions, Kubeflow, Terraform); ranking walkthrough
keeps MLflow only.
- Fraud cost curve labeled as a separate illustrative case with sharper
asymmetry than the ranking service.
- CUPED cited (deng2013improving, added and restyled); slice table rows
read from their cell; metaphors ("memory price", "price of
staleness"), footnote triple hyphen, table em-dash; appendix pointer
wording.
- Quiz: Silent Failure Q0 rebound to the ranking service's slide;
maturity question no longer uses the absent "over the wall" phrase.
- The composite edge board's accelerator is no longer named EdgeTPU:
invented 2 ms / 500 mW values sat under a real product name. Callout,
table, caption, and cell renamed (edge accelerator); the board's
2 ms / 15 ms latencies now set once in MobileNetLighthouseSetup and
read by the trade-off and speedup cells. Real Edge TPU mentions
(footnote, Pixel 4 figure, TOPS registry) unchanged.
- Generic smart-camera, doorbell, and fraud examples replaced with the
chapter's camera trap (procurement, thermal callout, duty cycle,
rare-species recall and class balance).
- CMOS power derivation (dynamic + leakage, cubic approximation) moved to
ch11's power and thermal management section; ch12 keeps the two
measurement consequences and Koomey's law with a pointer.
- ECE cited (Naeini et al. 2015, added via binder bib add, restyled).
- Citations that pandoc swallowed after a cross-reference ("@ref [@cite]")
moved onto their claims here and in ch11 (Jacob, Jouppi,
Micikevicius/Mellempudi, the last now on the training recipe).
- "the book's" for lighthouse labels; 22 em-dash sites rewritten.
- Fix the sentence the HTML build truncated (\mbox dropped ResNet-50).
- GPT-2 attention notebook: show the FFN term, readable totals, and say
why its 320.8 TFLOP step exceeds the 6PD rule's 295 TFLOP.
- Open Scaling Training Systems with why one accelerator is not enough
and what scaling costs, ahead of the historical lineage.
- Open the prefetching section on ResNet-50: overlap lifts the H100 from
about 36 to 55 percent busy and cannot raise host supply.
- Say 80 GB (85.9 GB decimal) where computed differences use bytes;
hundreds of millions, not millions, of MACs for the 7x7 example.
- Register: the book's (not the volume's), no ledger/priced/contract
framing, no @Fig-X diagrams announcers, em-dashes removed in prose.
- Cite Polyak (momentum) and SGDR (cosine annealing); quiz uses the
H100 ridge and drops communication tax; three guards bound their claims.
Neocognitron, im2col (Chellapilla), Zeiler-Fergus, Bengio 1994, Werbos BPTT,
Highway Networks, sparse transformers, mixtures of experts, Robbins-Monro,
Chow reject option, Wald, Glorot 2011, SVMs, production DLRM, Neurosurgeon,
distributed DNNs, MFCC, covariate shift, extreme clicking, Cityscapes,
geodiversity, Hey Siri, Montreal Forced Aligner, Google DR post; all added
through the gateway and restyled to house style with resealed hashes.
CrowdFlower note corrected.
Vijay Janapa Reddi