fix: remove incorrect hair dryer power comparison in footnote

This commit is contained in:
Vijay Janapa Reddi
2026-01-02 10:33:09 -05:00
parent 40feb719e7
commit 1e88b9ca83

View File

@@ -302,7 +302,7 @@ Consider modern language models like GPT-4 or image generation systems like DALL
These scale requirements reveal a technical reality: the primary constraint in modern ML systems is not compute capacity but memory bandwidth[^fn-memory-bandwidth], the rate at which data can move between storage and processing units. This memory wall represents the primary bottleneck that determines system performance. Modern ML systems are memory bound, with matrix multiply operations achieving only 1-10% of theoretical peak FLOPS because processors spend most of their time waiting for data rather than computing. Moving 1GB from DRAM costs approximately 1000x more energy than a 32-bit multiply operation, making data movement the dominant factor in both performance and energy consumption. Amdahl's Law[^fn-amdahls-law] quantifies this fundamental limitation: if data movement consumes 80% of execution time, even infinite compute capacity provides only 1.25x speedup (since only the remaining 20% can be accelerated). This memory wall drives all modern architectural innovations, from in-memory computing and near-data processing to specialized accelerators that co-locate compute and storage elements. These system-scale challenges represent core engineering problems that this book explores systematically.
[^fn-thermal-power-constraints]: **Thermal and Power Constraints**: The physical limits imposed by heat generation and power consumption in computing hardware. Modern GPUs consume 300-700W each (equivalent to 3-7 hair dryers running continuously) and generate enormous heat that must be removed via sophisticated cooling systems. A single AI training cluster with 1,000 GPUs consumes 300-700 kW of power just for computation, plus 30-50% more for cooling, totaling ~1MW—equivalent to powering 750 homes. Data centers hit thermal density limits: you can only pack so many hot chips together before cooling becomes impossible or prohibitively expensive. These constraints drive hardware design choices (chip architectures optimized for performance-per-watt), infrastructure decisions (liquid cooling vs. air cooling), and economic trade-offs (power costs can exceed hardware costs over 3-year lifespans). Power/thermal management explains many ML system architecture decisions, from edge deployment to model compression.
[^fn-thermal-power-constraints]: **Thermal and Power Constraints**: The physical limits imposed by heat generation and power consumption in computing hardware. Modern GPUs consume 300-700W each and generate enormous heat that must be removed via sophisticated cooling systems. A single AI training cluster with 1,000 GPUs consumes 300-700 kW of power just for computation, plus 30-50% more for cooling, totaling ~1MW—equivalent to powering 750 homes. Data centers hit thermal density limits: you can only pack so many hot chips together before cooling becomes impossible or prohibitively expensive. These constraints drive hardware design choices (chip architectures optimized for performance-per-watt), infrastructure decisions (liquid cooling vs. air cooling), and economic trade-offs (power costs can exceed hardware costs over 3-year lifespans). Power/thermal management explains many ML system architecture decisions, from edge deployment to model compression.
[^fn-memory-bandwidth]: **Memory Bandwidth**: The rate at which data can be transferred between memory and processors, measured in GB/s (gigabytes per second). Modern GPUs like the H100 provide ~3TB/s memory bandwidth, while CPUs typically provide 100-200 GB/s. This seemingly large number becomes the bottleneck for ML workloads: a transformer model with 70 billion parameters requires 140GB just to store weights, taking 47ms to load at 3TB/s before any computation begins. The bandwidth constraint explains why ML accelerators focus on higher bandwidth memory (HBM) rather than just faster compute units. For comparison, arithmetic operations are relatively cheap: a GPU can perform trillions of multiply-add operations in the time it takes to move 1GB from memory, creating a fundamental tension where processors spend more time waiting for data than computing.