mirror of
https://github.com/harvard-edge/cs249r_book.git
synced 2026-07-18 08:54:25 -05:00
fix(hw_acceleration): merge math-anchored multiplier into single span
$10^2$$\times$ produced two adjacent math spans rendering with a visible seam at print scale. Per book-prose.md $2 (math-anchored multiplier exception, added 2026-05-05), include \times inside the same span as the power: $10^2\times$.
This commit is contained in:
@@ -749,7 +749,7 @@ Machine learning constitutes a computational domain with unique characteristics
|
||||
Machine learning computational requirements reveal limitations in traditional processors. CPUs reach only `{python} CpuMlInefficiency.cpu_utilization_min_str`–`{python} CpuMlInefficiency.cpu_utilization_max_str` percent utilization on neural network workloads, delivering approximately `{python} CpuMlInefficiency.cpu_gflops_str` GFLOPS (billions of floating-point operations per second) while consuming hundreds of watts. This inefficiency results from architectural mismatches: CPUs optimize for single-thread performance and irregular memory access, while neural networks require massive parallelism and predictable data streams. The memory bandwidth constraint compounds the problem: a single neural network layer may require accessing gigabytes of parameters, overwhelming CPU cache hierarchies designed for kilobyte-scale working sets.
|
||||
|
||||
\index{Energy!data movement cost}
|
||||
The energy economics of data movement influence accelerator design. Accessing data from DRAM can consume on the order of $10^2$$\times$ more energy than a multiply-accumulate operation (exact values vary by technology node and design), making minimizing data movement a primary optimization target. This disparity helps explain the progression from repurposed graphics processors to purpose-built neural network accelerators. TPUs and other custom accelerators can sustain high utilization on dense kernels by implementing systolic arrays and other architectures that maximize data reuse while minimizing movement.
|
||||
The energy economics of data movement influence accelerator design. Accessing data from DRAM can consume on the order of $10^2\times$ more energy than a multiply-accumulate operation (exact values vary by technology node and design), making minimizing data movement a primary optimization target. This disparity helps explain the progression from repurposed graphics processors to purpose-built neural network accelerators. TPUs and other custom accelerators can sustain high utilization on dense kernels by implementing systolic arrays and other architectures that maximize data reuse while minimizing movement.
|
||||
|
||||
\index{Training vs. Inference!accelerator design}
|
||||
Training and inference present distinct computational profiles that influence accelerator design. Training requires high-precision arithmetic (FP32 or FP16) [@ieee_754_2019]\index{FP16!training precision}\index{FP32!gradient computation} for gradient computation and weight updates, bidirectional data flow for backpropagation\index{Backpropagation!memory requirements} (see @sec-model-training for activation memory analysis), and large memory capacity for storing activations. Inference can exploit reduced precision (INT8 or INT4), requires only forward computation, and prioritizes latency over throughput[^fn-latency-throughput-hw]. These differences drive specialized architectures: training accelerators maximize FLOPS and memory bandwidth, while inference accelerators optimize for energy efficiency and deterministic latency.
|
||||
|
||||
Reference in New Issue
Block a user