fix(prose): lowercase concept terms in body prose (6 sites)

Per book-prose.md $10.3, concept terms (data gravity, transformer
architecture) lowercase in body prose. Capitals reserved for sentence
start, bold first definitions, headings, callout titles, \index{}
entries, and proper model names like Vision Transformer.

Sites:
- vol1/data_engineering:351 'Data Gravity establishes' -> 'data gravity'
- vol1/conclusion:769 'Data Gravity Invariant' -> 'data gravity invariant'
- vol2/distributed_training:2049 'every Transformer block' -> 'transformer'
- vol2/network_fabrics:1078 'primarily Transformer training' -> 'transformer'
- vol2/sustainable_ai:1833 'a Transformer model' -> 'a transformer model'
- vol2/performance_engineering:737 'one Transformer layer' -> 'transformer'

Note: 8 'Transformers' instances in nn_architectures.qmd were initially
flagged but are all sentence-start (grammatical caps required), not
violations.

book/quarto/contents/vol1/conclusion/conclusion.qmd

@@ -766,7 +766,7 @@ Component expertise is necessary but insufficient. An engineer who understands d
 
 **Fallacy:** *More data always improves model quality.*
 
-The intuition that more data yields better models is seductive because it holds true early in model development. @sec-data-selection demonstrated the diminishing returns that set in once a dataset achieves sufficient coverage: beyond that threshold, doubling dataset size yields marginal accuracy gains while doubling storage, preprocessing, and labeling costs. The Data Gravity Invariant (Principle \ref{pri-data-gravity}) ensures that data scale decisions cascade through every downstream system, because larger datasets demand proportionally more I/O bandwidth, longer preprocessing pipelines, and costlier feature stores. The engineer who scales data without measuring the incremental return per additional sample optimizes the wrong variable.
+The intuition that more data yields better models is seductive because it holds true early in model development. @sec-data-selection demonstrated the diminishing returns that set in once a dataset achieves sufficient coverage: beyond that threshold, doubling dataset size yields marginal accuracy gains while doubling storage, preprocessing, and labeling costs. The data gravity invariant (Principle \ref{pri-data-gravity}) ensures that data scale decisions cascade through every downstream system, because larger datasets demand proportionally more I/O bandwidth, longer preprocessing pipelines, and costlier feature stores. The engineer who scales data without measuring the incremental return per additional sample optimizes the wrong variable.
 
 **Fallacy:** *A single accuracy metric captures model quality.*
 

book/quarto/contents/vol1/data_engineering/data_engineering.qmd

@@ -348,7 +348,7 @@ class FeedingProblem:
 
 ### The feeding problem: Flow rate and the "feeding tax" {#sec-data-engineering-feeding-problem}
 
-Data Gravity establishes the cost of moving the entire mass; **The Feeding Problem**\index{Feeding Problem!data pipeline throughput} establishes the cost of delivering it. In the Hennessy & Patterson tradition, we analyze this as a **Flow Rate**\index{Flow Rate!data pipeline} problem: the struggle to saturate a high-throughput "Machine" from a low-bandwidth "Data" source.
+Data gravity establishes the cost of moving the entire mass; **The Feeding Problem**\index{Feeding Problem!data pipeline throughput} establishes the cost of delivering it. In the Hennessy & Patterson tradition, we analyze this as a **Flow Rate**\index{Flow Rate!data pipeline} problem: the struggle to saturate a high-throughput "Machine" from a low-bandwidth "Data" source.
 
 According to the **iron law**, the system is only as fast as its slowest term. If a modern accelerator can process `{python} FeedingProblem.img_per_sec_str` samples per second, but the storage pipeline delivers only `{python} FeedingProblem.disk_bw_mbs_str` MB/s, the expensive silicon sits idle. We quantify this as **The Feeding Tax**\index{Feeding Tax!I/O wait cost}: the wall-clock time lost to I/O wait, which directly reduces the **System Efficiency ($\eta_{\text{hw}}$)**\index{System Efficiency!feeding tax} term. For a standard cloud volume, the feeding tax can exceed **`{python} FeedingProblem.feeding_tax_pct_str` percent**, meaning the GPU spends the majority of its time waiting for bits. This tax transforms the Data Pipeline from a simple storage concern into the primary regulator of the system's Duty Cycle. To saturate a 300 TFLOPS processor, the pipeline must often sustain **`{python} FeedingProblem.req_bw_gbs_str` GB/s** transfer rates—a requirement that forces the shift from traditional file systems to the specialized storage architectures we examine in @sec-data-engineering-ml-storage-systems-architecture-options-67fa.
 

book/quarto/contents/vol2/distributed_training/distributed_training.qmd

@@ -2046,7 +2046,7 @@ The memory budget for training a 175-billion parameter model is dominated by mod
 
 ### Communication analysis {#sec-distributed-training-systems-systems-hybrid-parallelism-communication-analysis}
 
-Each parallelism dimension imposes a distinct traffic profile on the network. Tensor Parallelism is the most chatty, requiring two `AllReduce` operations for every Transformer block (one for the Attention projection, one for the MLP) in both the forward and backward passes. These messages are relatively small but occur thousands of times per step, making them strictly latency-bound and necessitating NVLink. Pipeline Parallelism, in contrast, involves point-to-point transfers of activation tensors (size $B_{\mu} \times S \times H$) only at the boundaries of the pipeline stages. While these messages are moderate in size, they occur less frequently, making them manageable over standard InfiniBand links. Data Parallelism generates the largest burst of traffic, requiring a global `AllReduce` of the entire 350 GB gradient buffer. However, this communication occurs only once per global batch update. By using gradient bucketing to overlap this transmission with the compute-intensive backward pass, the effective cost of DP communication can often be hidden, provided the cluster maintains sufficient cross-sectional bandwidth.
+Each parallelism dimension imposes a distinct traffic profile on the network. Tensor Parallelism is the most chatty, requiring two `AllReduce` operations for every transformer block (one for the Attention projection, one for the MLP) in both the forward and backward passes. These messages are relatively small but occur thousands of times per step, making them strictly latency-bound and necessitating NVLink. Pipeline Parallelism, in contrast, involves point-to-point transfers of activation tensors (size $B_{\mu} \times S \times H$) only at the boundaries of the pipeline stages. While these messages are moderate in size, they occur less frequently, making them manageable over standard InfiniBand links. Data Parallelism generates the largest burst of traffic, requiring a global `AllReduce` of the entire 350 GB gradient buffer. However, this communication occurs only once per global batch update. By using gradient bucketing to overlap this transmission with the compute-intensive backward pass, the effective cost of DP communication can often be hidden, provided the cluster maintains sufficient cross-sectional bandwidth.
 
 ### Pipeline bubble {#sec-distributed-training-systems-systems-hybrid-parallelism-pipeline-bubble}
 

book/quarto/contents/vol2/network_fabrics/network_fabrics.qmd

@@ -1075,7 +1075,7 @@ A **Dragonfly**[^fn-dragonfly-isca] topology organizes switches into high-radix
 
 A **Torus** topology connects each node directly to its neighbors in a multidimensional grid, most commonly a 3D torus where every node links to its six adjacent peers (up/down, left/right, front/back). The design offers full local bandwidth with minimal switching hardware, as connections travel only 1–2 hops to reach neighbors. However, global communication requires traversing the diameter of the mesh ($O(N^{1/3})$ hops), making latency scale poorly with cluster size. Google adopted this architecture for its Tensor Processing Unit (TPU) pods because Transformer training is dominated by data parallelism and pipeline parallelism, both of which use nearest-neighbor communication patterns that map naturally onto the physical grid. The limitation becomes apparent with Mixture-of-Experts (MoE), which relies on AllToAll communication patterns. On a torus, these random permutations congest the limited bisection bandwidth of the mesh, causing performance to degrade by 2–4$\times$ compared to a switch-based fat-tree.
 
-The trade-offs between these topologies become stark when quantified for a large-scale deployment. Consider a 4,096-GPU cluster. A non-blocking fat-tree requires approximately 2,048 switches and 40,000 optical cables to deliver 100 percent bisection bandwidth, enabling any GPU to communicate with any other at full speed. A 3D torus connecting the same nodes might use zero external switches (relying on direct host-to-host links) and only short copper cables, but offers only a fraction of the bisection bandwidth (scaling with $N^{2/3}$). The divergence in industry architectures follows directly. Google's TPU Pods employ torus topologies because their workloads, primarily Transformer training, are predictable and dominated by nearest-neighbor communication that maps perfectly to the 3D grid. GPU clusters overwhelmingly favor fat-trees because their workloads are more diverse, ranging from recommendation systems to graph neural networks, and rely heavily on global AllReduce patterns that require the full bisection bandwidth only a tree can provide. A torus saves millions in switch costs but rigidly constrains the software; a fat-tree costs more but provides the universality needed for general-purpose AI research.
+The trade-offs between these topologies become stark when quantified for a large-scale deployment. Consider a 4,096-GPU cluster. A non-blocking fat-tree requires approximately 2,048 switches and 40,000 optical cables to deliver 100 percent bisection bandwidth, enabling any GPU to communicate with any other at full speed. A 3D torus connecting the same nodes might use zero external switches (relying on direct host-to-host links) and only short copper cables, but offers only a fraction of the bisection bandwidth (scaling with $N^{2/3}$). The divergence in industry architectures follows directly. Google's TPU Pods employ torus topologies because their workloads, primarily transformer training, are predictable and dominated by nearest-neighbor communication that maps perfectly to the 3D grid. GPU clusters overwhelmingly favor fat-trees because their workloads are more diverse, ranging from recommendation systems to graph neural networks, and rely heavily on global AllReduce patterns that require the full bisection bandwidth only a tree can provide. A torus saves millions in switch costs but rigidly constrains the software; a fat-tree costs more but provides the universality needed for general-purpose AI research.
 
 @fig-network-topologies illustrates the structural differences between these common families.
 

book/quarto/contents/vol2/performance_engineering/performance_engineering.qmd

@@ -734,7 +734,7 @@ Intermediate tensors $Z_1$ and $Z_2$ each occupy the same memory as the output $
 
 As @fig-fusion-before-after makes concrete, fusion reduces HBM transfers from seven to three per layer, eliminating 64 MB of redundant intermediate traffic that the unfused path forces through off-chip memory.
 
-In a naive implementation without operator fusion, executing one Transformer layer requires roughly 50 separate kernel launches. If each launch incurs a 10-microsecond overhead, the system spends 500 microseconds purely on dispatch latency. If the actual arithmetic execution of the layer takes only 2 milliseconds, the launch overhead consumes 20 percent of the total wall-clock time, leaving the GPU compute units idle for one-fifth of the inference cycle. This "launch-bound" regime limits the benefits of faster hardware; doubling the GPU's FLOPS does nothing to reduce the 500-microsecond fixed cost. Operator fusion addresses this by compiling these 50 discrete operations into a small handful of fused kernels, often reducing the count to 5--10 launches, thereby reclaiming the lost cycles and shifting the workload back toward a compute-bound profile.
+In a naive implementation without operator fusion, executing one transformer layer requires roughly 50 separate kernel launches. If each launch incurs a 10-microsecond overhead, the system spends 500 microseconds purely on dispatch latency. If the actual arithmetic execution of the layer takes only 2 milliseconds, the launch overhead consumes 20 percent of the total wall-clock time, leaving the GPU compute units idle for one-fifth of the inference cycle. This "launch-bound" regime limits the benefits of faster hardware; doubling the GPU's FLOPS does nothing to reduce the 500-microsecond fixed cost. Operator fusion addresses this by compiling these 50 discrete operations into a small handful of fused kernels, often reducing the count to 5--10 launches, thereby reclaiming the lost cycles and shifting the workload back toward a compute-bound profile.
 
 ### Fusion categories {#sec-performance-engineering-fusion-categories}
 

book/quarto/contents/vol2/sustainable_ai/sustainable_ai.qmd

@@ -1830,7 +1830,7 @@ Infrastructure sharing presents efficiency opportunities often overlooked in sus
 
 #### AI energy consumption compared to other industries {#sec-sustainable-ai-ai-energy-consumption-compared-industries-3d5d}
 
-The environmental impact of AI workloads has emerged as a concern, with carbon emissions approaching levels comparable to established carbon-intensive sectors. Research demonstrates that training a single large AI model generates carbon emissions equivalent to multiple passenger vehicles over their complete lifecycle [@strubell2019energy]. To contextualize AI's environmental footprint, @fig-carbonfootprint compares the carbon emissions of large-scale machine learning tasks to transcontinental flights, illustrating the energy demands of training and inference workloads. It shows a comparison from lowest to highest carbon footprints, starting with a roundtrip flight between NY and SF, human life average per year, American life average per year, US car including fuel over a lifetime, and a Transformer model with neural architecture search[^fn-nas-carbon-cost], which has the highest footprint. These comparisons underscore the need for more sustainable AI practices to mitigate the industry's carbon impact.
+The environmental impact of AI workloads has emerged as a concern, with carbon emissions approaching levels comparable to established carbon-intensive sectors. Research demonstrates that training a single large AI model generates carbon emissions equivalent to multiple passenger vehicles over their complete lifecycle [@strubell2019energy]. To contextualize AI's environmental footprint, @fig-carbonfootprint compares the carbon emissions of large-scale machine learning tasks to transcontinental flights, illustrating the energy demands of training and inference workloads. It shows a comparison from lowest to highest carbon footprints, starting with a roundtrip flight between NY and SF, human life average per year, American life average per year, US car including fuel over a lifetime, and a transformer model with neural architecture search[^fn-nas-carbon-cost], which has the highest footprint. These comparisons underscore the need for more sustainable AI practices to mitigate the industry's carbon impact.
 
 [^fn-nas-carbon-cost]: **Neural Architecture Search (NAS) Carbon Cost**: The 284,000 kg CO₂ figure from @strubell2019energy represents evaluating 12,800 architecture configurations, equivalent to the annual emissions of 140 average Americans. This extreme cost catalyzed efficient NAS research: weight-sharing methods like DARTS reduced search cost by 1,000$\times$, demonstrating that the meta-optimization of *how* we search for architectures is itself a sustainability lever. \index{Neural Architecture Search!carbon cost}