fix: align volume two math notation

Resolve print-facing notation collisions in Volume 2 so symbols for parameters, bandwidth, loss, and critical batch size match the shared notation contract.

book/quarto/contents/vol2/backmatter/appendix_assumptions.qmd

@@ -354,7 +354,7 @@ The distributed systems reasoning in this book builds upon the single-machine pe
 | **Memory**  | HBM3 Capacity     | `{python} H100Recap.cap_gb_str` GB          |
 | **Thermal** | TDP               | `{python} H100Recap.tdp_w_str` W            |
 
-: **Single-Node Foundational Constants**. Recapping the hardware specifications for the H100 accelerator. These values provide the $R_{\text{peak}}$ and $BW$ baselines used in the iron law calculations throughout this volume. {#tbl-fleet-foundation-recap}
+: **Single-Node Foundational Constants**. Recapping the hardware specifications for the H100 accelerator. These values provide the $R_{\text{peak}}$ and $\text{BW}$ baselines used in the iron law calculations throughout this volume. {#tbl-fleet-foundation-recap}
 
 ## Cluster Reference Configurations {.unnumbered}
 

book/quarto/contents/vol2/backmatter/appendix_fleet.qmd

@@ -246,7 +246,7 @@ Training workloads are synchronous and bandwidth-hungry like HPC, but long-runni
 
 The following provides a compact reference for the key foundational ideas that reappear throughout the distributed systems chapters.
 
-- **The iron law** ($T \approx D_{\text{vol}}/BW + O/(R_{\text{peak}} \cdot \eta) + L_{\text{lat}}$): Performance is bounded by data movement or compute. At fleet scale, the data movement term expands to include inter-node communication, not just memory bandwidth.
+- **The iron law** ($T \approx D_{\text{vol}}/\text{BW} + O/(R_{\text{peak}} \cdot \eta) + L_{\text{lat}}$): Performance is bounded by data movement or compute. At fleet scale, the data movement term expands to include inter-node communication, not just memory bandwidth.
 - **Roofline Model**: Distinguishes compute-bound from memory-bound workloads using arithmetic intensity. At fleet scale, a third ceiling appears: **network-bound** workloads whose performance is limited by inter-node bandwidth.
 - **Amdahl's Law**: Caps strong-scaling speedup at $1/s$. At fleet scale, the "serial fraction" includes not just sequential code but also synchronization barriers, collective communication, and pipeline bubbles.
 - **Training memory rule**: 16 bytes per parameter for mixed-precision Adaptive Moment Estimation (Adam) training. At fleet scale, this determines how model state is partitioned across nodes (ZeRO, tensor parallelism, pipeline parallelism).

book/quarto/contents/vol2/backmatter/appendix_systems.qmd

@@ -51,7 +51,7 @@ The most significant constraint in ML systems is the gap between logic speed and
 
 To reason quantitatively about any ML system, we use the **iron law**, which decomposes the time $T$ required for an operation into three physical terms:
 
-$$T = \underbrace{\frac{D_{\text{vol}}}{BW}}_{\text{Data Movement}} + \underbrace{\frac{O}{R_{\text{peak}}}}_{\text{Calculation}} + \underbrace{L_{\text{lat}}}_{\text{Fixed Latency}}$$
+$$T = \underbrace{\frac{D_{\text{vol}}}{\text{BW}}}_{\text{Data Movement}} + \underbrace{\frac{O}{R_{\text{peak}} \cdot \eta}}_{\text{Calculation}} + \underbrace{L_{\text{lat}}}_{\text{Fixed Latency}}$$
 
 Every optimization technique in this book targets one of these terms:
 

book/quarto/contents/vol2/collective_communication/collective_communication.qmd

@@ -2168,7 +2168,7 @@ In **Fault Tolerance** (@sec-fault-tolerance-reliability), we examine how to mai
 
 [^fn-allreduce-etymology]: **AllReduce**: A compound term from MPI (Message Passing Interface) indicating a **Reduce** operation (summing data from all nodes to one) followed by a **Broadcast** (sending the sum to all nodes). Ring-based AllReduce optimizes this by performing both phases simultaneously in $2(N-1)$ steps, ensuring that every GPU ends with the global sum without any single node becoming a bottleneck. \index{AllReduce!etymology}
 
-[^fn-zerocopy-rdma]: **Zero-Copy Communication**: RDMA (Remote Direct Memory Access) allows the NIC to transfer data directly from the application's memory on one node to the application's memory on another, bypassing the operating system's kernel buffers. For a 350 GB gradient exchange, zero-copy avoids moving 700 GB of data between the CPU and main memory, reclaiming significant memory bandwidth ($BW$). \index{Zero-Copy!RDMA}
+[^fn-zerocopy-rdma]: **Zero-Copy Communication**: RDMA (Remote Direct Memory Access) allows the NIC to transfer data directly from the application's memory on one node to the application's memory on another, bypassing the operating system's kernel buffers. For a 350 GB gradient exchange, zero-copy avoids moving 700 GB of data between the CPU and main memory, reclaiming significant memory bandwidth ($\text{BW}$). \index{Zero-Copy!RDMA}
 
 [^fn-speed-of-light-latency]: **The Speed of Light Constraint**: Light travels through optical fiber at approximately 200,000 km/s, or 200 meters per microsecond. In a massive datacenter where cables between racks span 100 meters, the "wire delay" alone contributes 500 ns to every message—a physical limit that no amount of better networking hardware can reduce. \index{Speed of Light!latency bound}
 

book/quarto/contents/vol2/compute_infrastructure/compute_infrastructure.qmd

@@ -847,9 +847,9 @@ The equation has a direct physical interpretation. If the workload's arithmetic
 
 ***ridge point***\index{Ridge Point!definition} is the specific arithmetic intensity where the memory bandwidth ceiling meets the compute ceiling ($I_{\text{ridge}} = \frac{R_{\text{peak}}}{BW}$).
 
-1.  **Significance (Quantitative):** It defines the **Hardware Efficiency Threshold**. Workloads with an intensity below the ridge point are **Bandwidth-Bound** ($BW$), while those above are **Compute-Bound**\index{Compute-Bound} ($R_{\text{peak}}$).
+1.  **Significance (Quantitative):** It defines the **Hardware Efficiency Threshold**. Workloads with an intensity below the ridge point are **Bandwidth-Bound** ($\text{BW}$), while those above are **Compute-Bound**\index{Compute-Bound} ($R_{\text{peak}}$).
 2.  **Distinction (Durable):** Unlike **Peak FLOPs**\index{Peak FLOPS} (which only describes the horizontal ceiling), the ridge point describes the **Balance** of the architecture. A rising ridge point over hardware generations indicates that compute is growing faster than bandwidth, making utilization harder.
-3.  **Common Pitfall:** A frequent misconception is that all GPUs have the same ridge point. In reality, it varies by **Precision**: because $R_{\text{peak}}$ is higher for INT8 than FP32 while $BW$ is constant, the ridge point for INT8 is much higher, requiring more data reuse to saturate the hardware.
+3.  **Common Pitfall:** A frequent misconception is that all GPUs have the same ridge point. In reality, it varies by **Precision**: because $R_{\text{peak}}$ is higher for INT8 than FP32 while $\text{BW}$ is constant, the ridge point for INT8 is much higher, requiring more data reuse to saturate the hardware.
 
 :::
 
@@ -1685,7 +1685,7 @@ Transfer time is $T = \text{Data} / \text{Bandwidth}$.
 
 ***Bandwidth Hierarchy***\index{Bandwidth Hierarchy!definition} is the physical ordering of data transfer rates across system boundaries, from on-chip SRAM (fastest) to the wide-area network (slowest).
 
-1.  **Significance (Quantitative):** It dictates the **Scaling Ceiling** for distributed training. At each physical boundary (die edge, package edge, chassis, rack), the **Effective Bandwidth ($BW$)** drops by approximately one order of magnitude while latency ($L_{\text{lat}}$) increases.
+1.  **Significance (Quantitative):** It dictates the **Scaling Ceiling** for distributed training. At each physical boundary (die edge, package edge, chassis, rack), the **Effective Bandwidth ($\text{BW}$)** drops by approximately one order of magnitude while latency ($L_{\text{lat}}$) increases.
 2.  **Distinction (Durable):** Unlike **Idealized Networking Models**\index{Idealized Networking Models}, the Bandwidth Hierarchy captures the **Physical Cost of Distance**\index{Physical Cost of Distance}: shorter signal paths over denser wiring achieve higher throughput at lower energy per bit.
 3.  **Common Pitfall:** A frequent misconception is that all cluster communication is equal. In reality, parallelism strategies must be **Hierarchy-Aware**\index{Hierarchy-Aware}: placing high-frequency synchronization (for example, Tensor Parallelism) on a slow tier will rapidly idle the compute units ($R_{\text{peak}}$), collapsing system efficiency ($\eta$).
 
@@ -2549,7 +2549,7 @@ where $T_N$ is the training time on $N$ nodes. An efficiency of 1.0 means perfec
 ***Scaling Efficiency ($\eta$)***\index{Scaling Efficiency!definition} is the ratio of actual throughput to ideal linear throughput when increasing the number of compute nodes ($N$).
 
 1.  **Significance (Quantitative):** It is the most important metric for cluster productivity ($\eta = T_1 / (N \cdot T_N)$). A scaling efficiency of 0.50 means that a 10,000-GPU cluster is delivering only the same useful work as a 5,000-GPU cluster, wasting 50 percent of the hardware investment.
-2.  **Distinction (Durable):** Unlike **Single-Node Efficiency** (which captures local bottlenecks like $BW$), Scaling Efficiency captures the **Cluster-Level Overhead** of communication ($L_{\text{lat}}$) and synchronization.
+2.  **Distinction (Durable):** Unlike **Single-Node Efficiency** (which captures local bottlenecks like $\text{BW}$), Scaling Efficiency captures the **Cluster-Level Overhead** of communication ($L_{\text{lat}}$) and synchronization.
 3.  **Common Pitfall:** A frequent misconception is that scaling efficiency is constant. In reality, it is a **Function of Problem Size**\index{Function of Problem Size}: as $N$ increases, the communication-to-compute ratio typically worsens (Amdahl's Law), making it harder to maintain high efficiency for small models.
 
 :::

book/quarto/contents/vol2/data_storage/data_storage.qmd

@@ -727,7 +727,7 @@ Local NVMe provides high bandwidth and low latency within a single node, but dis
 
 ***Parallel File System (PFS)***\index{Parallel File System!definition} is a distributed storage architecture that stripes data across many storage servers to provide aggregate throughput exceeding the capacity of any single device.
 
-1.  **Significance (Quantitative):** A PFS aggregates $BW_{io}$ linearly with the number of storage servers (Object Storage Servers). A Lustre cluster with 20 OSS nodes each delivering 10 GB/s provides 200 GB/s aggregate—vs. a single NAS server capped at 10 GB/s—enabling an 8,000-GPU training job to load each 512-token batch in under 50 ms rather than 1 second. This aggregate bandwidth directly reduces the $D_{\text{vol}}/BW$ term in the iron law.
+1.  **Significance (Quantitative):** A PFS aggregates $BW_{io}$ linearly with the number of storage servers (Object Storage Servers). A Lustre cluster with 20 OSS nodes each delivering 10 GB/s provides 200 GB/s aggregate—vs. a single NAS server capped at 10 GB/s—enabling an 8,000-GPU training job to load each 512-token batch in under 50 ms rather than 1 second. This aggregate bandwidth directly reduces the $D_{\text{vol}}/\text{BW}$ term in the iron law.
 2.  **Distinction (Durable):** Unlike Network Attached Storage (NAS), where every I/O request routes through a single server, a PFS client receives stripe location metadata from a dedicated Metadata Server (MDS) and then reads data directly from multiple OSS nodes in parallel—the MDS and OSS paths are architecturally separated, so data bandwidth scales with OSS count while metadata operations scale with MDS count.
 3.  **Common Pitfall:** A frequent misconception is that a PFS has unlimited throughput if enough OSS nodes are added. In reality, a Lustre MDS handles roughly 100,000–300,000 metadata operations per second; at 10,000 workers each opening one small file, the MDS saturates in under 1 second and becomes the serialization point that idles the entire cluster regardless of how many OSS nodes are present.
 
@@ -1364,7 +1364,7 @@ The traditional data path for loading training data follows three hops: storage
 
 ***GPU Direct Storage (GDS)***\index{GPU Direct Storage!definition} is a technology that enables a direct DMA path between NVMe storage devices and GPU memory, bypassing the host CPU and system DRAM.
 
-1.  **Significance (Quantitative):** It eliminates the "Bounce Buffer" through system memory, reducing data loading latency ($L_{\text{lat}}$) and doubling the effective bandwidth ($BW$) for I/O-intensive workloads. It allows the GPU to saturate the NVMe link speed (for example, 7 GB/s) while reducing CPU utilization for I/O by up to 90 percent.
+1.  **Significance (Quantitative):** It eliminates the "Bounce Buffer" through system memory, reducing data loading latency ($L_{\text{lat}}$) and doubling the effective bandwidth ($\text{BW}$) for I/O-intensive workloads. It allows the GPU to saturate the NVMe link speed (for example, 7 GB/s) while reducing CPU utilization for I/O by up to 90 percent.
 2.  **Distinction (Durable):** Unlike **Traditional I/O**\index{Traditional I/O}, where every byte must be processed by the CPU and stored in kernel buffers, GDS provides **Direct Memory Access** between the storage controller and the accelerator.
 3.  **Common Pitfall:** A frequent misconception is that GDS makes all storage "faster." In reality, it only accelerates **Local or RDMA-attached NVMe**; it does not eliminate the physical latency of network-attached file systems or object storage.
 

book/quarto/contents/vol2/distributed_training/distributed_training.qmd

@@ -156,7 +156,7 @@ Third, **dataset scale** exceeds single-machine storage when training data reach
 
 Distributed training introduces three primary complexity dimensions not present in single-machine scenarios:
 
-1.  **Communication Overhead**\index{Communication Overhead}: The cost of synchronizing gradients. For a model with $N$ parameters distributed across $D$ devices, all-reduce operations must transfer approximately $2N(D-1)/D$ bytes per step. On commodity networks, this can dominate computation time.
+1.  **Communication Overhead**\index{Communication Overhead}: The cost of synchronizing gradients. For a model with $P$ parameters distributed across $N_{\text{dev}}$ devices, all-reduce operations must transfer approximately $2P(N_{\text{dev}}-1)/N_{\text{dev}}$ gradient values per step, multiplied by the bytes per gradient element. On commodity networks, this can dominate computation time.
 2.  **Fault Tolerance**\index{Fault Tolerance}: Requirements increase exponentially with cluster size. A 100-node cluster with 99.9 percent per-node reliability experiences failures every few hours.
 3.  **Algorithmic Stability**\index{Algorithmic Stability}: Large batch sizes from data parallelism affect convergence behavior, requiring learning rate scaling and warmup strategies that single-machine training does not require [@goyal2017accurate].
 
@@ -513,14 +513,14 @@ Data parallelism is most effective when the dataset size is large but the model
 
 The effectiveness of data parallelism stems from a property of stochastic gradient descent. Gradients computed on different minibatches can be averaged while preserving mathematical equivalence to single-device training. This property enables parallel computation across devices, with the mathematical foundation following directly from the linearity of expectation.
 
-Consider a model with parameters $θ$ training on a dataset $D$. The loss function for a single data point $x_i$ is $L(θ, x_i)$. In standard SGD with batch size $B$, the gradient update for a minibatch is:
+Consider a model with parameters $\theta$ training on a dataset $D$. The loss function for a single data point $x_i$ is $\mathcal{L}(\theta, x_i)$. In standard SGD with batch size $B$, the gradient update for a minibatch is:
 $$
-g = \frac{1}{B} \sum_{i=1}^B \nabla_θ L(θ, x_i)
+g = \frac{1}{B} \sum_{i=1}^B \nabla_{\theta} \mathcal{L}(\theta, x_i)
 $$
 
 In data parallelism with $N$ devices, each device $k$ computes gradients on its own minibatch $B_k$:
 $$
-g_k = \frac{1}{|B_k|} \sum_{x_i \in B_k} \nabla_θ L(θ, x_i)
+g_k = \frac{1}{|B_k|} \sum_{x_i \in B_k} \nabla_{\theta} \mathcal{L}(\theta, x_i)
 $$
 
 The global update averages these local gradients:
@@ -530,7 +530,7 @@ $$
 
 The averaging is mathematically equivalent to computing the gradient on the combined batch $B_{\text{total}} = \bigcup_{k=1}^N B_k$:
 $$
-g_{\text{global}} = \frac{1}{|B_{\text{total}}|} \sum_{x_i \in B_{\text{total}}} \nabla_θ L(θ, x_i)
+g_{\text{global}} = \frac{1}{|B_{\text{total}}|} \sum_{x_i \in B_{\text{total}}} \nabla_{\theta} \mathcal{L}(\theta, x_i)
 $$
 
 The equivalence shows *why* data parallelism maintains the statistical properties of SGD training: distributing distinct data subsets across devices, computing local gradients independently, and averaging them approximates the full-batch gradient.
@@ -1198,21 +1198,21 @@ Hardware efficiency metrics govern throughput, but convergence theory determines
 
 ### Convergence rate for synchronous data parallel SGD {#sec-distributed-training-systems-systems-convergence-rate-synchronous-data-parallel-sgd-2ed5}
 
-The fundamental convergence result for distributed SGD provides the theoretical basis for understanding parallel training. For a loss function $L(\theta)$ with $L$-Lipschitz gradients (smoothness condition) and variance-bounded stochastic gradients $\mathbb{E}[\|g_i - \nabla L(\theta)\|^2] \leq \sigma^2$, synchronous data parallel SGD with $N$ workers achieves the following convergence rate.
+The fundamental convergence result for distributed SGD provides the theoretical basis for understanding parallel training. For a loss function $\mathcal{L}(\theta)$ with $L_s$-Lipschitz gradients (smoothness condition) and variance-bounded stochastic gradients $\mathbb{E}[\|g_i - \nabla \mathcal{L}(\theta)\|^2] \leq \sigma^2$, synchronous data parallel SGD with $N$ workers achieves the following convergence rate.
 
 ::: {.callout-theorem title="Convergence Rate for Distributed SGD"}
 
 For synchronous data parallel SGD with $N$ workers, each computing gradients on a local batch of size $b$, after $M$ total iterations the expected optimization error satisfies:
 
 $$
-\mathbb{E}[L(\theta_M)] - L \leq \underbrace{\frac{L \|\theta_0 - \theta\|^2}{2M}}_{\text{optimization error}} + \underbrace{\frac{\eta L \sigma^2}{2Nb}}_{\text{variance floor}}
+\mathbb{E}[\mathcal{L}(\theta_M)] - \mathcal{L}^{\star} \leq \underbrace{\frac{L_s \|\theta_0 - \theta^{\star}\|^2}{2M}}_{\text{optimization error}} + \underbrace{\frac{\eta L_s \sigma^2}{2Nb}}_{\text{variance floor}}
 $$
 
-where $\eta$ is the learning rate, $L$ is the optimal loss, and $\sigma^2$ is the gradient variance. The effective convergence rate is $O(1/\sqrt{NbM})$ when the learning rate is tuned optimally as $\eta = O(\sqrt{Nb/M})$.
+where $\eta$ is the learning rate, $L_s$ is the smoothness constant, $\mathcal{L}^{\star}$ is the optimal loss value, $\theta^{\star}$ is an optimal parameter vector, and $\sigma^2$ is the gradient variance. The effective convergence rate is $O(1/\sqrt{NbM})$ when the learning rate is tuned optimally as $\eta = O(\sqrt{Nb/M})$.
 
 :::
 
-The theorem reveals several important insights. First, the variance floor decreases linearly with the number of workers $N$, explaining why distributed training can achieve the same final loss with fewer iterations. The effective batch size is $B = Nb$, so $N$ workers with batch size $b$ each behave equivalently to a single worker with batch size $Nb$. Second, the convergence rate $O(1/\sqrt{NM})$ shows that $N$ workers can achieve the same error as a single worker in $1/N$ the iterations, assuming infinite bandwidth. This is the **statistical efficiency** of distributed training, distinct from hardware efficiency.
+The theorem reveals several important insights. First, the variance floor decreases linearly with the number of workers $N$, explaining why distributed training can achieve the same final loss with fewer iterations. The effective batch size is $B = Nb$, so $N$ workers with batch size $b$ each behave equivalently to a single worker with batch size $Nb$. Second, the convergence rate $O(1/\sqrt{NbM})$ shows that workers, local batch size, and iteration count jointly determine statistical progress, assuming infinite bandwidth. This is the **statistical efficiency** of distributed training, distinct from hardware efficiency.
 
 However, the theorem assumes perfect synchronization (BSP). When workers proceed at different rates or use stale gradients, convergence guarantees degrade, as we examine next.
 
@@ -1224,7 +1224,7 @@ Synchronization models fundamentally affect convergence behavior. The staleness
 
 ***Gradient Staleness ($\tau$)***\index{Gradient!staleness, definition} is the number of parameter updates that occur between the time a gradient is computed and the time it is applied to the global model state.
 
-1.  **Significance (Quantitative):** It represents the **Synchronization Error** in distributed optimization. Increasing $\tau$ can improve **Throughput ($\eta$)** by reducing barrier waits ($L_{\text{lat}}$), but it typically degrades the **Rate of Convergence**, requiring more operations ($O$) to reach the same accuracy.
+1.  **Significance (Quantitative):** It represents the **Synchronization Error** in distributed optimization. Increasing $\tau$ can improve throughput by reducing barrier waits ($L_{\text{lat}}$), but it typically degrades the **Rate of Convergence**, requiring more operations ($O$) to reach the same accuracy.
 2.  **Distinction (Durable):** Unlike **Network Latency**, which is a physical delay, Staleness is an **Algorithmic Offset** that arises from the choice of synchronization protocol (e.g., ASP, SSP).
 3.  **Common Pitfall:** A frequent misconception is that Staleness is "always bad." In reality, it is a **Throughput-Convergence Trade-off**: for some large-scale workloads, allowing bounded staleness is the only way to keep thousands of GPUs in use.
 
@@ -1237,7 +1237,7 @@ In Bulk Synchronous Parallel (BSP, $\tau = 0$), all workers compute gradients on
 Stale Synchronous Parallel (SSP, $\tau \leq s$) relaxes the barrier by allowing workers to proceed up to $s$ iterations ahead of the slowest worker. The convergence rate degrades to:
 
 $$
-\mathbb{E}[L(\theta_M)] - L \leq O\left(\frac{1}{\sqrt{NbM}}\right) + O\left(\frac{s^2 \eta^2 L^2}{Nb}\right)
+\mathbb{E}[\mathcal{L}(\theta_M)] - \mathcal{L}^{\star} \leq O\left(\frac{1}{\sqrt{NbM}}\right) + O\left(\frac{s^2 \eta^2 L_s^2}{Nb}\right)
 $$
 
 The second term represents the **staleness penalty**. For bounded staleness $s$, this penalty can be controlled by reducing the learning rate: $\eta' = \eta / \sqrt{1 + s}$. Typical production systems use $s \in \{2, 4, 8\}$, accepting 5--15 percent convergence degradation for 20--40 percent throughput improvement on heterogeneous clusters.
@@ -1245,7 +1245,7 @@ The second term represents the **staleness penalty**. For bounded staleness $s$,
 Asynchronous SGD (ASP, $\tau = \infty$) eliminates waiting entirely: workers update parameters immediately. While this maximizes throughput, convergence degrades:
 
 $$
-\mathbb{E}[L(\theta_M)] - L \leq O\left(\frac{1}{\sqrt{M}}\right) + O\left(\frac{\bar{\tau}^2 \eta^2 L^2}{1}\right)
+\mathbb{E}[\mathcal{L}(\theta_M)] - \mathcal{L}^{\star} \leq O\left(\frac{1}{\sqrt{M}}\right) + O\left(\bar{\tau}^2 \eta^2 L_s^2\right)
 $$
 
 where $\bar{\tau}$ is the average staleness. The staleness penalty now scales with the square of average delay, and critically, the variance reduction from $N$ workers disappears in the dominant term. Compensation techniques include:
@@ -1259,8 +1259,8 @@ where $\bar{\tau}$ is the average staleness. The staleness penalty now scales wi
 | **Model** | **Staleness**   |                    **Convergence Rate** | **Variance Reduction** | **Best For**                          |
 |:----------|:----------------|----------------------------------------:|:-----------------------|:--------------------------------------|
 | **BSP**   | $\tau = 0$      |                       $O(1/\sqrt{NbM})$ | Full ($1/N$)           | Final training, reproducibility       |
-| **SSP**   | $\tau \leq s$   |        $O(1/\sqrt{NbM}) + O(s^2\eta^2)$ | Partial                | Heterogeneous clusters                |
-| **ASP**   | $\tau = \infty$ | $O(1/\sqrt{M}) + O(\bar{\tau}^2\eta^2)$ | None                   | Maximum throughput, early exploration |
+| **SSP**   | $\tau \leq s$   |        $O(1/\sqrt{NbM}) + O(s^2\eta^2 L_s^2 / Nb)$ | Partial                | Heterogeneous clusters                |
+| **ASP**   | $\tau = \infty$ | $O(1/\sqrt{M}) + O(\bar{\tau}^2\eta^2 L_s^2)$ | None                   | Maximum throughput, early exploration |
 
 : **Convergence Properties by Synchronization Model**. BSP provides optimal convergence guarantees at the cost of synchronization overhead. SSP offers a tunable trade-off between throughput and convergence. ASP maximizes throughput but loses the variance reduction benefit of parallelism. {#tbl-convergence-comparison}
 
@@ -1306,57 +1306,57 @@ Layer-wise scaling prevents layers with small weights from receiving disproporti
 
 ### Critical batch size: When does parallelism hurt? {#sec-distributed-training-systems-systems-critical-batch-size-parallelism-hurt-4961}
 
-A fundamental question in distributed training is: when does adding more workers stop helping? The **critical batch size** $B*$ marks the transition point beyond which increasing batch size yields diminishing returns in convergence per sample seen.
+A fundamental question in distributed training is: when does adding more workers stop helping? The **critical batch size** $B^{*}$ marks the transition point beyond which increasing batch size yields diminishing returns in convergence per sample seen.
 
 ::: {.callout-definition title="Critical Batch Size"}
 
-***Critical Batch Size***\index{Critical Batch Size!definition} ($B^*$) is the batch size at which the **Gradient Noise Scale** equals the **Gradient Signal Scale**.
+***Critical Batch Size***\index{Critical Batch Size!definition} ($B^{*}$) is the batch size at which the **Gradient Noise Scale** equals the **Gradient Signal Scale**.
 
-1.  **Significance (Quantitative):** It marks the transition point for **Parallel Scaling Efficiency**. Below $B*$, increasing the batch size linearly improves the convergence per step. Above $B*$, larger batches yield diminishing returns, requiring proportionally more samples ($D_{\text{vol}}$) to reach the same loss.
-2.  **Distinction (Durable):** Unlike the **Memory-Limited Batch Size** (determined by $BW$ and capacity), the Critical Batch Size is an **Algorithmic Property** of the model and dataset.
-3.  **Common Pitfall:** A frequent misconception is that training can be speeded up indefinitely by adding GPUs. In reality, $B*$ defines the **Physical Ceiling for Data Parallelism**: adding workers beyond this point wastes energy and compute ($O$) without reducing total training time ($T$).
+1.  **Significance (Quantitative):** It marks the transition point for **Parallel Scaling Efficiency**. Below $B^{*}$, increasing the batch size linearly improves the convergence per step. Above $B^{*}$, larger batches yield diminishing returns, requiring proportionally more samples ($D$) to reach the same loss.
+2.  **Distinction (Durable):** Unlike the **Memory-Limited Batch Size** (determined by $\text{BW}$ and capacity), the Critical Batch Size is an **Algorithmic Property** of the model and dataset.
+3.  **Common Pitfall:** A frequent misconception is that training can be speeded up indefinitely by adding GPUs. In reality, $B^{*}$ defines the **Physical Ceiling for Data Parallelism**: adding workers beyond this point wastes energy and compute ($O$) without reducing total training time ($T$).
 
 :::
 
 The critical batch size can be estimated as:
 
 $$
-B* \approx \frac{\text{tr}(\Sigma)}{\|\nabla L(\theta)\|^2}
+B^{*} \approx \frac{\text{tr}(\Sigma)}{\|\nabla \mathcal{L}(\theta)\|^2}
 $$
 
-where $\text{tr}(\Sigma)$ is the trace of the gradient covariance matrix (total gradient variance) and $\|\nabla L(\theta)\|^2$ is the squared gradient norm (signal strength). Intuitively, $B*$ is the batch size at which averaging reduces gradient variance to the level of the true gradient magnitude.
+where $\text{tr}(\Sigma)$ is the trace of the gradient covariance matrix (total gradient variance) and $\|\nabla \mathcal{L}(\theta)\|^2$ is the squared gradient norm (signal strength). Intuitively, $B^{*}$ is the batch size at which averaging reduces gradient variance to the level of the true gradient magnitude.
 
 Empirical critical batch sizes vary by task and scale over three orders of magnitude:
 
-- **ImageNet ResNet-50**: $B* \approx 8,000 - 16,000$
-- **BERT-Large pretraining**: $B* \approx 32,000 - 65,000$
-- **GPT-3 scale models**: $B* \approx 1,000,000 - 4,000,000$
+- **ImageNet ResNet-50**: $B^{*} \approx 8,000 - 16,000$
+- **BERT-Large pretraining**: $B^{*} \approx 32,000 - 65,000$
+- **GPT-3 scale models**: $B^{*} \approx 1,000,000 - 4,000,000$
 
 The scaling law regime exhibits three distinct behaviors:
 
-1. **Below critical ($B < B*$)**: Linear scaling holds. Doubling batch size halves iterations to reach target loss. Hardware efficiency determines throughput.
+1. **Below critical ($B < B^{*}$)**: Linear scaling holds. Doubling batch size halves iterations to reach target loss. Hardware efficiency determines throughput.
 
-2. **At critical ($B \approx B*$)**: Optimal trade-off point. Maximum samples-per-second efficiency.
+2. **At critical ($B \approx B^{*}$)**: Optimal trade-off point. Maximum samples-per-second efficiency.
 
-3. **Above critical ($B > B*$)**: Diminishing returns. Doubling batch size requires $>2\times$ more samples total. Additional workers provide throughput but not sample efficiency.
+3. **Above critical ($B > B^{*}$)**: Diminishing returns. Doubling batch size requires $>2 \times$ more samples total. Additional workers provide throughput but not sample efficiency.
 
 @fig-critical-batch-size illustrates this relationship between batch size and training efficiency.
 
-::: {#fig-critical-batch-size fig-env="figure" fig-pos="htb" fig-cap="**Critical Batch Size and Scaling Regimes**. Below the critical batch size $B*$, larger batches reduce noise and improve sample efficiency (linear scaling regime). Above $B*$, larger batches provide diminishing returns: while throughput increases, total samples required also increases, reducing sample efficiency. The optimal operating point balances hardware utilization against convergence efficiency." fig-alt="Graph showing sample efficiency versus batch size. Efficiency is flat in the linear regime below B-star, then decreases in the diminishing returns regime above B-star. Vertical dashed line marks critical batch size."}
+::: {#fig-critical-batch-size fig-env="figure" fig-pos="htb" fig-cap="**Critical Batch Size and Scaling Regimes**. Below the critical batch size $B^{*}$, larger batches reduce noise and improve sample efficiency (linear scaling regime). Above $B^{*}$, larger batches provide diminishing returns: while throughput increases, total samples required also increases, reducing sample efficiency. The optimal operating point balances hardware utilization against convergence efficiency." fig-alt="Graph showing sample efficiency versus batch size. Efficiency is flat in the linear regime below B-star, then decreases in the diminishing returns regime above B-star. Vertical dashed line marks critical batch size."}
 ![](images/svg/critical-batch-size.svg){width=100%}
 :::
 
 The critical batch size has important implications for distributed training system design:
 
-1. **Worker count selection**: Adding workers beyond $B*/b$ (where $b$ is per-worker batch size) improves throughput but not sample efficiency. For cost optimization, this may still be worthwhile if the marginal cost of additional workers is low.
+1. **Worker count selection**: Adding workers beyond $B^{*}/b$ (where $b$ is per-worker batch size) improves throughput but not sample efficiency. For cost optimization, this may still be worthwhile if the marginal cost of additional workers is low.
 
-2. **Learning rate schedule**: Above $B*$, aggressive learning rate warmup becomes essential. The loss landscape near initialization may not support the large updates that linear scaling would produce.
+2. **Learning rate schedule**: Above $B^{*}$, aggressive learning rate warmup becomes essential. The loss landscape near initialization may not support the large updates that linear scaling would produce.
 
-3. **Communication trade-offs**: Above $B*$, the reduced benefit of larger batches makes communication overhead relatively more costly. This strengthens the case for gradient compression or asynchronous methods.
+3. **Communication trade-offs**: Above $B^{*}$, the reduced benefit of larger batches makes communication overhead relatively more costly. This strengthens the case for gradient compression or asynchronous methods.
 
 ::: {.callout-checkpoint title="Scaling Decisions"}
 
-Given a 7B parameter model distributed across a cluster of 64 A100 GPUs (80 GB HBM each), what is the maximum useful batch size? To answer this, you must calculate the **Critical Batch Size** ($B_{crit}$)—the point where the gradient noise scale equals the batch size. Beyond this point, doubling the batch size yields diminishing returns in convergence speed (perfect scaling stops). Using the Gradient Noise Scale metric ($\mathcal{B} \approx \frac{\text{tr}(\Sigma)}{\|\mu\|^2}$), determine if your proposed batch size keeps scaling efficiency above 80 percent, or if you are simply wasting compute cycles for marginal gains.
+Given a 7B parameter model distributed across a cluster of 64 A100 GPUs (80 GB HBM each), what is the maximum useful batch size? To answer this, you must calculate the **Critical Batch Size** ($B_{\text{crit}}$)—the point where the gradient noise scale equals the batch size. Beyond this point, doubling the batch size yields diminishing returns in convergence speed (perfect scaling stops). Using the Gradient Noise Scale metric ($\mathcal{B} \approx \frac{\text{tr}(\Sigma)}{\|\mu\|^2}$), determine if your proposed batch size keeps scaling efficiency above 80 percent, or if you are simply wasting compute cycles for marginal gains.
 
 :::
 
@@ -1787,7 +1787,7 @@ The transfer of hidden states between devices occurs continuously rather than in
 
 The classic problem with pipeline parallelism is the **pipeline bubble**: GPUs at the beginning of the pipeline are idle while waiting for gradients to flow back from the end, and vice versa. These bubbles represent wasted compute. The **1F1B** (one forward, one backward) schedule reduces the bubble by interleaving forward and backward microbatches. Once the pipeline is filled, each GPU alternates between executing one forward microbatch and one backward microbatch, keeping SMs busy most of the time.
 
-**Zero-bubble pipeline schedules**\index{Zero-Bubble Pipeline Parallelism} further reduce idle time by overlapping weight gradient computation with activation gradient communication. In a standard backward pass, the GPU computes $\partial L / \partial W$ (weight gradient) and $\partial L / \partial X$ (activation gradient, sent to the previous stage) together. Zero-bubble scheduling splits these into separate kernels: a **B** kernel that computes only the activation gradient $\partial L / \partial X$ and sends it to the previous stage, and a **W** kernel that computes the weight gradient $\partial L / \partial W$ locally. The B kernel must execute promptly (it is on the critical path), but the W kernel can be scheduled opportunistically to fill bubbles.
+**Zero-bubble pipeline schedules**\index{Zero-Bubble Pipeline Parallelism} further reduce idle time by overlapping weight gradient computation with activation gradient communication. In a standard backward pass, the GPU computes $\partial \mathcal{L} / \partial W$ (weight gradient) and $\partial \mathcal{L} / \partial X$ (activation gradient, sent to the previous stage) together. Zero-bubble scheduling splits these into separate kernels: a **B** kernel that computes only the activation gradient $\partial \mathcal{L} / \partial X$ and sends it to the previous stage, and a **W** kernel that computes the weight gradient $\partial \mathcal{L} / \partial W$ locally. The B kernel must execute promptly (it is on the critical path), but the W kernel can be scheduled opportunistically to fill bubbles.
 
 The scheduling freedom provided by this B/W split is substantial. In a 4-stage pipeline with 8 microbatches, the standard 1F1B schedule has a bubble fraction of approximately $(p-1)/(m+p-1)$ where $p$ is the number of stages and $m$ is the number of microbatches. For $p=4, m=8$, this is $3/11 \approx 27\%$ idle time. Zero-bubble scheduling can reduce this to near zero by filling the startup and teardown bubbles with W computations.
 

book/quarto/contents/vol2/edge_intelligence/edge_intelligence.qmd

@@ -1771,7 +1771,7 @@ Federated learning addresses these coordination constraints through privacy-pres
 
 ***Federated Learning***\index{Federated Learning!definition} is a decentralized training paradigm where distributed devices collaboratively train a shared model using local data while exchanging only model updates (gradients or weights).
 
-1.  **Significance (Quantitative):** It transforms the constraint of **Data Locality**\index{Data Locality} into a privacy feature. Within the **Iron Law**, Federated Learning is constrained by the **Wide-Area Bandwidth ($BW$)** and the extreme **Heterogeneity of the Fleet**\index{Heterogeneity of the Fleet}, where device-specific efficiency ($\eta$) and availability can vary by orders of magnitude.
+1.  **Significance (Quantitative):** It transforms the constraint of **Data Locality**\index{Data Locality} into a privacy feature. Within the **Iron Law**, Federated Learning is constrained by the **Wide-Area Bandwidth ($\text{BW}$)** and the extreme **Heterogeneity of the Fleet**\index{Heterogeneity of the Fleet}, where device-specific efficiency ($\eta$) and availability can vary by orders of magnitude.
 2.  **Distinction (Durable):** Unlike **Centralized Training**\index{Training!centralized}, where data is moved to the compute, Federated Learning moves the **Compute to the Data**\index{Compute to the Data}, ensuring that raw information never leaves the device.
 3.  **Common Pitfall:** A frequent misconception is that Federated Learning is "inherently private." In reality, model updates themselves can leak information through **Gradient Inversion** attacks, requiring additional protections like **Differential Privacy** or **Secure Aggregation**.
 

book/quarto/contents/vol2/fault_tolerance/fault_tolerance.qmd

@@ -1906,7 +1906,7 @@ To survive these inevitable hardware failures without catastrophic loss of compu
 
 1.  **Significance (Quantitative):** It minimizes the **Lost Work** after a system failure. Within the **iron law**\index{iron law}, checkpointing creates an **I/O Overhead** that reduces the total training throughput ($\eta$), with the optimal interval ($T_{\text{opt}}$) governed by the **Young-Daly Formula** ($T_{\text{opt}} = \sqrt{2 \cdot T_{\text{save}} \cdot \text{MTBF}}$).
 2.  **Distinction (Durable):** Unlike **Incremental Backups**\index{Incremental Backups}, Checkpointing must capture the **Exact Execution Context** (including random seeds and learning rate schedules) to ensure deterministic resumption of the optimization loop.
-3.  **Common Pitfall:** A frequent misconception is that Checkpointing is "just writing to disk." In reality, for large models, it is a **Storage System Stress Test**\index{Storage!system stress test}: the simultaneous write from thousands of GPUs can trigger a **checkpoint storm** that saturates the entire network fabric ($BW$).
+3.  **Common Pitfall:** A frequent misconception is that Checkpointing is "just writing to disk." In reality, for large models, it is a **Storage System Stress Test**\index{Storage!system stress test}: the simultaneous write from thousands of GPUs can trigger a **checkpoint storm** that saturates the entire network fabric ($\text{BW}$).
 
 :::
 

book/quarto/contents/vol2/fleet_orchestration/fleet_orchestration.qmd

@@ -520,7 +520,7 @@ The most pernicious of these is **priority inversion**, a scenario borrowed from
 
 ***Priority Inversion***\index{Priority Inversion!definition} is a scheduling pathology in which a high-priority task is forced to wait for a lower-priority task to release a shared resource.
 
-1.  **Significance (Quantitative):** It reduces the progress rate of the entire fleet to that of the lowest-priority job. In ML clusters, this typically occurs when a low-priority job holding GPUs is starved of auxiliary resources (for example, $BW$ for checkpointing), preventing it from finishing and releasing the accelerators needed by high-priority workloads.
+1.  **Significance (Quantitative):** It reduces the progress rate of the entire fleet to that of the lowest-priority job. In ML clusters, this typically occurs when a low-priority job holding GPUs is starved of auxiliary resources (for example, $\text{BW}$ for checkpointing), preventing it from finishing and releasing the accelerators needed by high-priority workloads.
 2.  **Distinction (Durable):** Unlike **Standard Queuing** (where tasks wait their turn), Priority Inversion involves an **Active Blockage**\index{Active Blockage}: the high-priority task is ready to run but is transitively dependent on a task that the scheduler does not prioritize.
 3.  **Common Pitfall:** A frequent misconception is that strict priority levels solve this. In reality, without **Holistic Preemption** (reserving all resources needed for a task to exit), increasing priority levels can actually *increase* the likelihood of inversion by creating more complex dependency chains.
 

book/quarto/contents/vol2/inference/inference.qmd

@@ -2857,7 +2857,7 @@ LLM inference consists of two distinct phases with different computational chara
 
 ***Prefill and Decode Phases***\index{Prefill and Decode!definition} are the two distinct computational regimes of transformer-based LLM inference.
 
-1.  **Significance (Quantitative):** The **Prefill Phase** (processing the prompt) is **Compute-Bound ($R_{\text{peak}}$)** with high arithmetic intensity, while the **Decode Phase** (generating tokens) is **Bandwidth-Bound ($BW$)** with extremely low arithmetic intensity. This mismatch means a single request's efficiency ($\eta$) varies wildly during its lifecycle.
+1.  **Significance (Quantitative):** The **Prefill Phase** (processing the prompt) is **Compute-Bound ($R_{\text{peak}}$)** with high arithmetic intensity, while the **Decode Phase** (generating tokens) is **Bandwidth-Bound ($\text{BW}$)** with extremely low arithmetic intensity. This mismatch means a single request's efficiency ($\eta$) varies wildly during its lifecycle.
 2.  **Distinction (Durable):** Unlike **Single-Pass Inference** (for example, ImageNet), where the resource bottleneck is constant, LLM inference switches between these regimes at every request, requiring **Iteration-Level Scheduling** to maintain utilization.
 3.  **Common Pitfall:** A frequent misconception is that both phases should be batched together naively. In reality, because they have opposing hardware requirements, mixing them in the same batch without **Chunked Prefill** or similar techniques can lead to significant queuing delays ($L_{\text{lat}}$) for decoding tokens.
 
@@ -3607,21 +3607,21 @@ The practical speedup from sharding depends critically on communication efficien
 
 @eq-allreduce-time quantifies AllReduce communication time for tensor parallelism, where data is combined from all devices with the result available on all devices.
 
-$$T_{\text{allreduce}} = 2 \times \frac{(N-1)}{N} \times \frac{M}{B}$$ {#eq-allreduce-time}
+$$T_{\text{allreduce}} = 2 \times \frac{(N-1)}{N} \times \frac{M}{\text{BW}}$$ {#eq-allreduce-time}
 
-where $N$ is the number of devices, $M$ is the message size, and $B$ is the interconnect bandwidth. The factor of 2 accounts for the reduce-scatter and all-gather phases.
+where $N$ is the number of devices, $M$ is the message size, and $\text{BW}$ is the interconnect bandwidth. The factor of 2 accounts for the reduce-scatter and all-gather phases.
 
 @eq-p2p-time expresses the simpler point-to-point communication for pipeline parallelism, where data flows from one device to the next.
 
-$$T_{\text{p2p}} = L + \frac{M}{B}$$ {#eq-p2p-time}
+$$T_{\text{p2p}} = L + \frac{M}{\text{BW}}$$ {#eq-p2p-time}
 
-where $L$ is the network latency and $M/B$ is the transfer time.
+where $L$ is the network latency and $M/\text{BW}$ is the transfer time.
 
 @eq-alltoall-time captures the more complex AllToAll communication for expert parallelism, where each device exchanges distinct data with every other device.
 
-$$T_{\text{alltoall}} = (N-1) \times \left(L + \frac{M/N}{B}\right)$$ {#eq-alltoall-time}
+$$T_{\text{alltoall}} = (N-1) \times \left(L + \frac{M/N}{\text{BW}}\right)$$ {#eq-alltoall-time}
 
-Both equations make clear that the achievable bandwidth $B$ and latency $L$ of the underlying interconnect determine real-world sharding performance. The following *interconnect technology comparison* quantifies these differences across production hardware.
+Both equations make clear that the achievable bandwidth $\text{BW}$ and latency $L$ of the underlying interconnect determine real-world sharding performance. The following *interconnect technology comparison* quantifies these differences across production hardware.
 
 ::: {.callout-notebook title="Interconnect Technology Comparison"}
 

book/quarto/contents/vol2/introduction/introduction.qmd

@@ -1333,7 +1333,7 @@ At extreme scales, models may approach the limits of what their training distrib
 Verify your understanding of *how* scaling laws guide resource allocation:
 
 - [ ] A team has a fixed compute budget but limited training data. According to the "Regimes" framework, should they train a large model for fewer steps or a small model for more steps?
-- [ ] If you double the model parameters ($N$) but keep the dataset size ($D$) constant, which "Scaling Breakdown" are you most likely to encounter?
+- [ ] If you double the model parameters ($P$) but keep the dataset size ($D$) constant, which "Scaling Breakdown" are you most likely to encounter?
 - [ ] *Why* is **Chinchilla Optimality** considered the "Gold Standard" for resource allocation? *What* is being minimized?
 - [ ] True or False: Scaling laws predict that doubling compute always yields a doubling of model capability.
 

book/quarto/contents/vol2/parts/distributed_ml_principles.qmd

@@ -5,8 +5,8 @@ Coordinating the **Fleet** requires solving problems that a single machine does
 At this scale, the engineering challenge is a trade-off between parallelization and coordination. We can partition a model to reduce per-device memory pressure, but we necessarily increase the communication volume required to keep the weights synchronized. We can scale to thousands of nodes to reduce training time, but we simultaneously decrease the Mean Time Between Failures (MTBF), making fault tolerance a mandatory system component rather than an operational luxury. These principles quantify the "Communication Tax" and the "Reliability Tax" of scale.
 
 ::: {#nte-universal-scaling .callout-principle icon=false title="The Universal Scaling Law"}
-**The Invariant**: Model performance ($L$) improves as a power-law function of compute ($C$), dataset size ($D$), and parameters ($N$).
-$$ L(C) \propto C^{-\alpha} $$
+**The Invariant**: Model loss ($\mathcal{L}$) improves as a power-law function of compute ($C$), dataset size ($D$), and parameters ($P$).
+$$ \mathcal{L}(C) \propto C^{-\alpha} $$
 
 **The Implication**: Scale is not an option; it is a requirement for capability. To achieve a 10$\times$ improvement in performance, you typically need a 100$\times$–1000$\times$ increase in compute. This exponential hunger drives the transition from single-server training to warehouse-scale clusters.
 :::

book/quarto/contents/vol2/performance_engineering/performance_engineering.qmd

@@ -267,22 +267,22 @@ The **Roofline Model**[^fn-roofline-origin] provides a quantitative framework fo
 
 ***Arithmetic Intensity ($I$)***\index{Arithmetic Intensity!definition} is the ratio of floating-point operations performed to the number of bytes transferred from memory ($FLOP/\text{byte}$).
 
-1.  **Significance (Quantitative):** It characterizes the **Computational Density** of a workload. It is the independent variable in the **Roofline Model**, determining whether a system operates in the **Bandwidth-Bound** ($BW$) or **Compute-Bound**\index{Compute-Bound} ($R_{\text{peak}}$) regime.
+1.  **Significance (Quantitative):** It characterizes the **Computational Density** of a workload. It is the independent variable in the **Roofline Model**, determining whether a system operates in the **Bandwidth-Bound** ($\text{BW}$) or **Compute-Bound**\index{Compute-Bound} ($R_{\text{peak}}$) regime.
 2.  **Distinction (Durable):** Unlike **Peak Throughput** (a hardware property), Arithmetic Intensity is an **Algorithmic Property** that measures how effectively a workload reuses data once it is loaded into the processor.
 3.  **Common Pitfall:** A frequent misconception is that AI is fixed for a model. In reality, it varies by **Implementation**\index{Implementation}: techniques like operator fusion increase AI by keeping data in local registers, while increasing batch size increases AI for layers with high parameter reuse.
 
 :::
 
-For a given accelerator with peak compute $P$ (in FLOPS) and peak memory bandwidth $B$ (in bytes/second), the achievable performance of a workload with arithmetic intensity $I$ (in FLOP/byte) is:
+For a given accelerator with peak compute $R_{\text{peak}}$ (in FLOP/s) and peak memory bandwidth $\text{BW}$ (in bytes/second), the achievable performance of a workload with arithmetic intensity $I$ (in FLOP/byte) is:
 
 $$
-\text{Achievable FLOPS} = \min(P, \; B \times I)
+\text{Achievable FLOPS} = \min(R_{\text{peak}}, \; \text{BW} \times I)
 $$ {#eq-performance-roofline}
 
 The transition point where these two limits intersect is the **ridge point**:
 
 $$
-I_{\text{ridge}} = \frac{P}{B}
+I_{\text{ridge}} = \frac{R_{\text{peak}}}{\text{BW}}
 $$ {#eq-ridge-point}
 
 Workloads with $I < I_{\text{ridge}}$ are memory-bound: their performance is limited by how fast data can be loaded, not how fast it can be processed. Workloads with $I > I_{\text{ridge}}$ are compute-bound: the arithmetic units are the bottleneck. @fig-roofline-model illustrates this relationship graphically.

book/quarto/contents/vol2/responsible_ai/responsible_ai.qmd

@@ -292,7 +292,7 @@ Suppose a model $h(x)$ predicts a binary outcome, such as loan repayment, and le
 
 ***Demographic Parity***\index{Demographic Parity!definition} is the fairness constraint where a model's positive prediction rate is independent of group membership ($P(\hat{Y}=1 | A=a) = P(\hat{Y}=1 | A=b)$).
 
-1.  **Significance (Quantitative):** It is the simplest and most restrictive fairness metric. It requires the model to produce **Equal Outcomes** across groups, regardless of the underlying base-rate differences in the data ($D_{\text{vol}}$).
+1.  **Significance (Quantitative):** It is the simplest and most restrictive fairness metric. It requires the model to produce **Equal Outcomes** across groups, regardless of the underlying base-rate differences in the dataset.
 2.  **Distinction (Durable):** Unlike **Equalized Odds** (which focuses on error rates like False Positives), Demographic Parity focuses only on the **Final Prediction**\index{Final Prediction}, ignoring the relationship between the prediction and the ground truth.
 3.  **Common Pitfall:** A frequent misconception is that Demographic Parity ensures "fairness." In reality, it can force the model to sacrifice **Calibration**\index{Calibration}: to meet the parity constraint, the model may have to intentionally misclassify qualified individuals in one group or unqualified individuals in another.