github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-07-16 06:07:17 -05:00
Code Issues 276 Packages Projects Releases 26 Wiki Activity

Refined the PDF layout for chapter 10 (model_compression.qmd)

Browse Source
This commit is contained in:
Zeljko Hrcek
2026-06-29 18:13:47 +02:00
parent 85ada0ffa6
commit 7ecd450360
1 changed files with 122 additions and 93 deletions
Download Patch File Download Diff File Expand all files Collapse all files

215
book/quarto/contents/vol1/model_compression/model_compression.qmd
View File

@@ -149,7 +149,7 @@ Model optimization is not a *single technique* but a *framework* with three comp
::: {#fig-3-sections fig-env="figure" fig-pos="htb" fig-cap="**Optimization Stack**: Model optimization progresses through three layers: efficient model representation, efficient numerics representation, and efficient hardware implementation." fig-alt="Three stacked rectangular boxes labeled from top to bottom: efficient model representation, efficient numerics representation, efficient hardware implementation. A vertical arrow spans the stack with More software at top and More hardware at bottom."}
```{.tikz}
\resizebox{.55\textwidth}{!}{
\resizebox{.5\textwidth}{!}{
\begin{tikzpicture}[font=\small\usefont{T1}{phv}{m}{n}]
\tikzset{
Box/.style={inner xsep=2pt,
@@ -157,13 +157,13 @@ Model optimization is not a *single technique* but a *framework* with three comp
line width=0.75pt,
anchor=west,
text width=54mm,align=flush center,
minimum width=54mm, minimum height=9mm
minimum width=54mm, minimum height=8mm
},
}
\node[Box,fill=red!30,anchor=south west](B1)at (0.33,0.5){Efficient Hardware Implementation};
\node[Box,fill=red!20,node distance=0.2,above=of B1](B2){Efficient Numerics Representation};
\node[Box,fill=red!10,node distance=0.2,above=of B2](B3){Efficient Model Representation};
\draw[latex-latex,line width=0.75pt](0,0.20)--(0,4);
\node[Box,fill=red!20,node distance=0.15,above=of B1](B2){Efficient Numerics Representation};
\node[Box,fill=red!10,node distance=0.15,above=of B2](B3){Efficient Model Representation};
\draw[latex-latex,line width=0.75pt](0,0.20)--(0,3.7);
\node[left=1.4 of B1,rotate=90,anchor=north,align=center,font=\footnotesize\usefont{T1}{phv}{m}{n}]{More\\ hardware};
\node[left=1.4 of B3,rotate=90,anchor=north,align=center,font=\footnotesize\usefont{T1}{phv}{m}{n}]{More \\software};
@@ -215,6 +215,11 @@ The relative importance of each dimension varies by deployment target. Cloud sys
The preceding optimization framework identifies three dimensions of compression, but which dimensions matter most depends entirely on where the model will run. A data center GPU with 80 GB of HBM faces different binding constraints than a smartphone with shared RAM or a microcontroller with only a few hundred kilobytes of SRAM. @Tbl-deployment-scenarios summarizes the key constraints across deployment environments.
```{=latex}
\begingroup
\renewcommand{\arraystretch}{1.25}
```
| **Context** | **Memory** | **Latency** | **Power** | **Primary Goal** |
|:----------------|:--------------------:|:-----------:|:----------|:-----------------|
| **Cloud** | tens of GB | 10–100 ms | Flexible | Throughput, cost |
@@ -223,6 +228,10 @@ The preceding optimization framework identifies three dimensions of compression,
: **Deployment Constraints**: Each deployment context imposes different optimization priorities. {#tbl-deployment-scenarios tbl-colwidths="[20,22,16,14,28]"}
```{=latex}
\endgroup
```
### Deployment scenarios {#sec-model-compression-deployment-scenarios-70c9}
Cloud inference centers on throughput (requests/second/dollar), where quantization enables serving more concurrent requests and operator fusion reduces per-request latency [@choudhary2020; @dean2018]. Mobile and edge deployments must fit device memory while meeting real-time targets. A camera app processing 30 fps has 33 ms per frame, so any optimization reducing inference below this threshold directly improves user experience.
@@ -336,6 +345,9 @@ class ModelDeviceComparison:
As @tbl-model-vs-device makes concrete, even aggressively optimized models like MobileNetV2 at INT8 precision exceed TinyML device memory by about `{python} ModelDeviceComparison.mobilenet_int8_tiny_ratio_mult_str`.
### Balancing trade-offs {#sec-model-compression-balancing-tradeoffs-6ae3}
The **accuracy-efficiency trade-off**\index{Model Compression!accuracy-efficiency trade-off}\index{Accuracy!compression trade-offs} drives every optimization decision. Increasing model capacity generally enhances predictive performance while increasing computational cost, resulting in slower, more resource-intensive inference. The improvements introduce challenges related to memory footprint\index{Memory Footprint!deployment constraint}, inference latency, power consumption, and training efficiency.
| **Model** | **Memory** **(Runtime)** | **Storage** **(Weights)** | **Cloud** **(`{python} ModelDeviceComparison.cloud_cap_gb_str`)** | **Mobile** **(`{python} ModelDeviceComparison.mobile_cap_gb_str`)** | **TinyML** **(`{python} ModelDeviceComparison.tiny_cap_kb_str`)** |
|:-----------------------|:------------------------------------------------------:|:------------------------------------------------------:|:-----------------------------------------------------------------:|:-------------------------------------------------------------------:|:------------------------------------------------------------------------:|
| **DLRM** | `{python} ModelDeviceComparison.dlrm_gb_str` | `{python} ModelDeviceComparison.dlrm_gb_str` | ok | no (`{python} ModelDeviceComparison.dlrm_mobile_overflow_mult_str`) | no (`{python} ModelDeviceComparison.dlrm_tiny_overflow_mult_str`) |
@@ -347,9 +359,6 @@ As @tbl-model-vs-device makes concrete, even aggressively optimized models like
: **The Deployment Gap**: Model memory requirements compared against typical device capacities. DLRM represents recommendation-model embedding pressure [@naumov2019deep]; DS-CNN represents the TinyML keyword-spotting case [@zhang2017hello]. Even MobileNetV2 quantized to INT8 exceeds the `{python} ModelDeviceComparison.tiny_cap_kb_str` TinyML envelope by `{python} ModelDeviceComparison.mobilenet_int8_tiny_ratio_mult_str`, while the purpose-built DS-CNN keyword spotter fits within it. Numbers in parentheses show how many times the model exceeds device memory. {#tbl-model-vs-device tbl-colwidths="[20,15,15,15,15,20]"}
### Balancing trade-offs {#sec-model-compression-balancing-tradeoffs-6ae3}
The **accuracy-efficiency trade-off**\index{Model Compression!accuracy-efficiency trade-off}\index{Accuracy!compression trade-offs} drives every optimization decision. Increasing model capacity generally enhances predictive performance while increasing computational cost, resulting in slower, more resource-intensive inference. The improvements introduce challenges related to memory footprint\index{Memory Footprint!deployment constraint}, inference latency, power consumption, and training efficiency.
This tension manifests differently across deployment contexts. Training requires computational resources that scale with model size; inference demands strict latency and power constraints in real-time applications. Understanding where each optimization technique falls on the compression-accuracy Pareto frontier is essential for informed technique selection\index{Pareto Frontier!compression-accuracy}\index{Compression Ratio!accuracy trade-off}.
::: {#psp-model-compression-compression-accuracy-tradeoff-curve .callout-perspective title="The compression-accuracy trade-off curve"}
@@ -373,7 +382,7 @@ The engineering decision is where to stop. Compression should halt at the "knee"
| **90% Pruning** | ~10$\times$ smaller model | Severe capacity loss | 5–15% accuracy drop | 3 |
| **↑ Batch Size (8$\times$)** | Higher throughput, better GPU util | Generalization gap | Requires LR scaling | — |
: **The Optimization Trade-Offs**: Region 1 = Free Lunch, Region 2 = Efficient Trade, Region 3 = Danger Zone. Batch size affects training dynamics rather than model quality directly. These are representative engineering ranges synthesized from published work on inference runtimes, quantization, pruning, low-bit LLM quantization, and distillation [@nvidia2024tensorrt; @jacob2018; @han2015deep; @lin2023awq; @hinton2015distilling]; actual results vary with architecture, task, calibration data, and implementation quality. {#tbl-optimization-tradeoffs tbl-colwidths="[22,27,17,25,9]"}
: **The Optimization Trade-Offs**: Region 1 = Free Lunch, Region 2 = Efficient Trade, Region 3 = Danger Zone. Batch size affects training dynamics rather than model quality directly. These are representative engineering ranges synthesized from published work on inference runtimes, quantization, pruning, low-bit LLM quantization, and distillation [@nvidia2024tensorrt; @jacob2018; @han2015deep; @lin2023awq; @hinton2015distilling]; actual results vary with architecture, task, calibration data, and implementation quality. {#tbl-optimization-tradeoffs tbl-colwidths="[22,27,19,23,9]"}
The table reveals a pattern: techniques that preserve model structure (fusion, precision reduction) tend to be "free" or cheap, while techniques that alter structure (pruning, distillation) extract more savings but require careful tuning. Each deployment context imposes a binding constraint: memory capacity on mobile devices, latency on real-time systems, energy on battery-powered sensors. The optimization stack follows those constraints downward. Structural methods modify *what* computations occur, reducing the model's parameter count and operation count to fit tighter memory and compute budgets. Precision techniques reduce how many bits represent each value, directly shrinking memory footprint and accelerating arithmetic. Architectural approaches improve how efficiently the remaining operations execute on physical hardware, closing the gap between theoretical savings and measured performance.
@@ -1348,8 +1357,6 @@ Follow the three rows of @fig-iterative-pruning to see this gradual process in a
Consider applying one-shot pruning to the same network from the iterative pruning example. Instead of removing two channels at a time over multiple iterations, one-shot pruning eliminates all six channels simultaneously. Compare the single-row workflow in @fig-oneshot-pruning to the iterative case: removing 27 percent of the network's channels simultaneously causes the accuracy to drop significantly, from 0.995 to 0.914. Even after fine-tuning, the network only recovers to an accuracy of 0.943, which is a 5 percent degradation from the original unpruned network. While both iterative and one-shot pruning ultimately produce identical network structures, the gradual approach of iterative pruning better preserves model performance.
The choice between strategies depends on three interrelated factors. First, the sparsity target: higher reduction targets often necessitate iterative approaches to maintain accuracy, while moderate goals may be achievable with one-shot methods. Second, available resources: iterative pruning demands significant compute for multiple fine-tuning cycles, whereas one-shot approaches trade accuracy for speed. Third, the deployment timeline and target platform: one-shot methods enable faster deployment, but certain hardware architectures better support specific sparsity patterns, making iterative approaches more advantageous when time permits.
::: {#fig-oneshot-pruning fig-env="figure" fig-pos="htb" fig-cap="**One-Shot Pruning Impact**: All six channels (27 percent) are removed simultaneously, causing accuracy to drop from 0.995 to 0.914. Fine-tuning recovers only to 0.943, a 5 percent degradation compared to the 0.4 percent loss from iterative pruning, illustrating why gradual removal preserves accuracy more effectively." fig-alt="Single-row workflow showing one-shot pruning. CNN with six red-highlighted channels to prune, followed by accuracy drop from 0.995 to 0.914, fine-tuning gears, and partial recovery to 0.943."}
```{.tikz}
@@ -1547,6 +1554,8 @@ minimum width=27mm,minimum height=29mm,
:::
The choice between strategies depends on three interrelated factors. First, the sparsity target: higher reduction targets often necessitate iterative approaches to maintain accuracy, while moderate goals may be achievable with one-shot methods. Second, available resources: iterative pruning demands significant compute for multiple fine-tuning cycles, whereas one-shot approaches trade accuracy for speed. Third, the deployment timeline and target platform: one-shot methods enable faster deployment, but certain hardware architectures better support specific sparsity patterns, making iterative approaches more advantageous when time permits.
#### Lottery ticket hypothesis {#sec-model-compression-lottery-ticket-hypothesis-1b3d}
The pruning strategies in this chapter share a common assumption: we start with a trained network and then decide which parameters to remove. The relationship between network structure and trainability may run deeper than pruning strategies suggest: pruning may reveal inherently efficient subnetworks that were already hidden within the dense model, rather than merely deleting unnecessary weights after training.
@@ -2097,8 +2106,6 @@ If we factorize it with rank $k$ = `{python} LowRankFactorization.rank_k_value_s
This bandwidth-compute trade-off is the local version of the memory wall: execution becomes bottlenecked by moving weights rather than multiplying them. @Sec-hardware-acceleration-understanding-ai-memory-wall-3ea9 later examines the same phenomenon from the hardware side.
To see why this matters, study @fig-matrix-factorization: the matrix $M$ can be approximated by the product of matrices $U_k$ and $V_k^T$. For intuition, most fully connected layers in networks are stored as a projection matrix $M$, which requires $m{\times}n$ parameters to be loaded during computation. However, by decomposing and approximating it as the product of two lower-rank matrices, we only need to store $m \times k + k \times n$ parameters in terms of storage. Applying the factors directly costs $\mathcal{O}(k(m+n))$ for a vector input, instead of $\mathcal{O}(mn)$ for the original dense matrix; explicitly forming the dense product $UV$ would cost $\mathcal{O}(mkn)$ and should be avoided in inference [@Denton2014].
::: {#fig-matrix-factorization fig-env="figure" fig-pos="htb" fig-cap="**Low-Rank Factorization**: A weight matrix $M$ of size $m{\times}n$ is approximated as the product of two smaller matrices, $U_k$ $(m{\times}k)$ and $V_k^T$ $(k{\times}n)$, reducing storage from $m \times n$ to $m \times k + k \times n$ parameters at the cost of one additional matrix multiplication during inference." fig-alt="Three rectangular boxes showing matrix factorization. Large M matrix of size m by n approximately equals product of narrower U matrix of size m by k and wider V-transpose matrix of size k by n."}
```{.tikz}
@@ -2128,6 +2135,8 @@ To see why this matters, study @fig-matrix-factorization: the matrix $M$ can be
:::
To see why this matters, study @fig-matrix-factorization: the matrix $M$ can be approximated by the product of matrices $U_k$ and $V_k^T$. For intuition, most fully connected layers in networks are stored as a projection matrix $M$, which requires $m{\times}n$ parameters to be loaded during computation. However, by decomposing and approximating it as the product of two lower-rank matrices, we only need to store $m \times k + k \times n$ parameters in terms of storage. Applying the factors directly costs $\mathcal{O}(k(m+n))$ for a vector input, instead of $\mathcal{O}(mn)$ for the original dense matrix; explicitly forming the dense product $UV$ would cost $\mathcal{O}(mkn)$ and should be avoided in inference [@Denton2014].
LRMF applies to fully connected layers (large weight matrices) and convolutional layers (via depthwise-separable convolutions). The key trade-off: storage reduces from $\mathcal{O}(mn)$ to $\mathcal{O}(mk + kn)$, but inference requires an additional matrix multiplication. Choosing rank $k$ balances compression against information loss.
#### Tensor decomposition {#sec-model-compression-tensor-decomposition-5e9e}
@@ -2137,7 +2146,7 @@ LRMF applies to fully connected layers (large weight matrices) and convolutional
::: {#fig-tensor-decomposition fig-env="figure" fig-pos="htb" fig-cap="**Tensor Decomposition**: A 3D tensor with dimensions $M{\times}N{\times}T$ is decomposed into a sum of rank-one components, each formed by the outer product of three factor vectors (U, V, W). This extends low-rank matrix factorization to multi-dimensional data, reducing storage and computation for convolutional layers. Source: [@gholami2022]." fig-alt="3D tensor cube with dimensions M, N, T decomposed into sum of three factor matrices U, V, W of reduced dimensions. Small highlighted element shows how single tensor entry decomposes into factor products."}
```{.tikz}
\scalebox{0.7}{%
\scalebox{0.65}{%
\begin{tikzpicture}[line width=0.35pt,line join=round]
\begin{scope}
@@ -2377,6 +2386,11 @@ The main tensor-decomposition families differ in how they trade compression agai
Tensor decomposition applies to convolutional filters (approximating 4D weight tensors), attention mechanisms in transformers, and embedding layers in NLP models. The trade-offs mirror LRMF: compression vs. information loss, and the additional computational overhead of tensor contractions during inference. @Tbl-lrmf-tensor compares LRMF and tensor decomposition across applicable data structure, compression mechanism, and computational cost.
```{=latex}
\begingroup
\renewcommand{\arraystretch}{1.3}
```
| **Feature** | **Low-Rank Matrix Factorization (LRMF)** | **Tensor Decomposition** |
|:------------------------------|:-----------------------------------------------------------------------|:----------------------------------------------------------------------------------|
| **Applicable Data Structure** | Two-dimensional matrices | Multi-dimensional tensors |
@@ -2390,6 +2404,10 @@ Tensor decomposition applies to convolutional filters (approximating 4D weight t
: **Dimensionality & Factorization**: Low-rank matrix factorization (LRMF) and tensor decomposition reduce model storage requirements by representing data with fewer parameters, but introduce computational trade-offs during inference; LRMF applies to two-dimensional matrices, while tensor decomposition extends this approach to multi-dimensional tensors for greater compression potential. {#tbl-lrmf-tensor tbl-colwidths="[26,37,37]"}
```{=latex}
\endgroup
```
In practice, LRMF and tensor decomposition can be combined: fully connected layers compressed via LRMF while convolutional kernels use tensor decomposition. The choice depends on the model's structure and whether memory or latency is the primary constraint.
The techniques explored so far (pruning, distillation, and factorization) all optimize *existing* architectures. Neural architecture search takes a different approach: discovering architectures that are efficient *by construction*.
@@ -2400,7 +2418,7 @@ Pruning, knowledge distillation, and other techniques explored in previous secti
The three-stage feedback loop in @fig-nas-flow captures the essence of how NAS works. NAS[^fn-nas-hardware-aware] operates through three interconnected stages: defining the search space (architectural components and constraints), applying search strategies (reinforcement learning [@zoph2017neural], evolutionary algorithms, or gradient-based methods) to explore candidate architectures, and evaluating performance to ensure discovered designs satisfy accuracy and efficiency objectives. The key insight is that this feedback loop allows the search to learn from each evaluation, progressively focusing on promising regions of the architecture space. This automation enables the discovery of novel architectures that often match or surpass human-designed models while requiring substantially less expert effort.
[^fn-nas-hardware-aware]: [offset=-55mm] **Hardware-Aware NAS**: Optimizes measured latency rather than FLOPs, which can diverge by 3--5$\times$ when memory access or operator support dominates. @tan2019 fed device latency into search and found architectures 1.8$\times$ faster than MobileNetV2 at higher accuracy. \index{NAS!hardware-aware}
[^fn-nas-hardware-aware]: **Hardware-Aware NAS**: Optimizes measured latency rather than FLOPs, which can diverge by 3--5$\times$ when memory access or operator support dominates. @tan2019 fed device latency into search and found architectures 1.8$\times$ faster than MobileNetV2 at higher accuracy. \index{NAS!hardware-aware}
::: {#fig-nas-flow fig-env="figure" fig-pos="htb" fig-cap="**Neural Architecture Search Flow**: Three components form a feedback loop: a Search Space defines candidate operations, a Search Strategy selects architectures, and a Performance Estimation Strategy evaluates each candidate. The strategy iterates by feeding performance estimates back into the search until convergence." fig-alt="Three-box flowchart showing NAS process. Search Space box feeds into Search Strategy box, which exchanges architecture and performance estimate with Performance Estimation Strategy box in a feedback loop."}
@@ -2524,23 +2542,23 @@ arc[start angle=270, end angle=360, radius=3mm]
filllcolor=orange!05!,Linewidth=0.7pt, filllcirclecolor=green!60}};
\node[below =-7pt of C1](B1){Search Space $\mathcal{A}$};
\node[Circ,right=3.5 of C1](C2){};
\node[Circ,right=4 of C1](C2){};
\pic[shift={(0,0)}] at (C2){griddot={scalefac=1,picname=1,drawcolor=BlueD,
filllcolor=green!10!,Linewidth=2.75pt, filllcirclecolor=cyan!20}};
\node[below =-7pt of C2](B2){Search Strategy};
\node[Circ,right=3.5 of C2](C3){};
\node[Circ,right=4 of C2](C3){};
\pic[shift={(-0.80,-0.9)}] at (C3){graph={scalefac=1,picname=1,filllcolor=BlueLine, Linewidth=1.0pt}};
\node[below =-7pt of C3,align=center](B3){Performance\\ Estimation Strategy};
\node[below =-7pt of C3,align=center](B3){Performance Estimation Strategy};
\scoped[on background layer]
\node[draw=BackLine,inner xsep=5mm,inner ysep=6mm,minimum height=40mm,
yshift=6.5mm,fill=BackColor!10,fit=(C2)(C3)(B3),line width=1pt](BB1){};
\node[draw=BackLine,inner xsep=5mm,inner ysep=5mm,minimum height=37mm,
yshift=5.5mm,fill=BackColor!10,fit=(C2)(C3)(B3),line width=1pt](BB1){};
\node[below=4pt of BB1.north,inner sep=0pt,
anchor=north,align=center]{One-shot approach:\\
learning model architecture parameters and weights together};
\draw[Arr](C1)--(C2);
\draw[Arr](C2.15)--node[text=black,above=2pt,align=center]{Architecture $A\in\mathcal{A}$}(C3.165);
\draw[Arr](C3.195)--node[text=black,below,align=center]{Performance\\ estimate of $A$}(C2.345);
\draw[Arr](C3.195)--node[text=black,below,align=center]{Performance\ estimate of $A$}(C2.345);
\end{tikzpicture}
```
@@ -2559,7 +2577,7 @@ $$
\theta^*(\alpha) = \operatorname{arg\,min}_{\theta} \mathcal{L}_{\text{train}}(\theta, \alpha)
$$
[^fn-bilevel-optimization-nas]: [offset=-20mm] **Bi-Level Optimization**: A formulation where one optimization problem sits inside another. In NAS, the outer level selects an architecture while the inner level trains that candidate's weights, so early methods paid a full training cost for every architecture evaluated. This nesting is why early NAS required 22,400 GPU-days; weight-sharing methods amortize one training run across many candidates, reducing search cost by roughly 1,000$\times$. \index{Optimization!bi-level, NAS cost}
[^fn-bilevel-optimization-nas]: **Bi-Level Optimization**: A formulation where one optimization problem sits inside another. In NAS, the outer level selects an architecture while the inner level trains that candidate's weights, so early methods paid a full training cost for every architecture evaluated. This nesting is why early NAS required 22,400 GPU-days; weight-sharing methods amortize one training run across many candidates, reducing search cost by roughly 1,000$\times$. \index{Optimization!bi-level, NAS cost}
The core challenge is the cost of the inner loop: evaluating each candidate requires expensive training. A search space with just 10 choices across 20 layers yields $10^{20}$ architectures, making exhaustive search impossible. Efficient NAS methods address this by restricting the search space, using faster search strategies, or accelerating evaluation.
@@ -2581,7 +2599,7 @@ Search strategies determine how to explore the architecture space efficiently wi
| **Evolutionary Algorithms** | 200--500 GPU-days | Parallel infrastructure available | Requires large populations |
| **Gradient-Based (DARTS)** | 1--4 GPU-days | Limited compute budget | May converge to suboptimal local minima |
: **NAS Search Strategy Comparison**: Representative trade-offs between search efficiency, use cases, and limitations for different NAS approaches. Reinforcement learning offers unconstrained exploration at high cost, evolutionary methods exploit parallelism, and gradient-based or weight-sharing approaches achieve dramatic speedups with potential optimality trade-offs [@zoph2017neural; @real2019; @liu2019darts; @pham2018efficient]. {#tbl-nas-strategies tbl-colwidths="[23,18,32,27]"}
: **NAS Search Strategy Comparison**: Representative trade-offs between search efficiency, use cases, and limitations for different NAS approaches. Reinforcement learning offers unconstrained exploration at high cost, evolutionary methods exploit parallelism, and gradient-based or weight-sharing approaches achieve dramatic speedups with potential optimality trade-offs [@zoph2017neural; @real2019; @liu2019darts; @pham2018efficient]. {#tbl-nas-strategies tbl-colwidths="[23,18,34,25]"}
Reinforcement learning\index{Reinforcement Learning!NAS application} based NAS\index{NAS!reinforcement learning strategy}\index{Neural Architecture Search|see{NAS}} treats architecture search as a sequential decision problem where a controller generates architectures and receives accuracy as reward. The controller (typically an LSTM) learns to propose better architectures over time through policy gradient optimization. This approach discovered high-performing architectures like NASNet\index{NASNet!RL-discovered architecture}, but its inner loop is expensive because every reward requires training a candidate architecture; @zoph2017neural evaluated 12,800--22,400 candidates, totaling 22,400 GPU-days. That cost explains why practical NAS work moved toward weight sharing and lower-cost performance predictors.
@@ -2706,7 +2724,7 @@ The viability of each approach depends on how much precision a model can shed be
import numpy as np
from book.tools.figures import style as book_style
fig, ax, COLORS, plt = book_style.setup_plot(figsize=(10, 5))
fig, ax, COLORS, plt = book_style.setup_plot(figsize=(10, 4.75))
# —Data ---
@@ -2817,14 +2835,6 @@ These same physics apply at data center scale: distributed training systems use
:::
| **Operation** | **Bit-Width** | **Relative Energy** |
|:----------------|:------------------------------------------------------------------------:|:--------------------------------------------------------------------:|
| **Integer Add** | 8-bit | 1$\times$ |
| **Float Add** | 32-bit | `{python} QuantizationEnergyPhysics.energy_float_add_ratio_mult_str` |
| **DRAM Read** | `{python} QuantizationEnergyPhysics.energy_dram_read_bits_value_str`-bit | `{python} QuantizationEnergyPhysics.energy_dram_read_ratio_mult_str` |
: **The Energy Hierarchy**: Relative energy cost of representative arithmetic and memory operations, normalized to an 8-bit integer add. A 32-bit float add costs `{python} QuantizationEnergyPhysics.energy_float_add_ratio_mult_str` more than its 8-bit integer counterpart, but a single `{python} QuantizationEnergyPhysics.energy_dram_read_bits_value_str`-bit DRAM read costs `{python} QuantizationEnergyPhysics.energy_dram_read_ratio_mult_str` more, placing DRAM access more than four orders of magnitude above on-chip compute. This gap is why bit-width reductions that shrink data movement dominate the energy budget of inference far more than the arithmetic savings alone would suggest [@horowitz2014]. {#tbl-model-compression-energy-hierarchy tbl-colwidths="[33,33,33]"}
These savings explain why neural networks tolerate aggressive quantization: the energy cost of higher precision exceeds the accuracy benefit for most applications, and reduced precision turns capacity pressure into bandwidth relief on the target device. How much precision a model can shed before accuracy collapses, and how large the resulting speedup proves to be, are the questions the quantization section develops in full.
```{python}
@@ -2875,6 +2885,14 @@ class EnergyCosts:
@Tbl-model-compression-energy-hierarchy established the canonical gap between arithmetic and data movement; precision granularity extends it. @Fig-quantized-energy gives representative operation-level energy costs across more formats and across the SRAM hierarchy: a 32-bit integer addition costs `{python} EnergyCosts.energy_add_int32_pj_per_op_str`, while an 8-bit integer addition is just `{python} EnergyCosts.energy_add_int8_pj_per_op_str`. Floating-point arithmetic follows the same direction: a 32-bit floating-point addition consumes approximately `{python} EnergyCosts.energy_add_fp32_pj_per_op_str`, whereas a 16-bit floating-point addition requires `{python} EnergyCosts.energy_add_fp16_pj_per_op_str`. The chart's headline is the gap between the two groups: even an on-chip SRAM read costs roughly two orders of magnitude more than the cheapest arithmetic operation, so where an operand lives matters more than how it is computed. DRAM access\index{DRAM!energy cost of access} is more expensive still. These savings compound across large-scale models operating over billions of operations. Reducing a model's energy footprint therefore contributes to sustainable and accessible AI deployment: it helps mitigate the environmental impact\index{Environmental Impact!model compression benefits} of large-scale ML training and inference as AI workloads scale [@patterson2021carbon], and it expands the reach of machine learning into low-resource environments, from rural healthcare to autonomous systems operating in the field.
| **Operation** | **Bit-Width** | **Relative Energy** |
|:----------------|:------------------------------------------------------------------------:|:--------------------------------------------------------------------:|
| **Integer Add** | 8-bit | 1$\times$ |
| **Float Add** | 32-bit | `{python} QuantizationEnergyPhysics.energy_float_add_ratio_mult_str` |
| **DRAM Read** | `{python} QuantizationEnergyPhysics.energy_dram_read_bits_value_str`-bit | `{python} QuantizationEnergyPhysics.energy_dram_read_ratio_mult_str` |
: **The Energy Hierarchy**: Relative energy cost of representative arithmetic and memory operations, normalized to an 8-bit integer add. A 32-bit float add costs `{python} QuantizationEnergyPhysics.energy_float_add_ratio_mult_str` more than its 8-bit integer counterpart, but a single `{python} QuantizationEnergyPhysics.energy_dram_read_bits_value_str`-bit DRAM read costs `{python} QuantizationEnergyPhysics.energy_dram_read_ratio_mult_str` more, placing DRAM access more than four orders of magnitude above on-chip compute. This gap is why bit-width reductions that shrink data movement dominate the energy budget of inference far more than the arithmetic savings alone would suggest [@horowitz2014]. {#tbl-model-compression-energy-hierarchy tbl-colwidths="[33,33,33]"}
::: {#fig-quantized-energy fig-env="figure" fig-pos="htb" fig-cap="**Energy per Operation by Precision**: Bar chart comparing energy in picojoules for representative integer arithmetic operations (INT8 add: 0.03 pJ; INT32 multiply: 3.10 pJ) and SRAM memory accesses (5 to 50 pJ by cache size). Lower precision yields order-of-magnitude energy savings, while memory accesses can dominate arithmetic costs. Energy figures from [@horowitz2014]." fig-alt="Bar chart comparing energy consumption in picojoules for integer arithmetic operations and memory accesses. INT8 add uses 0.03 pJ, INT32 multiply uses 3.10 pJ, and SRAM reads range from 5 to 50 pJ depending on cache size."}
```{.tikz}
@@ -3028,6 +3046,8 @@ The framework section's energy hierarchy already exposed this asymmetry; here it
These energy savings take on a different character for models where memory capacity, not compute, is the binding constraint\index{DLRM!embedding quantization}.
The DLRM example shows quantization as a way to make a huge memory object placeable. The same logic reappears at the other end of the scale, where the object is small but the device is smaller still: the keyword spotting (KWS) DS-CNN\index{Keyword Spotting!quantization imperative} lighthouse faces a TinyML deployment where compression becomes an existential requirement.
::: {#lhs-model-compression-dlrm-embedding-quantization .callout-lighthouse title="DLRM and embedding quantization"}
The DLRM lighthouse\index{DLRM Lighthouse!compression} (@sec-network-architectures-sparse-architectures-recommendation-systems-5d19) presents a compression challenge where memory capacity, rather than compute throughput or memory bandwidth, is the binding constraint [@naumov2019deep]. Its embedding tables can reach terabytes in size, far exceeding GPU memory.
@@ -3036,7 +3056,9 @@ For DLRM, quantization is not about faster math; it is about storage density\ind
:::
The DLRM example shows quantization as a way to make a huge memory object placeable. The same logic reappears at the other end of the scale, where the object is small but the device is smaller still: the keyword spotting (KWS) DS-CNN\index{Keyword Spotting!quantization imperative} lighthouse faces a TinyML deployment where compression becomes an existential requirement.
Together, the two lighthouses separate the two reasons quantization matters. DLRM uses lower precision to make enormous tables fit; TinyML uses lower precision to keep every inference within a tiny energy and SRAM budget. Beyond direct compute savings, reducing numerical precision also lowers memory energy consumption, which often dominates total system power. Lower-precision representations reduce data storage requirements and memory bandwidth usage, leading to fewer and more efficient memory accesses. Accessing memory, particularly off-chip DRAM, is far more energy-intensive than performing arithmetic operations: the representative `{python} EnergyDividend.dram_read_bits_value_str`-bit DRAM read used in this chapter costs `{python} EnergyDividend.energy_dram_pj_str`, compared with picojoule-scale cache and arithmetic operations. An instruction's total energy can therefore be dominated by memory access patterns rather than computation[^fn-int8-energy-deployment].
[^fn-int8-energy-deployment]: **INT8 Energy Impact**: The energy dominance of memory access is extreme: with the Horowitz constants used here, a single `{python} EnergyDividend.dram_read_bits_value_str`-bit DRAM read costs roughly `{python} EnergyDividend.dram_read_vs_int8_mac_mult_str` the energy of an INT8 multiply-accumulate [@horowitz2014]. Quantizing from FP32 to INT8 attacks this disparity on both fronts—4$\times$ fewer bytes moved *and* cheaper arithmetic per operation. Representative MobileNetV2 INT8 measurements show multi-fold energy-per-inference reductions, with most savings coming from reduced memory traffic rather than cheaper arithmetic. \index{Quantization!energy impact}
```{python}
#| echo: false
@@ -3087,10 +3109,6 @@ In FP32, even the compact DS-CNN architecture moves 4$\times$ more weight bytes
:::
Together, the two lighthouses separate the two reasons quantization matters. DLRM uses lower precision to make enormous tables fit; TinyML uses lower precision to keep every inference within a tiny energy and SRAM budget. Beyond direct compute savings, reducing numerical precision also lowers memory energy consumption, which often dominates total system power. Lower-precision representations reduce data storage requirements and memory bandwidth usage, leading to fewer and more efficient memory accesses. Accessing memory, particularly off-chip DRAM, is far more energy-intensive than performing arithmetic operations: the representative `{python} EnergyDividend.dram_read_bits_value_str`-bit DRAM read used in this chapter costs `{python} EnergyDividend.energy_dram_pj_str`, compared with picojoule-scale cache and arithmetic operations. An instruction's total energy can therefore be dominated by memory access patterns rather than computation[^fn-int8-energy-deployment].
[^fn-int8-energy-deployment]: **INT8 Energy Impact**: The energy dominance of memory access is extreme: with the Horowitz constants used here, a single `{python} EnergyDividend.dram_read_bits_value_str`-bit DRAM read costs roughly `{python} EnergyDividend.dram_read_vs_int8_mac_mult_str` the energy of an INT8 multiply-accumulate [@horowitz2014]. Quantizing from FP32 to INT8 attacks this disparity on both fronts—4$\times$ fewer bytes moved *and* cheaper arithmetic per operation. Representative MobileNetV2 INT8 measurements show multi-fold energy-per-inference reductions, with most savings coming from reduced memory traffic rather than cheaper arithmetic. \index{Quantization!energy impact}
Reducing numerical precision thus improves efficiency on two fronts: faster computation and less data movement. This dual benefit is especially valuable for hardware accelerators and edge devices, where memory bandwidth and power efficiency are binding constraints.
#### Performance gains {#sec-model-compression-performance-gains-0656}
@@ -3111,14 +3129,14 @@ The practical payoff of quantization becomes concrete in @fig-quantization-impac
/pgf/number format/.cd,
1000 sep={},
width=75mm,
height=75mm,
height=62mm,
axis line style={draw=none},
ybar stacked, ymin=0,
bar width=11mm,
title style={font=\fontsize{8pt}{8}\selectfont\usefont{T1}{phv}{m}{n},yshift=-2pt},
title style={font=\fontsize{8pt}{8}\selectfont\usefont{T1}{phv}{m}{n},yshift=-4pt},
symbolic x coords={Inception\_v3,MobileNet\_v1, ResNet\_v2},
xtick=data,
legend style={at={(0.85,0.92)}, anchor=north},
legend style={at={(0.85,0.9)}, anchor=north},
legend cell align=left,
legend style={fill=BrownL!40,draw=BrownLine,row sep=1.85pt,
font=\fontsize{7pt}{7}\selectfont\usefont{T1}{phv}{m}{n}},
@@ -3149,14 +3167,17 @@ The practical payoff of quantization becomes concrete in @fig-quantization-impac
({Inception\_v3},800)
({MobileNet\_v1},700)
({ResNet\_v2},300)};
\addplot [fill=Activation!90,draw=none] coordinates {
\addplot [fill=Activation!90,draw=none,
every node near coord/.append style={xshift=26pt,
/pgf/number format/assume math mode=true,
font=\fontsize{6pt}{6}\selectfont\usefont{T1}{phv}{m}{n}, anchor=center}] coordinates {
({Inception\_v3},500)
({MobileNet\_v1},30)
({ResNet\_v2},70)};
\end{axis}
\end{scope}
%%%%RIGHT
\begin{scope}[local bounding box=RR2,shift={(7,0)}]
\begin{scope}[local bounding box=RR2,shift={(7.5,0)}]
\begin{axis}[mybarstyle,
title={Model\_Size},
nodes near coords={\pgfmathprintnumber{\pgfplotspointmeta}~MB},
@@ -3165,19 +3186,22 @@ The practical payoff of quantization becomes concrete in @fig-quantization-impac
({Inception\_v3},135)
({MobileNet\_v1},45)
({ResNet\_v2},24)};
\addplot [fill=Activation!90,draw=none] coordinates {
\addplot [fill=Activation!90,draw=none,
every node near coord/.append style={xshift=26pt,
/pgf/number format/assume math mode=true,
font=\fontsize{6pt}{6}\selectfont\usefont{T1}{phv}{m}{n}, anchor=center}] coordinates {
({Inception\_v3},71)
({MobileNet\_v1},4)
({ResNet\_v2},13)};
\legend{FP32,INT8}
\coordinate (legend) at (axis description cs:0.85,0.92);
\coordinate (legend) at (axis description cs:0.85,0.9);
\end{axis}
\node[fill=white,above=1pt of legend,anchor=south,
font=\fontsize{8pt}{8}\selectfont\usefont{T1}{phv}{m}{n}]{Precision};
\end{scope}
\node[draw=none,inner sep=0pt,fit=(RR)(RR2)](BB){};
\node[above=0pt of BB]{Impact of Quantization on Inference Time and Model Size};
\node[above=-1pt of BB]{Impact of Quantization on Inference Time and Model Size};
\node[below=-2pt of BB]{Model};
\end{tikzpicture}
```
@@ -3667,7 +3691,7 @@ Compare the three bit layouts in @fig-3float to see exactly where the bits go—
](B3){\textbf{23-bit} mantissa};
\node[left=2mmof B1]{\textbf{FP32}};
\begin{scope}[shift={(0,-2)}]
\begin{scope}[shift={(0,-1.7)}]
\node[Box,fill=col1,anchor=south west,minimum width=30
](BB1){\textbf{1-bit}\\sign};
\node[Box,fill=col2,right=of BB1,minimum width=100
@@ -3677,7 +3701,7 @@ Compare the three bit layouts in @fig-3float to see exactly where the bits go—
\node[left=2mmof BB1]{\textbf{FP16}};
\end{scope}
\begin{scope}[shift={(0,-4)}]
\begin{scope}[shift={(0,-3.4)}]
\node[Box,fill=col1,anchor=south west,minimum width=30
](DB1){\textbf{1-bit}\\sign};
\node[Box,fill=col2,right=of DB1,minimum width=160
@@ -3769,7 +3793,6 @@ Three approaches form a complexity ladder. Post-training quantization (PTQ) redu
```{.tikz}
\begin{tikzpicture}[line join=round,font=\usefont{T1}{phv}{m}{n}]
\tikzset{
Box/.style={align=flush center,
inner xsep=2pt,
@@ -3779,7 +3802,7 @@ Box/.style={align=flush center,
rounded corners,
fill=OrangeL!40,
text width=50mm,
minimum width=50mm, minimum height=24mm
minimum width=50mm, minimum height=22mm
}
}
@@ -3933,13 +3956,13 @@ pics/brick/.style = {
\draw[Line,line width=2.6pt,violet]
(S\k)+(\i:0.7*\ra) arc[start angle=\i, end angle=\newX, radius=0.7*\ra];
}
\draw[LineA](S1.220)--++(220:2.15)coordinate(MA);
\draw[LineA](S1.220)--++(210:2.15)coordinate(MA);
\node[Box,anchor=north](FO)at(MA){\textbf{Foundational}\\ Post-Training Quantization
FP32/FP16/INT8 Basic Calibration};
\draw[LineA](S2.220)--++(220:2.15)coordinate(ST);
\draw[LineA](S2.220)--++(210:2.15)coordinate(ST);
\node[Box,anchor=north](PR)at(ST){\textbf{Production}\\ Quantization-Aware Training
Mixed-Precision Per-Channel Quantization};
\draw[LineA](S3.220)--++(220:2.15)coordinate(ST1);
\draw[LineA](S3.220)--++(210:2.15)coordinate(ST1);
\node[Box,anchor=north](RE)at(ST1){\textbf{Research Frontier}\\ INT4/INT2
Binary/Ternary Networks Extreme Quantization};
%
@@ -3976,10 +3999,6 @@ where:
@Lst-quantization_example demonstrates uniform quantization from FP32 to INT8, achieving 4$\times$ memory reduction while measuring the resulting quantization error. Once a model is quantized, the runtime performs inference using integer arithmetic, which is significantly more efficient than floating-point operations on most hardware platforms [@gholami2022].
An alternative, nonuniform quantization\index{Quantization!nonuniform}, assigns finer-grained precision to numerical ranges that are more densely populated, which can preserve accuracy for models whose weight distributions concentrate around specific values. Nonuniform schemes require more complex calibration and are less common in production, but they can be effective for models particularly sensitive to precision changes.
PTQ works well for computer vision models, where CNNs often tolerate quantization without significant accuracy loss. Models that rely on small numerical differences, such as NLP transformers or speech recognition systems, may require quantization-aware training or nonuniform strategies to retain performance.
::: {#lst-quantization_example lst-cap="**Uniform Quantization**: Converts FP32 weights to INT8 format, achieving 4$\times$ memory reduction while measuring quantization error."}
```{.python}
@@ -4013,6 +4032,10 @@ print(
:::
An alternative, nonuniform quantization\index{Quantization!nonuniform}, assigns finer-grained precision to numerical ranges that are more densely populated, which can preserve accuracy for models whose weight distributions concentrate around specific values. Nonuniform schemes require more complex calibration and are less common in production, but they can be effective for models particularly sensitive to precision changes.
PTQ works well for computer vision models, where CNNs often tolerate quantization without significant accuracy loss. Models that rely on small numerical differences, such as NLP transformers or speech recognition systems, may require quantization-aware training or nonuniform strategies to retain performance.
##### Calibration {#sec-model-compression-calibration-c650}
An important aspect of PTQ is the calibration\index{Quantization!calibration} step, which selects the clipping range $[\alpha, \beta]$ for quantizing model weights and activations. The effectiveness of precision reduction depends heavily on this chosen range: without proper calibration, quantization may cause significant accuracy degradation. Calibration ensures that the chosen range minimizes information loss and preserves model performance.
@@ -4082,8 +4105,6 @@ For example, consider quantizing activations that originally range between -6 an
Common calibration methods include **Max**\index{Quantization Calibration!max method} (uses maximum absolute value, simple but susceptible to outliers), **Entropy**\index{Quantization Calibration!entropy method} (minimizes KL divergence between original and quantized distributions, TensorRT's default), and **Percentile**\index{Quantization Calibration!percentile method} (clips to a percentile, for example, 99 percent, avoiding outlier impact). @Fig-resnet-activations-histogram shows why outlier handling matters: ResNet50 activations exhibit long tails where outliers can skew the quantization range.
Calibration ranges can be **symmetric**\index{Quantization!symmetric} (equal positive and negative scaling) or **asymmetric**\index{Quantization!asymmetric} (different scaling factors for each side, useful when distributions are skewed). The choice of method and range significantly affects quantized model accuracy.
::: {#fig-resnet-activations-histogram fig-env="figure" fig-pos="htb" fig-cap="**Activation Distribution**: ResNet-50 layer activations exhibit a long tail, with outlier values that can lead to inefficient precision use if not handled carefully. Source: [@wu2020integer]." fig-alt="Histogram of ResNet50 activation values showing right-skewed distribution. Most values cluster near zero with long tail extending to outliers around 2.1, demonstrating challenge for quantization range selection."}
```{.tikz}
@@ -4091,7 +4112,7 @@ Calibration ranges can be **symmetric**\index{Quantization!symmetric} (equal pos
\definecolor{barfill}{cmyk}{0.7, 0.4, 0, 0}
\begin{axis}[
width=16cm,
height=80mm,
height=78mm,
xmin=0, xmax=2.5,
ymin=0, ymax=1.06,
axis lines=left,
@@ -4178,6 +4199,8 @@ font=\usefont{T1}{phv}{m}{n}\bfseries]{Outlier tail\\
:::
Calibration ranges can be **symmetric**\index{Quantization!symmetric} (equal positive and negative scaling) or **asymmetric**\index{Quantization!asymmetric} (different scaling factors for each side, useful when distributions are skewed). The choice of method and range significantly affects quantized model accuracy.
##### Tuning quantization ranges {#sec-model-compression-tuning-quantization-ranges-439d}
A key challenge in post-training quantization is selecting the appropriate calibration range $[\alpha, \beta]$ to map floating-point values into a lower-precision representation. The choice of this range directly affects the quantization error and, consequently, the accuracy of the quantized model. @Fig-calibration-ranges contrasts the two primary calibration strategies: symmetric calibration and asymmetric calibration.
@@ -4264,6 +4287,10 @@ LineD/.style={dashed,black,line width=0.75pt},
Compare the two mapping diagrams side by side in @fig-calibration-ranges. Symmetric calibration (left) maps $[-1, 1]$ to $[-127, 127]$ with zero preserved, making it simpler to implement and well suited for zero-centered weight distributions. Asymmetric calibration (right) uses different ranges ($\alpha = -0.5$, $\beta = 1.5$), better using the quantized range for skewed distributions at the cost of additional complexity. Most frameworks (TensorRT, PyTorch) support both modes. The conceptual difference is clear from the diagrams, but the actual computation of scale and zero-point parameters requires a concrete formula. A worked example makes those parameters explicit.
##### Granularity {#sec-model-compression-granularity-93b3}
After determining the clipping range, the next optimization step is adjusting the granularity of that range to retain as much accuracy as possible. In CNNs, the input activations of a layer undergo convolution with multiple filters, each of which may have a unique range of values. The quantization process must account for these differences to preserve model performance.
```{python}
#| echo: false
#| label: quantization-math-calc
@@ -4353,10 +4380,6 @@ Thus, the real value $0.0$ is represented by the integer `{python} QuantizationM
:::
##### Granularity {#sec-model-compression-granularity-93b3}
After determining the clipping range, the next optimization step is adjusting the granularity of that range to retain as much accuracy as possible. In CNNs, the input activations of a layer undergo convolution with multiple filters, each of which may have a unique range of values. The quantization process must account for these differences to preserve model performance.
This variation is strikingly visible in @fig-quantization-granularity: notice how the range for Filter one is significantly smaller than that for Filter 3. The two annotated columns contrast the strategies this variation motivates. A single shared layerwise range clips the narrow filters' headroom to fit the widest one, while per-filter channelwise ranges fit each distribution independently, the difference the red and blue dashed bounds illustrate. The precision with which the clipping range $[\alpha, \beta]$ is determined becomes an important factor in effective quantization. This variability in ranges is why different granularity-based quantization strategies are employed.
::: {#fig-quantization-granularity fig-env="figure" fig-pos="htb" fig-cap="**Quantization Range Variation**: Different convolutional filters exhibit unique activation ranges, necessitating per-filter quantization to minimize accuracy loss during quantization. Adjusting the granularity of clipping ranges, as shown by the differing scales for each filter, optimizes the trade-off between model size and performance. Source: [@gholami2022]." fig-alt="Four rows showing CNN filters with Gaussian weight distributions. Each filter has different clipping ranges shown as red and blue dashed lines. Layer-wise clipping uses same range; channel-wise uses per-filter ranges."}
@@ -4426,7 +4449,7 @@ Box/.style={inner xsep=2pt,
\end{scope}
%%%%%%%%%%%%%%%%%%%%%%%%%%
%second row
\begin{scope}[shift={(0,-3.0)}]
\begin{scope}[shift={(0,-2.75)}]
\begin{scope}[line width=0.5pt]
\newcommand{\Depth}{1.3}
\newcommand{\Height}{1.3}
@@ -4469,7 +4492,7 @@ Box/.style={inner xsep=2pt,
\end{scope}
%%%%%%%%%%%%%%%%%%%%%%%%%
%third row
\begin{scope}[shift={(0,-6.0)}]
\begin{scope}[shift={(0,-5.5)}]
\begin{scope}[local bounding box=SF3,line width=0.5pt]
\newcommand{\Depth}{1.3}
\newcommand{\Height}{1.3}
@@ -4512,7 +4535,7 @@ Box/.style={inner xsep=2pt,
\end{scope}
%%%%%%%%%%%%%%%%%%%%%%%%%%
%fourth row
\begin{scope}[shift={(0,-9.25)}]
\begin{scope}[shift={(0,-8.65)}]
\begin{scope}[local bounding box=SFC,line width=0.5pt]
\newcommand{\Depth}{1.3}
\newcommand{\Height}{1.3}
@@ -4589,7 +4612,7 @@ Quantization granularity\index{Quantization!granularity} is the control knob tha
| Channelwise\index{Quantization!channelwise} | One range per filter | The current standard, balancing accuracy and efficiency |
| Sub-channelwise\index{Quantization!sub-channelwise} | Ranges within each filter | Maximum precision but significant overhead |
: **Quantization Granularity Levels**: Granularity trades calibration overhead for accuracy, from a single per-layer range to per-filter sub-ranges. {#tbl-quantization-granularity tbl-colwidths="[22,33,45]"}
: **Quantization Granularity Levels**: Granularity trades calibration overhead for accuracy, from a single per-layer range to per-filter sub-ranges. {#tbl-quantization-granularity tbl-colwidths="[15,29,56]"}
Channelwise quantization has become the dominant approach, providing significant accuracy improvements over layerwise quantization with minimal computational overhead. With granularity determined, the next consideration is what to quantize. Neural networks contain two primary numerical components: the static weights learned during training and the dynamic activations computed during inference. Each presents distinct quantization challenges.
@@ -4601,7 +4624,6 @@ The granularity choice controls range sharing; the next decision is which tensor
```{.tikz}
\begin{tikzpicture}[font=\small\usefont{T1}{phv}{m}{n}]
\tikzset{%
helvetica/.style={align=flush center,font=\small\usefont{T1}{phv}{m}{n}},
Line/.style={line width=1.0pt,black!50,text=black},
@@ -4620,12 +4642,12 @@ Box/.style={inner xsep=2pt,
\node[Box,right=of B2,fill=GreenL,draw=GreenLine](B3){Quantization};
\node[Box,node distance=1.4,right=of B3,fill=BrownL,draw=BrownLine](B4){Activation};
\node[Box,right=of B4,fill=RedL,draw=RedLine](B5){FP16\\ Output};
\node[Box,above left=0.8 and 0 of B1,fill=VioletL2,draw=VioletLine2](GB2){Quantization};
\node[Box,above left=0.2 and 0 of B1,fill=VioletL2,draw=VioletLine2](GB2){Quantization};
\node[Box,left=of GB2,fill=VioletL2,draw=VioletLine2](GB1){Float input};
\node[Box,below left=0.8 and 0 of B1,fill=VioletL2,draw=VioletLine2](DB2){Quantization};
\node[Box,below left=0.2 and 0 of B1,fill=VioletL2,draw=VioletLine2](DB2){Quantization};
\node[Box,left=of DB2,fill=VioletL2,draw=VioletLine2](DB1){Float input};
%%%
\begin{scope}[font=\fontsize{7pt}{7}\selectfont\usefont{T1}{phv}{m}{n},scale=0.8,shift={($(B4)+(0,-3.5)$)}]
\begin{scope}[font=\fontsize{7pt}{7}\selectfont\usefont{T1}{phv}{m}{n},scale=0.75,shift={($(B4)+(0,-3.25)$)}]
\draw[line width=0.5pt, -{Latex[length=6pt,width=4pt]}] (-3,0)--(3,0)node[below, xshift=-0.12cm]{$x$};
\draw[line width=0.5pt, -{Latex[length=6pt,width=4pt]}] (0,-1.8)--(0,2.2)node[left, yshift=-0.15cm]{$y$};
\draw[xscale=1, yscale=1, line width=1.25pt, domain=-2.9:2.9,smooth,
@@ -4659,7 +4681,7 @@ Activation quantization refers to the process of quantizing the activation value
Activation-aware methods such as **Activation-aware Weight Quantization (AWQ)**\index{Quantization!activation-aware weight (AWQ)}[^fn-awq-salient-weights] compress and accelerate LLMs. This approach is particularly relevant for our GPT-2/Llama Lighthouse\index{Llama!AWQ quantization}, which is memory-bandwidth bound. By protecting only a small fraction of the most salient weights (approximately 1 percent) based on activation magnitude, AWQ enables effective INT4 weight quantization. This reduces the memory traffic required to load the massive parameter set for every token generation, directly attacking the primary bottleneck of generative inference [@lin2023awq].
[^fn-awq-salient-weights]: **Activation-Aware Weight Quantization (AWQ)**: Salience is determined by activation magnitude, not weight magnitude--a distinction that matters because a small weight multiplied by a large activation produces a large output contribution. AWQ protects about 1 percent of salient weight channels through activation-aware scaling while quantizing most weights to low-bit formats, so a 7-billion-parameter FP16 weight set that would occupy about 14 GB can move toward an INT4-class weight footprint of about 3.5 GB before metadata, scales, and kernel-specific packing overheads [@lin2023awq]. \index{AWQ!bandwidth reduction}
[^fn-awq-salient-weights]: **Activation-Aware Weight Quantization (AWQ)**: Salience is determined by activation magnitude, not weight magnitude--a distinction that matters because a small weight multiplied by a large activation produces a large output contribution. AWQ protects about 1 percent of salient weight channels through activation-aware scaling while quantizing most weights to low-bit formats, so a 7-billion-parameter FP16 weight set that would occupy about 14 GB can move toward an INT4-class weight footprint of about 3.5 GB before metadata, scales, and kernel-specific packing overheads [@lin2023awq]. \index{AWQ!bandwidth reduction}
##### Static vs. dynamic quantization {#sec-model-compression-static-vs-dynamic-quantization-4791}
@@ -5273,40 +5295,39 @@ The systems trade-off depends on where those exits run. On mobile processors and
```{.tikz}
\begin{tikzpicture}[font=\small\usefont{T1}{phv}{m}{n}]
\tikzset{%
helvetica/.style={align=flush center,font=\small\usefont{T1}{phv}{m}{n}},
Line/.style={line width=1.0pt,black!50,text=black},
Box/.style={inner xsep=2pt,
node distance=1.3,
node distance=1.7,
draw=GreenLine,
line width=0.75pt,
fill=GreenL,
text width=25mm,align=flush center,
minimum width=25mm, minimum height=9mm
minimum width=25mm, minimum height=8mm
},
}
\node[Box, ellipse,text width=14mm,minimum width=12mm,
minimum height=11mm, fill=RedL,draw=RedLine](B1){Input};
\node[Box, ellipse,text width=14mm,minimum width=12mm,inner ysep=-1pt,
minimum height=10mm, fill=RedL,draw=RedLine](B1){Input};
\node[Box,right=of B1](B2){Transformer 1};
\node[Box,right=of B2](B3){Transformer 2};
\node[Box, node distance=2.5,right=of B3](B4){Transformer $n$};
\node[font=\tiny\usefont{T1}{phv}{m}{n}](B0)at($(B3)!0.5!(B4)$){$\bullet$ $\bullet$ $\bullet$};
%
\def\di{0.55}
\def\di{0.45}
\node[Box,node distance=\di,below=of B2,fill=VioletL2,draw=VioletLine](C1){Classifier 1};
\node[Box,node distance=\di,below=of C1,fill=BlueL,draw=BlueLine](C2){Confidence/LTE};
\node[Box,node distance=\di,below=of C2,ellipse,text width=14mm,minimum width=12mm,
minimum height=11mm, fill=RedL,draw=RedLine](C3){Exit};
minimum height=10mm, fill=RedL,draw=RedLine](C3){Exit};
%
\node[Box,node distance=\di,below=of B3,fill=VioletL2,draw=VioletLine](2C1){Classifier 2};
\node[Box,node distance=\di,below=of 2C1,fill=BlueL,draw=BlueLine](2C2){Confidence/LTE};
\node[Box,node distance=\di,below=of 2C2,ellipse,text width=14mm,minimum width=12mm,
minimum height=11mm, fill=RedL,draw=RedLine](2C3){Exit};
minimum height=10mm, fill=RedL,draw=RedLine](2C3){Exit};
%
\node[Box,node distance=\di,below=of B4,fill=VioletL2,draw=VioletLine](3C1){Classifier $n$};
\node[Box,node distance=\di,below=of 3C1,fill=BlueL,draw=BlueLine](3C2){Confidence/LTE};
\node[Box,node distance=\di,below=of 3C2,ellipse,text width=14mm,minimum width=12mm,
minimum height=11mm, fill=RedL,draw=RedLine](3C3){Exit};
minimum height=10mm, fill=RedL,draw=RedLine](3C3){Exit};
%
\node[font=\tiny\usefont{T1}{phv}{m}{n}](2B0)at($(2C1)!0.5!(3C1)$){$\bullet$ $\bullet$ $\bullet$};
\node[font=\tiny\usefont{T1}{phv}{m}{n}](3B0)at($(2C2)!0.5!(3C2)$){$\bullet$ $\bullet$ $\bullet$};
@@ -5327,7 +5348,7 @@ Box/.style={inner xsep=2pt,
\draw[Line,-latex](3C1)--(3C2);
\draw[Line,-latex](3C2)--(3C3);
%
\draw[Line,-latex](C2)-|node[left=6pt,pos=0.76,rotate=90]{Continue}($(B2)!0.5!(B3)$);
\draw[Line,-latex](C2)-|node[left=6pt,pos=0.8,rotate=90]{Continue}($(B2)!0.5!(B3)$);
\draw[Line,-latex](2C2.east)-|node[right=9pt,pos=0.35,rotate=90]{Continue}($(B3)!0.68!(B0)$);
\end{tikzpicture}
```
@@ -5348,7 +5369,6 @@ As @fig-switch-transformer illustrates, the Switch Transformer replaces the trad
```{.tikz}
\begin{tikzpicture}[line join=round,font=\small\usefont{T1}{phv}{m}{n}]
\tikzset{%
helvetica/.style={align=flush center,font=\small\usefont{T1}{phv}{m}{n}},
Line/.style={line width=1.0pt,black!50,text=black},
@@ -5616,7 +5636,7 @@ yshift=0mm,fill=BackColor!70,fit=(FFN1)(2FFN4)(CI2)(R1),line width=0.75pt](GBB2)
\node[left=2pt of CI3,align=center]{Positional\\ embedding};
\node[left=2pt of CI4,align=center]{Positional\\ embedding};
%
\begin{scope}[local bounding box=X1,line width=0.5pt,shift={($(DCI3)+(-1.2,-0.4)$)}]
\begin{scope}[local bounding box=X1,line width=0.5pt,shift={($(DCI3)+(-1.2,-0.24)$)}]
\def\side{0.4}
\foreach \i/\col in {0/white,1/green!40,2/white,3/green!40,4/white,5/green!40}{
\draw[fill=\col,thick] (\i*\side,0) rectangle ++(\side,\side);
@@ -5624,7 +5644,7 @@ yshift=0mm,fill=BackColor!70,fit=(FFN1)(2FFN4)(CI2)(R1),line width=0.75pt](GBB2)
\end{scope}
\node[left=2pt of X1]{$x_1$};
\node[below=2pt of X1]{More};
\begin{scope}[local bounding box=X2,line width=0.5pt,shift={($(DCI4)+(-1.2,-0.4)$)}]
\begin{scope}[local bounding box=X2,line width=0.5pt,shift={($(DCI4)+(-1.2,-0.24)$)}]
\def\side{0.4}
\foreach \i/\col in {0/white,1/green!40,2/white,3/green!40,4/white,5/green!40}{
\draw[fill=\col,thick] (\i*\side,0) rectangle ++(\side,\side);
@@ -5637,7 +5657,7 @@ yshift=0mm,fill=BackColor!70,fit=(FFN1)(2FFN4)(CI2)(R1),line width=0.75pt](GBB2)
\draw[Line,-latex]($(GSA2)!0.5!(SA2)$)--++(0:3.8)|-(P2);
%%%%%%%%%%%%
%left diagram
\begin{scope}[local bounding box=LD,line width=0.5pt,shift={(-12,1.8)}]
\begin{scope}[local bounding box=LD,line width=0.5pt,shift={(-12.5,1.8)}]
\node[Box3,fill=RedL,draw=RedLine](2P1){Self-Attention};
\node[Box3,above=of 2P1,fill=BrownL,draw=BrownLine](2P2){Add + Normalize};
\node[Box3,above=of 2P2,fill=BackColor,draw=BackLine](2P3){Switching FFN Layer};
@@ -5722,7 +5742,7 @@ A sparse pattern earns hardware speedup only when it is regular enough for kerne
::: {#fig-block-sparse-gemm fig-env="figure" fig-pos="htb" fig-cap="**Block Sparse Representation**: NVIDIA's cuSPARSE library efficiently stores block sparse matrices by exploiting dense submatrix structures, enabling accelerated matrix operations while maintaining compatibility with dense matrix computations through block indexing. This approach reduces memory footprint and arithmetic complexity for sparse linear algebra, important for scaling machine learning models. Adapted from NVIDIA's cuSPARSE documentation [@nvidia_cusparse_block]." fig-alt="Grid of 3×3 matrix blocks with varying shades indicating dense submatrices. Adjacent index array shows non zero block positions. Gray blocks represent zeros, colored blocks represent dense submatrices stored separately."}
```{.tikz}
\scalebox{0.7}{\begin{tikzpicture}[line join=round,font=\usefont{T1}{phv}{m}{n}]
\scalebox{0.57}{\begin{tikzpicture}[line join=round,font=\usefont{T1}{phv}{m}{n}]
\tikzset{%
helvetica/.style={align=flush center,font=\small\usefont{T1}{phv}{m}{n}},
Line/.style={line width=1.0pt,black!50,text=black},
@@ -6560,7 +6580,7 @@ yshift=0mm,fill=none,fit=(cell-1-1DM3),line width=3.5pt](BB3){};
%%%%%%%%%%
%right part
%%%%%%%%%%%%
\begin{scope}[local bounding box=RIGHT,shift={(14.5,0)}]
\begin{scope}[local bounding box=RIGHT,shift={(15.15,0)}]
\node[draw,circle,line width=0.75pt,cross,minimum width=6mm](CI1){};
\node[draw=black, line width=1.2pt,fill=GreenL, minimum width=0.9*\cellsize,
minimum height=0.9*\cellheight,below=0.5 of CI1](AR) {};
@@ -6847,7 +6867,7 @@ Before choosing a sequence, separate the major levers by what they change. Pruni
| **Quantization** | Reduce Size/Latency | Low | Low (PTQ)/High (QAT) | High (INT8 support) | Edge/Mobile deployment |
| **Distillation** | Reduce Size | Low-Moderate | High (student training) | Low | Creating smaller, high-quality models |
: **Optimization Technique Trade-Offs**: Comparison of the three major optimization approaches across key performance dimensions, highlighting how each technique addresses different constraints and deployment scenarios. Pruning excels for computational reduction but requires sparse hardware support, quantization provides balanced size and speed improvements with wide hardware compatibility, while distillation produces high-quality compressed models at higher training cost. {#tbl-optimization-comparison tbl-colwidths="[13,13,16,16,20,22]"}
: **Optimization Technique Trade-Offs**: Comparison of the three major optimization approaches across key performance dimensions, highlighting how each technique addresses different constraints and deployment scenarios. Pruning excels for computational reduction but requires sparse hardware support, quantization provides balanced size and speed improvements with wide hardware compatibility, while distillation produces high-quality compressed models at higher training cost. {#tbl-optimization-comparison tbl-colwidths="[13,13,16,20,16,22]"}
### Mapping constraints to techniques {#sec-model-compression-mapping-constraints-techniques-ff2c}
@@ -7140,6 +7160,8 @@ Sequencing matters because these levers interact through the same weights and ac
A mobile BERT pipeline\index{BERT!mobile deployment pipeline} makes that sequencing dependency concrete.
This example illustrates why sequencing matters: pruning first concentrates important weights into smaller ranges, making subsequent quantization more effective. Applying quantization before pruning reduces numerical precision available for importance-based pruning decisions, degrading final accuracy. Effective combination requires understanding these dependencies and developing application sequences that maximize cumulative benefits.
::: {#exmp-model-compression-bert-base-mobile-deployment-pipeline .callout-example title="BERT-Base mobile deployment pipeline"}
**Scenario**: An illustrative BERT-Base mobile deployment pipeline that combines published compression patterns rather than reproducing one paper's exact experiment [@sanh2019distilbert; @jiao2020; @zafrir2019].
@@ -7150,8 +7172,6 @@ A mobile BERT pipeline\index{BERT!mobile deployment pipeline} makes that sequenc
:::
This example illustrates why sequencing matters: pruning first concentrates important weights into smaller ranges, making subsequent quantization more effective. Applying quantization before pruning reduces numerical precision available for importance-based pruning decisions, degrading final accuracy. Effective combination requires understanding these dependencies and developing application sequences that maximize cumulative benefits.
With dozens of techniques across three optimization dimensions, rigorous measurement is essential for validating that optimizations achieve their intended goals. A practitioner who prunes, quantizes, and fuses without profiling the actual impact on target hardware is optimizing blindly.
## Efficiency Measurement {#sec-model-compression-efficiency-measurement-2424}
@@ -7211,14 +7231,14 @@ A critical caveat applies when translating profiling metrics into optimization e
*Compression hits an end-to-end ceiling once non-model work dominates the pipeline.*
:::
Consider profiling a Vision Transformer (ViT) for edge deployment. Using PyTorch Profiler reveals three key findings: attention layers consume 65 percent of total FLOPs (highly amenable to structured pruning), layer normalization consumes 8 percent of latency despite only 2 percent of FLOPs (a memory-bound operation), and the final classification head consumes 1 percent of computation but 15 percent of parameter memory. This profile suggests a clear priority ordering: first, apply magnitude-based pruning to attention layers for high FLOP reduction; second, quantize the classification head to INT8 for large memory savings with minimal accuracy impact; third, fuse layer normalization operations to reduce the memory bandwidth bottleneck.
::: {#psp-model-compression-flops-speedup .callout-perspective title="FLOPs reduction is not proportional speedup"}
Reducing a model's FLOPs by `{python} AmdahlCompression.flops_reduction_pct_str` does not guarantee `{python} AmdahlCompression.flops_reduction_pct_str` latency reduction. Memory-bound operations (common in LLM inference and normalization layers) see minimal benefit from compute reduction because they are bottlenecked by data movement, not arithmetic. Critically, Amdahl's Law\index{Amdahl's Law!model compression} applies at the system level (@sec-appdx-machine-foundations-amdahls-law-gustafsons-law-b741 derives its strong-scaling form): if model inference accounts for only `{python} AmdahlCompression.model_fraction_pct_str` of end-to-end latency (with the remaining `{python} AmdahlCompression.non_model_pct_str` spent on data loading, preprocessing, and postprocessing), then even perfect model optimization yields at most `{python} AmdahlCompression.max_speedup_mult_str` overall speedup. In language model serving, CPU-bound tokenization and network round-trip time frequently dominate that non-model fraction; in computer vision pipelines, JPEG decoding and image augmentation (resize, crop, normalize) on the CPU often consume the remainder. Always profile on target hardware before estimating optimization benefits.
:::
Consider profiling a Vision Transformer (ViT) for edge deployment. Using PyTorch Profiler reveals three key findings: attention layers consume 65 percent of total FLOPs (highly amenable to structured pruning), layer normalization consumes 8 percent of latency despite only 2 percent of FLOPs (a memory-bound operation), and the final classification head consumes 1 percent of computation but 15 percent of parameter memory. This profile suggests a clear priority ordering: first, apply magnitude-based pruning to attention layers for high FLOP reduction; second, quantize the classification head to INT8 for large memory savings with minimal accuracy impact; third, fuse layer normalization operations to reduce the memory bandwidth bottleneck.
Beyond these baseline measurements, modern optimization requires understanding model sensitivity to different types of modifications. Not all parameters contribute equally to accuracy. Layer-wise sensitivity analysis\index{Sensitivity Analysis!layer-wise} reveals which network components are most important for maintaining accuracy, guiding decisions about where to apply aggressive pruning or quantization and where to use conservative approaches.
### Measuring optimization effectiveness {#sec-model-compression-measuring-optimization-effectiveness-a3b2}
@@ -7333,6 +7353,11 @@ class ResNet50Int8Metrics:
Quantizing ResNet-50 from FP32 to INT8 should be evaluated as a before-and-after system profile, not as an accuracy number alone. @Tbl-resnet50-int8-profile compares the baseline artifact against the quantized one across the resource dimensions that determine deployment feasibility.
```{=latex}
\begingroup
\renewcommand{\arraystretch}{1.3}
```
| **Metric** | **FP32 baseline** | **INT8 result** | **Deployment reading** |
|:----------------------|:----------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------|
| **Top-1 accuracy** | `{python} ResNet50Int8Metrics.fp32_top1_str` | `{python} ResNet50Int8Metrics.int8_top1_str` (`{python} ResNet50Int8Metrics.top1_drop_pp_str` drop) | Aggregate accuracy mostly holds, but subgroup checks still matter. |
@@ -7343,6 +7368,10 @@ Quantizing ResNet-50 from FP32 to INT8 should be evaluated as a before-and-after
: **ResNet-50 INT8 Deployment Profile**: Quantization changes accuracy, latency, size, energy, and calibration together. The table separates aggregate wins from validation risks so the optimization can be judged as a deployment artifact rather than a single metric. {#tbl-resnet50-int8-profile tbl-colwidths="[21,19,24,36]"}
```{=latex}
\endgroup
```
The aggregate numbers make INT8 look almost free, but the diagnostic measurements explain why validation still matters. Per-class accuracy degradation spans `{python} ResNet50Int8Metrics.per_class_drop_pp_range_str`, with the highest impact on fine-grained categories, and the latency gain is hardware-dependent: this profile shows `{python} ResNet50Int8Metrics.latency_speedup_mult_str` on GPU but only `{python} ResNet50Int8Metrics.cpu_speedup_mult_str` on CPU.
With these comprehensive baselines in place, the measurement framework must track optimization impact systematically. Rather than evaluating techniques in isolation, applying our three-dimensional framework requires understanding how different approaches interact when combined. Sequential application can lead to compounding benefits or unexpected interactions that diminish overall effectiveness. @Sec-benchmarking-compression-validation-efficiencyquality-frontier-e9c4 later expands this into a full efficiency-quality evaluation framework.