github-starred/cs249r_book
2
0
Fork 0
mirror of https://github.com/harvard-edge/cs249r_book.git synced 2026-03-11 17:49:25 -05:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/fix/ch15' into feature/book-volumes

Browse Source
This commit is contained in:
Vijay Janapa Reddi
2026-02-25 07:54:56 -05:00
parent ad6229a899 1f85111486
commit c990d0037e
1 changed files with 51 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

52
book/quarto/contents/vol1/responsible_engr/responsible_engr.qmd
View File

@@ -763,7 +763,57 @@ This pattern automates what manual auditing cannot achieve at scale: continuous
To understand *why* aggregate metrics hide these disparities, look closely at @fig-fairness-threshold. When a single threshold is applied to populations with different score distributions, the same decision boundary produces vastly different outcomes for each group [@barocas2016big]. The figure exposes a fundamental tension: any fixed threshold is simultaneously "correct" for the combined population while being systematically wrong for each subpopulation.
![**Threshold Effects on Subgroup Outcomes**. A single classification threshold (vertical lines) applied to two subgroups with different score distributions produces disparate outcomes. Circles represent positive outcomes (loan repayment), crosses represent negative outcomes (default). The 75% threshold approves most of Subgroup A but rejects most of Subgroup B, even when qualified individuals exist in both groups. The 81.25% threshold shows how threshold adjustment changes the fairness-accuracy tradeoff. This visualization explains why aggregate accuracy can mask severe subgroup disparities.](images/svg/fairness_threshold.svg){#fig-fairness-threshold fig-alt="Diagram showing two subgroups A and B with different score distributions. Vertical threshold lines at 75% and 81.25% show how the same threshold produces different approval rates for each group."}
::: {#fig-fairness-threshold fig-env="figure" fig-pos="htb" fig-cap="**Threshold Effects on Subgroup Outcomes**. A single classification threshold (vertical lines) applied to two subgroups with different score distributions produces disparate outcomes. Circles represent positive outcomes (loan repayment), crosses represent negative outcomes (default). The 75% threshold approves most of Subgroup A but rejects most of Subgroup B, even when qualified individuals exist in both groups. The 81.25% threshold shows how threshold adjustment changes the fairness-accuracy tradeoff. This visualization explains why aggregate accuracy can mask severe subgroup disparities." fig-alt="Diagram showing two subgroups A and B with different score distributions. Vertical threshold lines at 75% and 81.25% show how the same threshold produces different approval rates for each group."}
```{.tikz}
\begin{tikzpicture}[line join=round,font=\small\usefont{T1}{phv}{m}{n}]
\tikzset{%
LineA/.style={line width=2.5pt,black!50,text=black},
LineD/.style={line width=1.5pt,black!50,text=black,dashed,dash pattern=on 4pt off 3pt},
circR/.style={draw=red, fill=white,line width=2.5pt,circle, minimum size=5mm, inner sep=0pt},
circB/.style={draw=blue!70!black, fill=white,line width=2.5pt,circle, minimum size=5mm, inner sep=0pt},
}
\newcommand{\fplus}[1][black]{%
\tikz\draw[#1,scale=0.55,line width=3.5pt] (0,0) -- (1,0)(0.5,0.5) -- (0.5,-0.5);
}
\draw[LineA](0,0)coordinate(A1)node[left](SA){Subgroup A}--(12.5,0)coordinate(A2);
\draw[LineA](0,-2.5)coordinate(B1)node[left]{Subgroup B}--(12.5,-2.5)coordinate(B2);
\coordinate(D7G)at($(A1)!0.3!(A2)$);
\coordinate(D8G)at($(A1)!0.63!(A2)$);
\coordinate(D7D)at($(B1)!0.3!(B2)$);
\coordinate(D8D)at($(B1)!0.63!(B2)$);
\draw[LineD]($(D7G)+(0,1)$)node[above](SE){75\%}--($(D7D)+(0,-0.77)$);
\draw[LineD]($(D8G)+(0,1)$)node[above]{81.25\%}--($(D8D)+(0,-0.77)$)coordinate(DO);
\foreach \x in{0.25,0.35,0.42,0.49}{
\node[circB,yshift=4mm]at($(A1)!\x!(A2)$){};
}
\foreach \x in{0.68,0.75,0.82,0.89}{
\node[yshift=4mm]at($(A1)!\x!(A2)$){\fplus[blue!70!black]};
}
\foreach \x in{0.05,0.12,0.19,0.42}{
\node[circR,yshift=4mm]at($(B1)!\x!(B2)$){};
}
\foreach \x in{0.35,0.49,0.56,0.75}{
\node[yshift=4mm]at($(B1)!\x!(B2)$){\fplus[red]};
}
\node[draw=none,fit=(SA)(A2)(SE)(DO)](FI){};
\node[circB,below=-2pt of FI.210](CI1){};
\node[circR,right=5pt of CI1](CI2){};
\node[right=0pt of CI2]{Positive outcome};
\node[below=-7pt of FI.300](CI3){\fplus[blue!70!black]};
\node[right=-6pt of CI3](CI4){\fplus[red]};
\node[right=0pt of CI4]{Negative outcome};
\node[below=20pt of FI.south]{\footnotesize Color indicates subgroup; marker shape indicates outcome.};
\end{tikzpicture}
```
:::
Several mitigation approaches exist, each with distinct trade-offs. Threshold adjustment\index{Threshold Adjustment!fairness mitigation} lowers the approval threshold for Group B to equalize TPR but may increase false positives for that group. Reweighting[^fn-reweighting-bias]\index{Reweighting!bias mitigation} increases the weight of Group B samples during training to give the model stronger signal about this population but may reduce overall accuracy. Adversarial debiasing\index{Adversarial Debiasing!fairness constraints} trains with an adversary that prevents the model from learning group membership but adds training complexity.[^fn-adversarial-debiasing-cost] The choice among these approaches requires stakeholder input about which trade-offs are acceptable in the specific application context. How, then, should engineers present these trade-offs to stakeholders? The answer lies in making the trade-offs explicit and quantifiable.