Fairness-Accuracy Tradeoffs

Throughout this module, we've encountered a recurring theme: improving fairness often comes at the cost of predictive accuracy, and vice versa. This is not a failure of technique or a problem to be solved—it's a fundamental property of fair machine learning that emerges from deep mathematical and philosophical considerations.

Understanding this tradeoff is essential for responsible ML practice. It transforms fairness from a checklist item ('did we add the fairness constraint?') into a thoughtful design choice ('what level of accuracy are we willing to sacrifice for what level of fairness, and who decides?').

Why Tradeoffs Are Unavoidable:

At its core, a machine learning model is an optimization engine. When we train on historical data to maximize accuracy, we find the model that best captures patterns in that data—including any discriminatory patterns. When we add fairness constraints, we're asking the optimizer to deviate from the accuracy-maximizing solution.

Mathematically, if we denote the unconstrained optimal classifier as $h^*$ and a fairness-constrained classifier as $h_F$:

$$L(h_F) \geq L(h^*)$$

with equality only when $h^*$ already satisfies the fairness constraint. This is simply the nature of constrained optimization—adding constraints cannot improve the objective.

Several landmark results demonstrate that certain combinations of fairness criteria are mathematically impossible to satisfy simultaneously. These aren't limitations of current algorithms—they're fundamental constraints on what any algorithm can achieve.

Impossibility Theorem 1: Chouldechova (2017)

For a binary classifier with different base rates across groups ($P(Y=1|A=0) eq P(Y=1|A=1)$), the following three conditions cannot all hold simultaneously:

1. Calibration: $P(Y=1 | \hat{Y}=1, A=a) = P(Y=1 | \hat{Y}=1)$ for all $a$
2. Equal False Positive Rates: $P(\hat{Y}=1 | Y=0, A=0) = P(\hat{Y}=1 | Y=0, A=1)$
3. Equal False Negative Rates: $P(\hat{Y}=0 | Y=1, A=0) = P(\hat{Y}=0 | Y=1, A=1)$

Except in degenerate cases (perfect prediction or equal base rates).

Impossibility Theorem 2: Kleinberg, Mullainathan & Raghavan (2016)

Building on similar intuitions, this work proves that except when base rates are equal or prediction is perfect:

• Calibration within groups
• Balance for the positive class (equal TPR)
• Balance for the negative class (equal FPR)

cannot all be satisfied. At least one must be violated.

Intuition Behind the Impossibility:

Consider two groups where 50% of Group A and 20% of Group B will re-offend. A calibrated algorithm that predicts 'high risk' for 50% of Group A and 20% of Group B:

• Has different selection rates (violates demographic parity)
• May have different error rates when the underlying distributions differ

Trying to equalize error rates requires mis-calibration—predicting higher risk than justified for one group or lower for another.

Implications of Impossibility:

1. No Universal Fairness: There is no single 'fair' algorithm. Fairness requires choosing which criteria matter most in context.

2. Normative Decisions Required: Selecting fairness criteria is an ethical and policy choice, not a technical one. Different stakeholders may legitimately disagree.

3. Perfect Fairness is Impossible: When base rates differ, some unfairness (by some measure) is mathematically inevitable. The goal is to minimize harm, not achieve perfection.

4. Context Matters: The 'right' fairness criterion depends on the application. Equal opportunity may matter most in hiring; calibration may matter most in medicine.

The Pareto frontier (or Pareto boundary) is a powerful tool for visualizing and analyzing fairness-accuracy tradeoffs. It represents the set of solutions where you cannot improve one objective without worsening another.

Formal Definition:

A solution $(accuracy, fairness)$ is Pareto optimal if there is no other achievable solution that:

• Has higher accuracy with equal or better fairness, OR
• Has better fairness with equal or higher accuracy

The Pareto frontier is the set of all Pareto optimal solutions.

Constructing the Frontier:

1. Define accuracy metric $A(h)$ (e.g., AUC, accuracy, F1)
2. Define fairness metric $F(h)$ (e.g., demographic parity gap, equalized odds gap)
3. For various values of fairness constraint $\epsilon$:
   • Train model maximizing $A(h)$ subject to $F(h) \leq \epsilon$
   • Record $(A(h), F(h))$
4. Plot the achievable points; the upper boundary is the Pareto frontier

Key Properties of the Pareto Frontier:

1. Monotonicity: The frontier is generally monotonic—more fairness costs accuracy (or at least doesn't improve it).

2. Context Dependence: Different datasets and models produce different frontiers. The 'cost' of fairness varies.

3. Beyond the Frontier: Points inside the frontier (dominated points) represent suboptimal choices—you could do better on both dimensions.

4. Multi-Objective View: With multiple fairness criteria, the frontier becomes a surface in higher dimensions.

Using the Frontier for Decision-Making:

The Pareto frontier makes tradeoffs explicit. Rather than asking 'is this model fair?' (binary), ask:

• 'What accuracy can we achieve at various fairness levels?'
• 'How much accuracy are we sacrificing for this level of fairness?'
• 'Is the marginal accuracy cost of additional fairness acceptable?'

A natural question arises: How much does fairness actually cost? This can be measured as the 'price of fairness' (PoF)—the accuracy loss incurred by imposing fairness constraints.

Formal Definition:

Let $h^$ be the unconstrained optimal classifier and $h^_F$ be the optimal classifier satisfying fairness constraint $F$:

$$\text{Price of Fairness} = L(h^_F) - L(h^)$$

or as a ratio: $$\text{PoF Ratio} = \frac{L(h^_F)}{L(h^)}$$

Empirical Findings on the Price of Fairness:

Research has found that the price of fairness varies significantly:

1. Often Moderate: Many studies find accuracy drops of 1-5% for significant fairness improvements. Fairness isn't always expensive.

2. Depends on Base Rate Gap: When group base rates are similar, fairness is cheap. When they differ dramatically, it's expensive.

3. Depends on Feature Correlation: When features are highly correlated with protected attributes, removing discrimination is costlier.

4. Model Complexity Matters: More flexible models (e.g., neural networks) can sometimes achieve both high accuracy and fairness, while simpler models face starker tradeoffs.

5. Marginal Cost Increases: The first fairness improvements are often cheap; approaching perfect fairness becomes increasingly expensive.

When is the Price Low?

1. Near-fair data: When historical data is already approximately fair, constraints cost little.

2. Redundant features: If protected attributes are encoded in multiple features, removing one pathway may not hurt predictions much.

3. Suboptimal baseline: If the unconstrained model isn't fully optimized, imposing fairness + better optimization may improve both.

4. Constraint slack: When the fairness constraint isn't binding (already satisfied), there's no cost.

When is the Price High?

1. Large base rate gaps: Forcing equal predictions when groups genuinely differ sacrifices accuracy.

2. Protected attribute is predictive: When the protected attribute directly predicts the outcome (e.g., age in medical contexts), hiding it loses information.

3. Limited features: With few features, each carries more predictive weight, making it costlier to ignore correlations.

4. Tight constraints: Demanding perfect fairness (ε=0) is more expensive than approximate fairness.

Given that tradeoffs are unavoidable, how should practitioners navigate them? Here are principled strategies.

Strategy 1: Stakeholder-Driven Constraint Selection

Different stakeholders have different fairness priorities:

• Affected communities: May prioritize equal treatment or equal outcomes
• Legal/compliance: May require specific non-discrimination criteria
• Business: May balance fairness with operational needs
• Regulators: May mandate specific metrics or thresholds

Approach: Engage stakeholders early to determine which fairness criteria matter most. Let this drive metric selection rather than choosing post-hoc.

Strategy 2: Multi-Objective Optimization

Rather than treating fairness as a hard constraint, optimize a weighted combination:

$$L_{total} = \alpha \cdot L_{accuracy} + (1-\alpha) \cdot L_{fairness}$$

Varying $\alpha$ traces out the Pareto frontier. This approach:

• Makes the tradeoff explicit
• Allows continuous rather than binary fairness levels
• Enables finding 'sweet spots' that balance both objectives

Strategy 3: Fairness as Constraint with Slack

Set fairness constraints with slack variables that are penalized but not hard:

$$\min L_{accuracy} + \lambda \cdot \max(0, \text{FairnessGap} - \epsilon)$$

This allows small violations when they dramatically improve accuracy, while still incentivizing fairness.

The framing of 'fairness vs. accuracy' may itself be problematic. Several perspectives suggest the tradeoff is more nuanced than it appears.

Perspective 1: Fairness IS Accuracy (for Subgroups)

Traditional accuracy averages over the entire population, potentially masking poor performance for minority groups. If we define accuracy as 'minimum subgroup accuracy,' then improving fairness (equalizing group performance) directly improves this alternative accuracy measure.

$$\text{Worst-Group Accuracy} = \min_a \text{Accuracy}(h; A=a)$$

Optimizing worst-group accuracy explicitly connects fairness and accuracy.

Perspective 2: Long-Term vs. Short-Term Accuracy

An unfair model might have higher short-term accuracy but:

1. Feedback loops: Biased predictions create biased future data, reducing long-term accuracy
2. Population shift: Alienated subgroups may leave the population, changing the distribution
3. Strategic behavior: Individuals may game unfair systems, undermining their validity

Considering temporal dynamics may reveal that fair models have better long-term accuracy.

Perspective 3: The Right Metric Might Not Show a Tradeoff

Accuracy metrics are choices. If we measure the right thing, there may be no tradeoff:

• Instead of predicting arrests (biased), predict actual criminal behavior (unbiased)
• Instead of predicting healthcare costs (reflects access), predict health needs

The tradeoff often reflects measuring proxies rather than true outcomes.

Perspective 4: Costs of Unfairness

The 'cost' of fairness is only half the equation. What's the cost of unfairness?

1. Legal Risk: Discrimination lawsuits, regulatory penalties
2. Reputational Damage: Public backlash, loss of trust
3. Market Exclusion: Underserved groups represent lost opportunity
4. Human Harm: Real impacts on real people's lives

A full accounting includes both the accuracy cost of fairness AND the business/ethical cost of unfairness. Often, the latter dwarfs the former.

Fairness tradeoffs cannot—and should not—be resolved by individual engineers. They require organizational processes that involve appropriate stakeholders and create accountability.

Key Principles:

1. Tradeoffs are Policy Decisions: Choosing the operating point on the Pareto frontier is a policy choice, not a technical one. It should involve leadership, legal, ethics, and affected communities.

2. Document and Justify: Every choice should be documented with explicit justification for why this point was chosen over alternatives.

3. Create Accountability: Someone (a role, not just a person) should be responsible for fairness outcomes and have authority to require changes.

4. Enable Review: Fairness decisions should be reviewable and reversible based on new information or changing values.

Example: Fairness Decision Framework

A structured process for choosing an operating point:

1. Define Stakeholders: Who is affected? Who has authority? Who has relevant expertise?

2. Map the Frontier: Compute achievable (accuracy, fairness) points for relevant fairness criteria.

3. Identify Constraints: Are there hard legal or policy constraints? What's the minimum acceptable accuracy?

4. Present Options: Show stakeholders 3-5 representative points on the frontier with concrete implications.

5. Deliberate: Allow discussion of values, priorities, and downstream impacts.

6. Decide and Document: Record the chosen point, rationale, dissenting views, and conditions for revisiting.

7. Monitor and Revisit: Track performance and revisit the decision periodically or when conditions change.

The study of fairness-accuracy tradeoffs is an active research area with many open questions.

Open Technical Questions:

1. Tighter Characterization: Can we better characterize when tradeoffs are severe vs. mild? What data properties predict the 'price of fairness'?

2. Beyond Binary: Most theory considers binary protected attributes and binary outcomes. How do results extend to multi-class, multi-group, and continuous settings?

3. Causal Approaches: Can causal modeling help distinguish 'legitimate' from 'illegitimate' correlations, reducing the apparent tradeoff?

4. Dynamic Settings: How do tradeoffs evolve in online learning settings with feedback loops and distribution shift?

Open Normative Questions:

1. Who Decides? What's the right process for determining fairness criteria and acceptable tradeoffs? How do we include affected communities meaningfully?

2. Intersectionality: How should we handle fairness for intersectional identities (e.g., Black women)? Optimizing for each attribute separately may not help intersections.

3. Individual vs. Group: When are group fairness criteria appropriate vs. individual fairness? How do we reconcile them?

4. Across Applications: Should we have domain-specific fairness standards (e.g., stricter for criminal justice than advertising)?