about
ML Fairness Breaks Under Distribution Shift–Here's the Fix (arxiv.org)
1 point by WASDAai on Oct 1, 2025 | hide | past | pdf | 1 comment on HN

In plain words: Without retraining, it adjusts each group's set of plausible answers using weights that correct for changed data, plus a penalty against answers that flip when someone's group changes. It kept true-answer inclusion more even across groups than shift-aware and fairness alternatives, with similar speed.

Abstract · Calibrated Counterfactual Conformal Fairness ($C^3F$): Post-hoc, Shift-Aware Coverage Parity via Conformal Prediction and Counterfactual Regularization

We present Calibrated Counterfactual Conformal Fairness ($C^3F$), a post-hoc procedure that targets group-conditional coverage parity under covariate shift. $C^3F$ combines importance-weighted conformal calibration with a counterfactual regularizer based on path-specific effects in a structural causal model. The method estimates group-specific nonconformity quantiles using likelihood-ratio weights so that coverage degrades gracefully with the second moment of the weights. We derive finite-sample lower bounds on group-wise coverage and a bound on the equalized conditional coverage gap, and we show first-order control of a counterfactual coverage-parity surrogate via smooth threshold regularization. The approach is model-agnostic, label-efficient, and deployable without retraining. Empirical evaluations on standard classification benchmarks demonstrate improved group-conditional coverage and competitive efficiency relative to shift-aware and fairness-oriented conformal baselines. We discuss practical considerations, including partial availability of sensitive attributes and robustness to structural causal misspecification.

Faruk Alpay, Taylan Alpay
arXiv:2509.25295 · stat.ME · submitted Sep 29, 2025
abstract · pdf · html · 5 pages

add comment on HN

C3F achieves group-conditional coverage parity under distribution shift without model retraining. This matters because every deployed ML system faces covariate shift, yet current fairness methods assume static distributions. The method provides finite-sample lower bounds on group-wise coverage with degradation proportional to chi-squared divergence between distributions. Empirical results show it outperforms existing fairness-aware conformal methods while remaining computationally efficient.