about
Measuring Fairness in Ranked Outputs (arxiv.org)
1 point by rch on Oct 28, 2016 | hide | past | pdf | discuss on HN

In plain words: They created ways to score how fairly a ranked list treats protected groups, plus a fake-data generator that builds lists with a chosen amount of unfairness to test those scores. On real lists the scores exposed unfairness, and a first attempt at fixing rankings improved fairness without hurting accuracy.

Abstract

Ranking and scoring are ubiquitous. We consider the setting in which an institution, called a ranker, evaluates a set of individuals based on demographic, behavioral or other characteristics. The final output is a ranking that represents the relative quality of the individuals. While automatic and therefore seemingly objective, rankers can, and often do, discriminate against individuals and systematically disadvantage members of protected groups. This warrants a careful study of the fairness of a ranking scheme. In this paper we propose fairness measures for ranked outputs. We develop a data generation procedure that allows us to systematically control the degree of unfairness in the output, and study the behavior of our measures on these datasets. We then apply our proposed measures to several real datasets, and demonstrate cases of unfairness. Finally, we show preliminary results of incorporating our ranked fairness measures into an optimization framework, and show potential for improving fairness of ranked outputs while maintaining accuracy.

Ke Yang, Julia Stoyanovich
arXiv:1610.08559 · cs.DB · submitted Oct 26, 2016
abstract · pdf · html · 5 pages, 7 figures, FATML 2016

add comment on HN