Access the full text.
Sign up today, get DeepDyve free for 14 days.
Loading next page...
/lp/arxiv-cornell-university/operationalizing-individual-fairness-with-pairwise-fair-qUc4RC0F1t
References
References for this paper are not available at this time. We will be adding them shortly, thank you for your patience.
- ISSN
- 2150-8097
- eISSN
- ARCH-3347
- DOI
- 10.14778/3372716.3372723
- Publisher site
- See Article on Publisher Site
Abstract
Operationalizing Individual Fairness with Pairwise Fair Representations Preethi Lahoti Krishna P. Gummadi Gerhard Weikum Max Planck Institute for Max Planck Institute for Max Planck Institute for Informatics Software Systems Informatics Saarland Informatics Campus Saarland Informatics Campus Saarland Informatics Campus Saarbrücken, Germany Saarbrücken, Germany Saarbrücken, Germany plahoti@mpi-inf.mpg.de gummadi@mpi-sws.org weikum@mpi-inf.mpg.de ABSTRACT ing demographic parity [15], equality of opportunity [19], equalized odds [19], and envy-free group fairness [33]. These We revisit the notion of individual fairness proposed by operationalizations di er in the measures used to quantify Dwork et al. A central challenge in operationalizing their a group's \fair share of bene cial outcomes" as well as the approach is the diculty in eliciting a human speci cation mechanisms used to optimize for the fairness measures. of a similarity metric. In this paper, we propose an oper- While e ective at countering group-based discrimination ationalization of individual fairness that does not rely on a in decision outcomes, group fairness notions do not address human speci cation of a distance metric. Instead, we pro- unfairness in outcomes at the level of individual users. For pose novel approaches to elicit and leverage side-information instance, it is natural for individuals to compare their out- on equally deserving individuals to counter subordination comes with those of others with similar quali cations (inde- between social groups. We model this knowledge as a fair- pendently of their group membership) and perceive any dif- ness graph, and learn a uni ed Pairwise Fair Representation ferences in outcomes amongst individuals with similar stand- (PFR) of the data that captures both data-driven similar- ing as unfair. ity between individuals and the pairwise side-information in fairness graph. We elicit fairness judgments from a variety Individual Fairness: In their seminal work [13], Dwork et of sources, including human judgments for two real-world al. introduced a powerful notion of fairness called individ- datasets on recidivism prediction (COMPAS) and violent ual fairness, which states that \similar individuals should be neighborhood prediction (Crime & Communities). Our ex- treated similarly". In the original form of individual fairness periments show that the PFR model for operationalizing introduced in [13], the authors envisioned that a task-speci c individual fairness is practically viable. similarity metric would be provided by human experts which captures the similarity between individuals (e.g., \a student 1. INTRODUCTION who studies at University W and has a GPA X is similar to another student who studies at University Y and has GPA 1.1 Motivation Z"). The individual fairness notion stipulates that individ- uals who are deemed similar according to this task-speci c Machine learning based prediction and ranking models are similarity metric should receive similar outcomes. Opera- playing an increasing role in decision making scenarios that tionalizing this strong notion of fairness can help in avoiding a ect human lives. Examples include loan approval deci- unfairness at an individual level. sions in banking, candidate rankings in employment, wel- However, eliciting such a quantitative measure of similar- fare bene t determination in social services, and recidivism ity from humans has been the most challenging aspect of risk prediction in criminal justice. The societal impact of the individual fairness framework, and little progress has these algorithmic decisions has raised concerns about their been made on this open problem. Two noteworthy subse- fairness [3, 12], and recent research has started to investi- quent works on individual fairness are [37] and [26], wherein gate how to incorporate formalized notions of fairness into the authors operationalize a simpli ed notion of similarity machine prediction models (e.g., [13, 19, 22, 34]). metric. Concretely, they assume a distance metric (simi- Individual vs Group Fairness: The fairness notions ex- larity metric) such as a weighted Euclidean distance over a plored by the bulk of the works can be broadly categorized feature space of data atttributes, and aim to learn fair fea- as targeting either group fairness [29, 15] or individual fair- ture weights for this distance metric. This simpli cation of ness [13]. Group fairness notions attempt to ensure that the individual fairness notion largely limits the scope of the members of all protected groups in the population (e.g., original idea of [13]: \. . . a (near ground-truth) approxima- based on demographic attributes like gender or race) receive tion agreed upon by the society of the extent to which two their \fair share of bene cial outcomes" in a downstream individuals are deemed similar with respect to the task . . . ". task. To this end, one or more protected attributes and re- In this work we revisit the original notion of individual spective values are speci ed, and given special treatment fairness. There are two main challenges in its operationaliza- in machine learning models. Numerous operationalizations tion: First, it is very dicult, if not impossible for humans of group fairness have been proposed and evaluated includ- to come up with a precise quantitative similarity metric that can be used to measure \who is similar to whom". Second, This is a preprint of a full paper in the proceedings of the VLDB Endowment, Vol. 13, No. 4. even if we assume that humans are capable of giving a precise arXiv:1907.01439v2 [cs.LG] 2 Dec 2019 similarity metric, it is still challenging for experts to model for the application task. Naturally, no human judgments are subjective side-information such as \who should be treated elicited for test data (unseen data). So once the prediction similar to whom" as a quantitative similarity metric. model for the application at hand has been learned, only the regular data attributes of individuals are needed. Examples: The challenge is illustrated by two scenarios: [RQ2] Learning Pairwise Fair Representations. Consider the task of selecting researchers for academic Given a fairness graph G, the goal of an individually fair jobs. Due to the di erence in publication culture of algorithm is to minimize the inconsistency (di erences) in various communities, the citation counts of successful outcomes for pairs of individuals connected in graph G. researchers in programming language are known to be Thus, every edge in graph G represents a fairness constraint typically lower than that of successful machine learn- that the algorithm needs to satisfy. In Section 3, we pro- ing researchers. An expert recruiter might have the pose a model called PFR (for Pairwise Fair Representa- background information for fair selection that \an ML tions), which learns a new data representation with the aim researcher with high citations is similarly strong and of preserving the utility of the input feature space (i.e., re- thus equally deserving as a PL researcher with rela- taining as much information of the input as possible), while tively lower citations". It is all but easy to specify this incorporating the fairness constraints captured in the fair- background knowledge as a similarity metric. ness graph. Consider the task of selecting students for Graduate Speci cally, PFR aims to learn a latent data representa- School in the US. It is well known that SAT tests can tion that preserves the local neighborhoods in the input data be taken multiple times, and only the best score is space, while ensuring that individuals connected in the fair- reported for admissions. Further, each attempt to re- ness graph are mapped to nearby points in the learned rep- take the SAT test comes at a nancial cost. Due to resentation. Since local neighborhoods in the learned rep- complex interplay of historical subordination and so- resentation capture individual fairness, once a fair represen- cial circumstances, it is known that, on average, SAT tation is learned, any out-of-the-box downstream predictor scores for African-American students are lower than can be directly applied. PFR takes as input for white students [7]. Keeping anti-subordination in data records for individuals in the form of a feature ma- mind, a fairness expert might deem an African-American trix X for training a predictor, and student with a relatively lower SAT score to be sim- a (sparse) fairness graph G that captures pairwise simi- ilar to and equally deserving as a white student with larity for a subsample of individuals in the training data. a slightly higher score. Once again, it is not easy to The output of PFR is a mapping from the input feature model this information as a similarity metric. space to the new representation space that can be applied to data records of novel unseen individuals. Research Questions: We address the following research questions in this paper. 1.3 Contribution - [RQ1] How to elicit and model various kinds of back- The key contributions of this paper are: ground information on individual fairness? A practically viable operationalization of the individual - [RQ2] How to encode this background information, such fairness paradigm that overcomes the challenge of human that downstream tasks can make use of it for data-driven speci cation of a distance metric, by eliciting easier and predictions and decision making? more intuitive forms of human judgments. 1.2 Approach Novel methods for transforming such human judgments into pairwise constraints in a fairness graph G. [RQ1] From Distance Metric to Fairness Graph. A mathematical optimization model and representation Key Idea: It is dicult, if not impossible, for human ex- learning method, called PFR, that combines the input perts to judge \the extent to which two individuals are sim- data X and the fairness graph G into a uni ed represen- ilar", much less formulate a precise similarity metric. In tation by learning a latent model with graph embedding. this paper, we posit that it is much easier for experts to Demonstrating the e ectiveness of our approach at achiev- make pairwise judgments about who is equally deserving ing both individual and group fairness using comprehen- and should be treated similar to whom. sive experiments with synthetic as well as real-life data We propose to capture these pairwise judgments as a fair- on recidivism prediction (Compas) and violent neighbor- ness graph, G, with edges between pairs of individuals deemed hoods prediction (Crime and Communities). similar with respect to the given task. We view this as valu- able side information, but we consider it to be subjective and noisy. Aggregation over many users can mitigate this, but 2. RELATED WORK we cannot expect G to be perfectly fair. Further, for gener- Operationalizing Fairness Notions: Prior works on al- ality, we do not assume that these are always complete. In gorithmic fairness explore two broad families of fairness no- many applications, only partial and sometimes sparse fair- tions: group fairness and individual fairness. ness judgments would be available. In our experiments, we study the sensitivity to the amount of data in G in Subsec- Group Fairness: Two popular notions of group fairness tion 4.5. In Section 3.2 we address some of the practical are demographic parity, which requires equality of bene cial challenges that arise in eliciting pairwise judgments such as outcome prediction rates between di erent socially salient comparing individuals from diverse groups, and we present groups, [8, 21, 29], and equalized odds that aims to achieve various methods to construct fairness graphs. equality of prediction error rates between groups [19]. Ap- It is worth highlighting that we only need pairwise judg- proaches to achieve group fairness include de-biasing the in- ments for a small sample of individuals in the training data put data via data perturbation, re-sampling, modifying the value of protected attribute/class labels [30, 21, 29, 15] as space where D M . well as incorporating group fairness as an additional con- DN Z = [z ; z ; z ; z ] 2 R 1 2 3 N straint in the objective function of machine learning models [23, 8, 35]. Similar approaches to achieve group fairness S is a random variable representing the values that have been proposed for fair ranking [4, 14, 36], fair set selec- the protected-group attribute can take. We assume a tion and clustering [9, 32] Recently, several researchers have single attribute in this role; if there are multiple at- highlighted the inherent incompatibility between di erent tributes which require fair-share protection, we simply notions of group fairness and the inherent trade-o s when combine them into one. We allow more than two val- attempting to achieve them simultaneously [25, 10, 16, 11]. ues for this attribute, going beyond the usual binary Bridging Individual and Group Fairness: Approaches model (e.g., gender = male or female, race = white or to enforcing group fairness have mostly ignored individual others). X X denotes the subset of individuals in fairness and vice versa. In [37] and [26], authors opera- X who are members of group s 2 S. tionalize individual fairness by learning a restricted form of W is the adjacency matrix of a k-nearest-neighbor distance metric from the data. Some recent works use the graph over the input space X given by: objective of the learning algorithm itself to implicitly de ne the similarity metric [31, 5, 24]. For instance, when learn- 2 kx x k i j exp ; if x 2 N (x ) or x 2 N (x ) i p j j p i ing a classi er, these works would use the class labels in the t W = ij training data or predicted class labels to measure similar- 0 ; otherwise ity. However, fairness notions are meant to address societal where N (x ) denotes the set of p nearest neighbors p i inequities that are not captured in the training data (with of x in Euclidean space (excluding the protected at- potentially biased labels and missing features). In such sce- i tributes), and t is a scalar hyper-parameter. narios, the fairness objectives are in con ict with the learn- ing objectives. W is the adjacency matrix of the fairness graph G In this work, we assume that human experts with back- whose nodes are individuals and whose edges are con- ground knowledge of past societal unfairness and future soci- nections between individuals that are equally deserv- etal goals could provide coarse-grained judgments on whether ing and must be treated similarly. pairs of individuals deserve similar outcomes. Other works like [17] [20] make similar arguments. Further, we show that 3.2 From Distance Metric to Fairness Graph by appropriately constraining outcomes for pairs of individ- In this section we address the question of how to elicit uals belonging to di erent groups, we are able to achieve side-information on individual fairness and model it as a both individual and group fairness to a large degree. fairness graph G and its corresponding adjacency matrix as Learning Pairwise Fair Representations: In terms of W . The key idea of our approach is rooted in the following our technical machinery, the closest prior work is [37, 26] observations: that aim to learn new representations for individuals that Humans have a strong intuition about whether two in- \retain as much information in the input feature space as dividuals are similar or not. However, it is dicult for possible, while losing any information that can identify indi- humans to specify a quantitative similarity metric. viduals' protected group membership". Our approach aims In contrast, it is more natural to make other forms of to learn new representations for individuals that retain the judgments such as (i)\Is A similar to B with respect to input data to the best possible extent, while mapping equally the given task?", or (ii)\How suitable is A for the given deserving individuals as closely as possible. Like [37, 26] our task (e.g., on a Likert scale)". method can be used to nd representations for new individ- However, these kinds of judgments are dicult to elicit uals not seen in the training data. when the pairs of individuals belong to diverse, incom- Finally, the core optimization problem we formulate re- parable groups. In such cases, it is easier for humans lates to graph embedding and representation learning [18]. to compare individuals within the same group, as op- The aim of graph embedding approaches is to a learn a rep- posed to comparing individuals between groups. Pair- resentation for the nodes in the graph encoding the edges wise judgements can be bene cial even if they are avail- between nodes as well as the attributes of the nodes [27, 1]. able only sparsely, that is, for samples of pairs. Similarly, we wish to learn a representation encoding both Next, we present two models for constructing fairness graphs, the features of individuals as well as their interconnecting which overcome the outlined diculties via edges in the fairness graph. (i) eliciting (binary) pairwise judgments of individuals who should be treated similarly, or grouping individuals into 3. MODEL equivalence classes (see Subsection 3.2.1) and (ii) eliciting within-group rankings of individuals and con- 3.1 Notation necting individuals across groups who fall within the same quantiles of the per-group distributions (see Sub- X is an input data matrix of N data records and M section 3.2.2). numerical or categorical attributes. We use X to de- note both the matrix and the population of individuals 3.2.1 Fairness Graph for Comparable Individuals x : The most direct way to create a fairness graph is to elicit MN (binary) pairwise similarity judgments about a small sample X = [x ; x ; x ; x ] 2 R 1 2 3 N of individuals in the input data, and to create a graph W Z is a low-rank representation of X in a D-dimensional such that there is an edge between two individuals if they F are deemed similarly quali ed for a certain task (e.g., being a multipartite graph W whose edges are given by: invited for job interviews). k k 0 1 ; if x 2 X and x 2 X 0 ; s 6= s F i s j Another alternative is to elicit judgments that map indi- W = (2) ij 0 ; otherwise viduals into discrete equivalence classes. Given a number of such judgments for a sample of individuals in the input That is, there exists an edge between a pair of individuals dataset, we can construct a fairness graph W by creating fx ; x g 2 X if x and x have di erent group memberships i j i j an edge between two individuals if they belong to the same and their scores fy ; y g lie in the same quantile. For the i j equivalence class irrespective of their group membership. case of two groups (e.g., gender is male or female), the graph is a bipartite graph. De nition 1. (Equivalence Class Graph) Let [x ] denote the equivalence class of an element x 2 X . We construct This model of creating between-group quantile graphs is an undirected graph W associated to X , where the nodes general enough to consider any kind of per-group ranked of the graph are the elements of X , and two nodes x and x i j judgment. Therefore, this model is not necessarily limited are connected if and only if [x ] = [x ]. i j to legally protected groups (e.g., gender, race), it can be used for any socially salient groups that are incomparable for the The fairness graph built from such equivalence classes given task (e.g., machine learning vs. programming language identi es equally deserving individuals { a valuable asset researchers). Note again that the pairwise judgements may for learning a fair data representation. Note that the graph be sparse, if such information is obtained only for sampled may be sparse, if information on equivalence can be obtained representatives. merely for sampled representatives. 3.3 Learning Pairwise Fair Representations 3.2.2 Fairness Graph for Incomparable Individuals In this section we address the question: How to encode However, at times, our individuals are from diverse and the background information such that downstream tasks can incomparable groups. In such cases, it is dicult if not infea- make use of it for the decision making? sible to ask humans for pairwise judgments about individuals 3.3.1 Objective Function across groups. Even with the best intentions of being fair, In fair machine learning, such as fair classi cation models, human evaluators may be misguided by wide-spread bias. the objective usually is to maximize the classi er accuracy If we can elicit a ranked ordering of individuals per-group, (or some other quality metric) while satisfying constraints and pool them into quantiles (e.g., the top-10-percent), then on group fairness statistics such as parity. For learning fair one could assume that individuals from di erent groups who data representations that can be used in any downstream belong to the same quantile in their respective rankings, are application { classi ers or regression models with varying similar to each other. Arguments along these lines have been target variables unknown at learning time { the objective made also by [24] in their notion of meritocratic fairness. needs to be generalized accordingly. To this end, the PFR Speci cally, our idea is to rst obtain within-group rank- ings of individuals (e.g., rank men and women separately) model aims to combine the utility of the learned represen- tation and, at the same time, preserve the information from based on their suitability for the decision task at hand, and the pairwise fairness graph. Starting with matrix X of then construct a between-group fairness graph by linking all th N data records x : : : x and M numeric or categorial at- individuals ranked in the same k quantile across the di er- 1 N tributes, PFR computes a lower-dimensional latent matrix ent groups (e.g., link programming language researcher and Z of N records each with D < M values. machine learning researcher who are similarly ranked in their We model utility into the notion of preserving local neigh- own groups). The relative rankings of individuals within a borhoods of user records in the attribute space X in the group, whether they are obtained from human judgments or latent representation Z from secondary data sources, are less prone to be in uenced Re ecting the fairness graph in the learner's optimization by discriminatory (group-based) biases. for Z is a demanding and a priori open problem. Our so- Formally, given (X ; Y ) for all s 2 S, where Y is a ran- s s s lution PFR casts this issue into a graph embedding that is dom variable depicting the ranked position of individuals incorporated into the overall objective function. The follow- in X . We construct a between-group quantile graph using De nitions 2 and 3. ing discusses the technical details of PFR 's optimization. Preserving the input data: For each data record x in De nition 2. (k-th quantile) Given a random variable Y , the input space, we consider the set N (x ) of its p nearest p i the k-th quantile Q is that value of y in the range of Y , neighbors with regard to the distance de ned by the kernel denoted y , for which the probability of having a value less X function given by W . For all points x within N (x ), we j p i ij than or equal to y is k. want the corresponding latent representations z to be close to the representation z , in terms of their L2-norm distance. This is formalized by the Loss in W , denoted by Loss . Q(k) = fy : Pr(Y y) = kg where 0 < k < 1 (1) 2 X Loss = kz z k W (3) X i j ij For the non-continuous behavior of discrete variables, we i;j=1 would add appropriate ceil functions to the de nition, but Note that this objective requires only local neighborhoods in we skip this technicality. X to be preserved in the transformed space. We disregard De nition 3. (Between-group quantile graph) Let X X data points outside of p-neighborhoods. This relaxation in- denote the subset of individuals who belong to group s 2 S creases the feasible solution space for the dimensionality re- and whose scores lie in the k-th quantile. We can construct duction. Learning a fair graph embedding: Given a fairness We aim to learn an M D matrix V such that for each graph W , the goal for Z is to preserve neighborhood prop- input vector x 2 X , we have the low-dimensional represen- F T erties in W . In contrast to Loss , however, we do not need tation z = V x , where z 2 Z is the mapping of the data X i i i any distance metric here, but can directly leverage the fair- point x on to the learned basis V . The objective function F T ness graph. If two data points x ; x are connected in W , is subjected to the constraint V V = I to eliminate trivial i j we aim to map them to representations z and z close to solutions. i j each other. This is formalized by the Loss in W , denoted Applying Lagrangian multipliers, we can formulate the trace by Loss . optimization problem in Equation 6 as an eigenvector prob- lem 2 F X F T Loss = kz z k W (4) F i j ij X ((1 )L + L )X v = v (7) i i i;j=1 It follows that the columns of optimal V are the eigenvectors Intuitively, for data points connected in W , we add a penalty corresponding to D smallest eigenvalues denoted by V = when their representations are far apart in Z . [v v v v ], and is a regularization hyper-parameter. 1 2 3 D Finally, the d-dimensional representation of input X is given Combined objective: Based on the above considerations, by Z = V X . a fair representation Z is computed by minimizing the com- bined objectives of Equations 3 and 4. The parameter Implementation: The above standard eigenvalue problem X F weighs the importance tradeo between W and W . As for symmetric matrices can be solved in O(N ) using itera- increases in uence of the fairness graph W increases. tive algorithms. In our implementation we use the standard An additional orthonormality constraint on Z is imposed to eigenvalue solver implementation from scipy.linalg.lapack avoid trivial results. The trivial result being that all the python library [2]. datapoints are mapped to same point. 3.3.4 Inference N N X X 2 X 2 F Given an input vector x for a previously unseen individ- Minimize (1 ) kz z k W + kz z k W i j ij i j ij ual, the PFR method computes its fair representation as i;j=1 i;j=1 z = V x where z is the projection of the datapoint x on i i i i subject to Z Z = I (5) the learned basis V . It is important to note that the fair- ness graph W is only required during the training phase 3.3.2 Equivalence to Trace Optimization Problem to learn the basis V . Once the M D matrix V is learned, Next, we show that the optimization problem in Equation 5 we do not need any fairness labels for newly seen data. can be transformed and solved as an equivalent eigenvector problem. To do so, we assume that the learnt representation 3.3.5 Kernelized Variants of PFR Z is a linear transformation of X given by Z = V X . In this paper, we restrict ourselves to assume that the We start by showing that minimizing kz z k W is i j ij representation Z is a linear transformation of X given by T T equivalent to minimizing the trace Tr(V XLX V ). Here Z = V X . However, PFR can be generalized to a non-linear X F we use W to denote W or W , as the following mathe- setting by replacing X with a non-linear mapping (X ) and matical derivation holds for both of them analogously: then performing PFR on the outputs of (potentially in a higher-dimensional space). For this purpose, assume that Z = V (X ) and V = Loss = kz z k W i j ij i;j=1 (x ) with a Mercer kernel matrix K where K = i i i;j N i=1 k(x ; x ) = (x ) (x ). We can show that the trace opti- = Tr((z z ) (z z ))W i i i j i j i j ij mization problem in Equation 7 can be generalized to this i;j=1 non-linear kernel setting, and it can be conveniently solved N N X X T T by working with Mercer kernels without having to compute = 2 Tr( z z D z z W ) i ii j ij i i (X ). We arrive at the following generalized optimization i;j=1 i;j=1 problem. T T = 2 Tr(V XLX V ) X F K ((1 )L + L )K = (8) i i where Tr(:) denotes the trace of a matrix, D is a diagonal Analogously to the solution of Equation 7, the solution matrix whose entries are column sums of W , and L = D W to the kernel PFR is given by A = [ ] where 1 2 3 D is the graph Laplacian constructed from matrix W . Analo- X X are the D smallest eigenvectors. Finally, the learned gous to L, we use L to denote graph laplacian of W , and 1 D T T F F representation of X is given by Z = V (X ) = A K . L to denote graph laplacian of W . In this paper we present results only for linear PFR, leav- 3.3.3 Optimization Problem ing the investigation of kernel PFR for future work. Considering the results of Subsection 3.3.2, we can trans- form the above combined objective in Equation 5 into a trace 4. EXPERIMENTS optimization problem as follows: This section reports on experiments with synthetic and real-life datasets. We compare a variety of fairness-enhancing T X F T methods on a binary classi cation task as a downstream ap- Minimize J (V ) = TrfV X ((1 )L + L )X Vg plication. We address the following key questions in our subject to V V = I (6) main results in Subsection 4.2, 4.3.2 and 4.3.3: - [Q1] What do the learned representations look like? numerical features and 46 one-hot encoded features (for cat- egorical attributes), and (iii) compas dataset for recidivism - [Q2] What is the e ect on individual fairness? prediction with 9 numerical and 420 one-hot encoded fea- tures. In order to check the \true" dimensionality of the - [Q3] What is the in uence on the trade-o between datasets we computed the smallest rank k for SVD that fairness and utility? achieves a relative error of at most 0.01 for the Frobenius norm di erence between the SVD reconstruction and the - [Q4] What is the in uence on group fairness? original data. For the three datasets, these dimensionalities In addition, to understand the robustness of our model to are 156, 69, and 117 respectively. Table 1 summarizes the the main hyper-parameter , as well as the sensitivity of statistics for each dataset, including base-rate (fraction of the model to the number of labels in the fairness graph, we samples belonging to the positive class, for both the pro- report additional results in Subsection 4.4, and 4.5. tected group and its complement). In all experiments, the representation learning methods are followed by an out-of- 4.1 Experimental Setup the-box logistic regression classi er trained on the corre- Baselines: We compare the performance of the following sponding representations. methods Table 1: Experimental settings and dataset statistics Original representation: a naive representation of the in- put dataset wherein the protected attributes are masked. Dataset No of. No. of True Base-rate Base-rate Protected iFair [26]: an unsupervised representation learning method, records features Rank (s = 0) (s = 1) attribute which optimizes for two objectives: (i) individual fairness X Synthetic 1000 203 156 0.51 0.48 Race in W , and (ii) obfuscating protected attributes. Crime 1993 142 69 0.35 0.86 Race LFR [37]: a supervised representation learning method, Compas 8803 429 117 0.41 0.55 Race which optimizes for three objectives: (i) accuracy (ii) individual fairness in W and (iii) demographic parity. Hardt [19]: a post-processing method that aims to mini- Evaluation Measures: mize the di erence in error rates between groups by opti- Utility is measured as AUC (area under the ROC curve). mizing for the group-fairness measure EqOdd (Equality Individual Fairness is measured as the consistency of of Odds). outcomes between individuals who are similar to each PFR: Our unsupervised representation learning method other. We report consistency values as per both the sim- X F that optimizes for two objectives (i) individual fairness ilarity graphs, W and W . F X as per W and (ii) individual fairness as per W . PP jy ^ y ^ j W i j ij Augmenting Baselines: For fair comparison we compare i j PP Consistency = 1 8 i 6= j PFR with augmented versions of all methods (named with ij sux +). In the augmented version, we give each method i j an advantage by enhancing it with the information in the Group Fairness fairness graph W . Since none of the methods can be nat- urally extended to incorporate the fairness graph as it is, Disparate Mistreatment (aka. Equality of Odds): A binary classi er avoids disparate mistreatment if we make our best attempt at modeling the fairness labels that are used to construct W as additional numerical fea- the group-wise error rates are the same across all groups. In our experiments, we report per-group false tures in the training data. Since we only have judgments positive rate (FPR) and false negative rate (FNR). for a sample of training data, we treat the rest as missing values and set them to -1. Note that this enhancement is Disparate Impact (aka. Demographic Parity): only for training data as fairness labels are not available for A binary classi er avoids disparate impact if the rate unseen test data. This is in line with how PFR uses the of positive predictions is the same across all groups pairwise comparisons: its representation is learned from the s 2 S: training data, but at test time, only data attributes X are ^ ^ P (Y = 1js = 0) = P (Y = 1js = 1) (9) available. Concrete details for each of the datasets follow in their respective subsections. In our experiments, we report per-group rate of pos- Hyper-parameter Tuning: We use the same experimen- itive predictions. tal setup and hyper-parameter tuning techniques for all meth- ods. Each dataset is split into separate training and test 4.2 Experiments on Synthetic Data sets. On the training set, we perform 5-fold cross-validation We simulate the US graduate admissions scenario of Sec- (i.e., splitting into 4 folds for training and 1 for validation) tion 1.1 where our task is it to predict the ability of a can- to nd the best hyper-parameters for each model via grid didate to complete graduate school (binary classi cation). search. Once hyper-parameters are tuned, we use a in- To this end, we imagine that the features in a college ad- dependent test set to measure performance. All reported mission task can be grouped into two categories. First set results are averages over 10 runs on independent test sets. of features which are related to their academic performance Datasets and Task: We compare all methods on down- such as overall GPA, grades in each of the high schools sub- stream classi cation using three datasets: (i) a synthetic jects like Mathematics, Science, Languages, etc. Second set dataset for US university admission with 203 numerical fea- of features are related to their supplementary performance tures, and two real-world datasets: (ii) crime and commu- which constitute their overall application package such as nities dataset for violent neighbourhood predictions with 96 SAT scores, admission essay, extracurricular activities, etc. Original Representation iFair+ [Lahoti et al.] LFR+ [Zemel et al.] PFR (Proposed Approach) 1.00 1.00 0.8 2 0.8 0.75 0.75 0.7 0.75 0.75 1 0.6 0.6 1 0 0.50 0.5 0.50 0.50 0.50 0.4 0.4 0.25 0.3 0.25 0.2 0.2 0.25 0.25 3 2 0.1 0.00 3 2 1 0 1 2 3 3 0.00 0.0 0.0 Academic Performance 2.0 1.5 1.0 0.5 0.0 0.5 1.0 0.5 0.0 0.5 1.0 0.2 0.3 0.4 0.5 0.6 0.7 0.8 (d) PFR Representation (a) Original+ Representation (b) iFair + Representation (c) LFR+ Representation Figure 1: Comparison of (a)Original+ representation (b)iFair+ (c)LFR+ and (d)PFR representations on a synthetic dataset. Colors depict membership to protected group (S): orange (non-protected) and green (protected). Markers denote true class labels: Y = 1 (marker +) and Y = 0 (marker o). Contour plots visualize decision boundary of a classi er trained on the representations. Blue color corresponds to predicted positive classi cation, red to predicted negative class. F X AUC Consistency (W ) Consistency (W ) dataset with all 203 features. Experiments on the low- 1.0 1.0 1.0 dimensional dataset are performed in order to be able to Method 0.9 0.9 0.9 Original + visually compare the original and learned representations. 0.8 0.8 0.8 iFair + Dataset statistics are shown in Table 1. LFR + 0.7 0.7 0.7 Ground Truth Labels: Despite average score on supple- PFR 0.6 0.6 0.6 mentary performance features for group X being higher s=0 0.5 0.5 0.5 than for the protected group X , we assume that the abil- s=1 ity to complete graduate school is the same for both groups; Figure 2: Results for Synthetic low dimension dataset: that is, members of X and X are equally deserving if s=0 s=1 Comparison of utility vs individual fairness trade-o across we adjust their supplementary performance scores. To im- methods. Higher values are better. plement this scenario, we set the true class label for group X to positive (1) if academic + supplementary score 0 s=1 We assume that the scores for the second set of features and for group X as positive (1) if academic + supplemen- s=0 can be in ated for individuals who have higher privilege in tary score 1. Figure 1a visualizes the generated dataset. the society, for instance by re-taking SAT exam, and receiv- The colors depict the membership to groups (S): S = 0 (or- ing professional coaching. Suppose we live in a society where ange) and S = 1 (green). The markers denote true class our population consists of two groups s = 0 or 1, and the labels Y = 1 (marker +) and Y = 0 (marker o). group membership has a high correlation with individual's Fairness Graph W : In this experiment we simulate the privilege. This would result in a scenario where the two scenario for eliciting human input on fairness, wherein we groups have di erent feature distributions. Further, if we have access to a fairness oracle who can make the judgments assume that the in ation in the scores does not increase the of the form \Is A similar to B?" described in Subsection 3.2.1. ability of the candidate to complete college, the relevance To this end, we randomly sample N log N := 5538 pairs functions for the two groups would also be di erent. (out of the possible N := 600 600). We then constructed Creating Synthetic Datasets: We simulate this scenario ours fairness graphs W by querying a fairness oracle for by generating synthetic data for two population groups X Yes/No answers to similarity pairs. If the two points are and X . Our dataset consists of three main features: group, similar we add an edge between the two nodes. academic performance, and supplementary performance. The Fairness oracle for this task is a machine learning model correlation between academic performance and supplemen- consisting of two separate logistic regression models, one tary performance is set to 0.3. We have additional 100 nu- for each group, X and X respectively. Given a pair S=0 S=1 merical features with high correlation to academic perfor- of points, if their prediction probabilities fall in the same mance, and 100 numerical features with high correlation to quantile, they are deemed similar by the fairness oracle. supplementary performance. We set the value of correlation Augmenting Baselines: We cast each row of the matrix between related features by drawing uniformly from [0.75, W (of the fairness graph) into n additional binary features 1.0]. We use the correlation between features to construct for the respective individual. That is, for every user record, the covariance matrix for a multivariate Gaussian distribu- n additional 0/1 features indicate pairwise equivalence. All tion of dimensionality 203. To re ect the point that one baselines have access to this information via the augmented groups has in ated scores for the features related to supple- input matrix X . mentary performance, we set the mean for these features for the non-protected group one standard deviation higher than 4.2.1 Results on Synthetic Low Dimension Dataset the mean for the protected group. In total we generate 600 samples for training, and 400 [Q1] What do the learned representations look like? samples as a withheld test set. We run our experiments on In this subsection we inspect the original representations two versions of the synthetic dataset: (i) a low-dimensional and contrast them with learned representations via iFair+ dataset, which is a subset of the high-dimensional data con- [26], LFR+ [37], and our proposed model PFR. Figure 1 vi- sisting of only three features: Group, Academic Performance sualizes the original dataset and the learned representations and Supplementary performance, and (ii) a high-dimensional for each of the models with the number of latent dimen- Supplementary Perf. P(Y = 1) P(Y = 1) P(Y = 1) P(Y = 1) sions set to d = 2 during the learning. The contour plots Disparate Mistreatment (Figure 3a): We observe that in (b), (c) and (d) denote the decision boundaries of logistic Original+ model has high di erence in error rates (aka. regression classi ers trained on the respective learned repre- Equality of Odds). iFair+ and LFR+ balance the er- sentations. Blue color corresponds to positive classi cation, ror rates across groups fairly well, but still have fairly red to negative; the more intensive the color, the higher or high error rates, indicating their loss on utility. PFR lower the score of the classi er. We observe several interest- and Hardt have well balanced error rates and generally ing points: lower error. For Hardt, this is the expected e ect, as it is optimized for the very goal of Equality of Odds. PFR First, in the original data, the two groups are separated achieves the best balance and lowest error rates, which from each other: green and orange datapoints are rela- is remarkable as its objective function does not directly tively far apart. Further, the deserving candidates of one consider group fairness. Again, the e ect is explained group are relatively far away from the deserving candi- by PFR succeeding in mapping equally deserving indi- dates of the other group. That is, \green plus" are far viduals from both groups to close proximity in its latent from \orange plus", illustrating the inherent unfairness space. in the original data. Disparate Impact (Figure 3b): The Original+ approach In contrast, for all three representation learning tech- exhibits a substantial di erence in the per-group posi- niques { iFair+, LFR+ and PFR { the green and orange tive predictions rates of the two groups. In contrast, data points are well-mixed. This shows that these repre- iFair+, LFR+, and PFR representation have the orange sentations are able to make protected and non-protected and green data points well-mixed, and this way achieve group members indistinguishable from each other { a key nearly equal rates of positive predictions for both groups. property towards fairness. Likewise Hardt+ has the same desired e ect. The major di erence between the learned representations is that PFR succeeds in mapping the deserving candi- dates of one group close to the deserving candidates of Protected Group (S) the other group (i.e., \green plus" are close to \orange 0 1 plus"). Neither iFair+ nor LFR+ can achieve this de- Original + iFair + LFR + PFR Hardt + sired e ect to the same extent. 0.6 0.6 0.6 0.6 0.6 [Q2] E ect on Individual Fairness: Figure 2 shows the 0.4 0.4 0.4 0.4 0.4 best achievable trade-o between utility and the two notions 0.2 0.2 0.2 0.2 0.2 of individual fairness. 0.0 0.0 0.0 0.0 0.0 FNR FPR FNR FPR FNR FPR FNR FPR FNR FPR Individual fairness regarding W : We observe that PFR signi cantly outperforms all competitors in terms of con- (a) Di erence in error rates (FPR and FNR) sistency (W ). This follows from the observation that, unlike Original+, iFair+ and LFR+ representations, PFR Protected Group (S) maps similarly deserving individuals close to each other 0 1 in its latent space. Original + iFair + LFR + PFR Hardt + Individual fairness regarding W : We observe that PFR's 0.6 0.6 0.6 0.6 0.6 performance for consistency (W ) is as good as other 0.4 0.4 0.4 0.4 0.4 approaches, however PFR manages to achieve high per- 0.2 0.2 0.2 0.2 0.2 formance for signi cantly better performance on AUC 0.0 0.0 0.0 0.0 0.0 and consistency (W ). P(Y = 1) P(Y = 1) P(Y = 1) P(Y = 1) P(Y = 1) [Q3] Trade-o between Utility and Fairness: The AUC bars in Figure 2 show the results on classi er utility (b) Di erence in rates of positive prediction for the di erent methods under comparison. Figure 3: Results for Synthetic low dimension dataset: Dif- Utility (AUC): PFR achieves by far the best AUC, even ference in (a) error rates and (b) rate of positive predictions outperforming the original representation. While this between protected and non-protected groups may surprise on rst glance, it is indeed an expected outcome. The fairness edges in W help PFR overcome the challenge of di erent groups having di erent feature 4.2.2 Results on Synthetic High Dimension Dataset distributions (observe Figure 1a). In contrast, PFR is able to learn a uni ed representation that maps deserving The results for the high-dimensional synthetic data are candidates of one group close to deserving candidates largely consistent with the results for the low-dimensional of the other group (observe Figure 1d), which helps in case of Subsection 4.2.1. Therefore, we discuss them only improving AUC. brie y. Figure 4 shows results for AUC, consistency (W ), and consistency (W ). Figure 5 shows results on group fair- [Q4] In uence on Group Fairness: In addition to Orig- ness measures. inal+, iFair+, LFR+ and PFR, we include the Hardt model in the comparison here, as it is widely viewed as the state- Utility vs. Individual fairness regarding W : On rst of-the-art method for group fairness. glance, LFR+ seems to perform best on consistency with Figure 3a shows the per-group error rates, and Figure 3b regard to W . However, this is trivially achieved by giving shows the per-group positive prediction rates. The smaller the same prediction to almost all datapoints: the classi er the di erence in the values of the two groups, the higher the using the learned LFR+ representation accepts virtually all group fairness. We make the following interesting observa- individuals, hence its very poor AUC of around 0.55. In tions: essence, LFR+ fails to learn how to cope with the utility- fairness trade-o . Therefore, we consider this method as for a binary classi cation task. We consider the commu- degenerated (for this dataset) and dismiss it as a real base- nities with majority population white as non-protected line. group and the rest as protected group. Among the other methods, PFR signi cantly outperforms Compas data collected by ProPublica [3] contains crim- all competitors by achieving the best performance on con- inal records comprising o enders' criminal histories and sistency (W ), similar performance as other approaches on demographic features (gender, race, age etc.). We use the consistency (W ), but for a signi cantly better performance information on whether the o ender was re-arrested as on AUC, as shown in Figure 4. the target variable for binary classi cation. As protected attribute s 2 f0; 1g we use race: African-American (1) Group Fairness: Once again, PFR clearly outperforms vs. others (0). all other methods on group fairness. It achieves near-equal error rates across groups, and near-equal rates of positive predictions as shown in Figures 5a and 5b. Again, PFR's 4.3.1 Constructing the Fairness Graph W performance on group fairness is as good as that of Hardt Crime & Communities: We need to elicit pairwise judg- which is solely designed for equalizing error rates by post- ments of similarity that model whether two neighborhoods processing the classi er's outcomes. LFR+ seems to achieve are similar in terms of crime and safety. To this end, we col- good results as well, but this is again due to accepting vir- lected human reviews on crime and safety for neighborhoods tually all individuals (see above). in the US from http://niche.com. The judgments are given F X in the form of 1-star to 5-star ratings by current and past AUC Consistency (W ) Consistency (W ) 1.0 1.0 1.0 residents of these neighborhoods. We aggregate the judg- Method 0.9 0.9 0.9 ments and compute mean ratings for all neighborhoods. We Original + 0.8 0.8 0.8 iFair + were able to collect reviews for about 1500 (out of 2000) LFR + F 0.7 0.7 0.7 communities. W is then constructed by the technique of PFR 0.6 0.6 0.6 Subsection 3.2.1. 0.5 0.5 0.5 Although this kind of human input is subjective, the ag- gregation over many reviews lifts it to a level of inter-subjective side-information re ecting social consensus by rst-hand ex- Figure 4: Results for Synthetic high dimension dataset: Comparison of utility vs individual fairness trade-o across perience of people. Nevertheless, the fairness graph may be biased in favor of the African-American neighborhoods, methods. Higher values are better. since residents tend to have positive perception of their neigh- borhood's safety. Protected Group (S) 0 1 Compas: We need to elicit pairwise judgments of sim- ilarity that model whether two individuals are similar in Original + iFair + LFR + PFR Hardt + 1.0 1.0 1.0 1.0 1.0 terms of deserving to be granted parole and not becoming 0.8 0.8 0.8 0.8 0.8 0.6 0.6 0.6 0.6 0.6 re-arrested later. However, it is virtually impossible for a 0.4 0.4 0.4 0.4 0.4 human judge to fairly compare people from the groups of 0.2 0.2 0.2 0.2 0.2 0.0 0.0 0.0 0.0 0.0 African-Americans vs. Others, without imparting the his- FNRFPR FNRFPR FNRFPR FNRFPR FNRFPR toric bias. So this is a case, where we need to elicit pairwise judgments between diverse and incomparable groups. (a) Di erence in error rates (FPR and FNR) We posit that it is fair, though, to elicit within-group rank- ings of risk assessment for each of the two groups, to cre- Protected Group (S) ate edges between individuals who belong to the same risk 0 1 quantile of their respective group. To this end, we use North- Original + iFair + LFR + PFR Hardt + pointe's Compas decile scores [6] as background information 1.0 1.0 1.0 1.0 1.0 0.8 0.8 0.8 0.8 0.8 about within-in group ranking. These decile scores are com- 0.6 0.6 0.6 0.6 0.6 puted by an undisclosed commercial algorithm which takes 0.4 0.4 0.4 0.4 0.4 0.2 0.2 0.2 0.2 0.2 as input ocial criminal history and interview/questionnaire 0.0 0.0 0.0 0.0 0.0 answers to a variety of behavioral, social and economic ques- P(Y = 1) P(Y = 1) P(Y = 1) P(Y = 1) P(Y = 1) tions (e.g., substance abuse, school history, family back- ground etc.). The decile scores assigned by this algorithm (b) Di erence in rates of positive prediction are within-group scores and are not meant to be compared Figure 5: Results for Synthetic high dimension dataset: across groups. Di erence in (a) error rates between protected and non- We sort these decile scores for each group seprately to protected groups and (b) rate of positive predictions. simulate per-group ranking fairness judgments. We then use these per-group rankings as the fairness judgment to construct the fairness graphs for incomparable individuals 4.3 Experiments on Real-World Datasets as discussed in Subsection 3.2.2. Speci cally, we compute k We evaluate the performance of PFR on the following two quantiles over the ranking as per De nition 2 and then, con- real world datasets struct W as described in De nition 3. Note that this fair- Crime & Communities [28] is a dataset consisting of ness graph has an implicit anti-subordination assumption. socio-economic (e.g., income), demographic (e.g., race), That is, it assumes that individuals in k-th risk quantile of and law/policing data (e.g., patrolling) records for neigh- one group are similar to the individuals in k-th quantile of borhoods in the US. We set isViolent as target variable other group - irrespective of their true risk. Augmenting Baselines: We give our baselines access to Protected Group (S) the elicited fairness labels by adding them as numerical fea- 0 1 tures to the rows of the input matrix X . For the Crime and Communities data, we added the elicited ratings (1 to Original + iFair + LFR + PFR Hardt + 1.0 1.0 1.0 1.0 1.0 5 stars) as numerical features, with missing values set to -1. 0.8 0.8 0.8 0.8 0.8 0.6 0.6 0.6 0.6 0.6 For the Compas data, where the fairness labels are per-group 0.4 0.4 0.4 0.4 0.4 rankings, we added the ranking position of each individual 0.2 0.2 0.2 0.2 0.2 within its respective group as a numerical feature. 0.0 0.0 0.0 0.0 0.0 FNR FPR FNR FPR FNR FPR FNR FPR FNR FPR 4.3.2 Results on Crime & Communities Dataset (a) Di erence in per-group error rates (FPR and FNR) [Q2] E ect on Individual Fairness: Results on indi- vidual fairness and utility are given in Figure 6. We ob- serve that even though all the methods have access to the Protected Group (S) 0 1 same fairness information, only PFR shows an improvement in consistency W over the baseline. PFR outperforms Original + iFair + LFR + PFR Hardt + 1.0 1.0 1.0 1.0 1.0 all other methods on individual fairness (consistency W ). 0.8 0.8 0.8 0.8 0.8 0.6 0.6 0.6 0.6 0.6 However, this gain for W comes at the cost of losing in 0.4 0.4 0.4 0.4 0.4 consistency as per W . So in this case, the pairwise in- 0.2 0.2 0.2 0.2 0.2 put from human judges exhibits pronounced tension with 0.0 0.0 0.0 0.0 0.0 P(Y = 1) P(Y = 1) P(Y = 1) P(Y = 1) P(Y = 1) the data-attributes input. Deciding which of these sources should take priority is a matter of application design. (b) Di erence in per-group rates of positive prediction [Q3] Trade-o between Utility and Fairness: The higher performance of PFR on individual fairness regard- Figure 7: Crime & Communities data: (a) error rates and ing W comes with a drop in utility as shown by the AUC (b) positive prediction rates. bars in Figure 6. This is because, unlike the case of the syn- thetic data in Subsection 4.2, the side-information for the fairness graph W is not strongly aligned with the ground- truth for the classi er. In terms of relative comparison, we The results for the Compas dataset are mostly in line observe that only PFR shows an improvement in consis- with the results for the synthetic data (in Subsection 4.2) tency W over the baseline, the other approaches show no and Crime & Communities datasets (in Subsection 4.3.2). improvement. The performance of iFair+ and LFR+ on Therefore, we report only brie y on them. F X consistency on W and consistency on W is same as that Utility vs. Individual Fairness: PFR performs similarly of Original+, however for a lower AUC. Hardt+ loses on all as the other representation learning methods in terms of the three measures. utility and individual fairness on W , as shown in Figure 8. Group Fairness: However, PFR clearly outperforms all Crime 1.0 other methods on group fairness. It achieves near-equal rates Method 0.8 of positive predictions as shown in Figure 9b, and near-equal Original + 0.6 error rates across groups as shown in Figure 9a. Again, iFair + PFR's performance on group fairness is as good as that of 0.4 LFR + Hardt+ which is solely designed for equalizing error rates by 0.2 Hardt + post-processing the classi er's outcomes. PFR 0.0 AUC Consistency Consistency F X (W ) (W ) Compas 1.0 Figure 6: Crime & Communities data: utility vs. individ- 0.8 Method ual fairness (higher is better). Original + 0.6 iFair + 0.4 [Q4] In uence on Group Fairness Figure 7a shows the LFR + per-group error rates, and 7b shows the per-group positive 0.2 Hardt + prediction rates. Smaller di erences in the values between PFR 0.0 the two groups are preferable. The following observations AUC Consistency Consistency F X (W ) (W ) are notable: Figure 8: Compas data: utility vs. individual fairness Disparate Mistreatment (aka. Equality of Odds): PFR (higher is better). signi cantly outperforms all other methods on balanc- ing the error rates of the two groups. Furthermore, 4.4 Influence of PFR Hyper-Parameter it achieves nearly equal error rates comparable to the In this subsection we analyze the in uence of on the Hardt+ model, whose sole goal is to achieve equal error rates between groups via post-processing. trade-o between individual fairness (consistency W ) and utility (AUC) of the downstream classi ers. To this end, we Disparate Impact (aka. Demographic parity): PFR clearly keep all other hyper-parameters set to their values for the outperforms all the methods by achieving near perfect best result in the main experiments, and systematically vary balance (i.e., near-equal rates of positive predictions). the value of hyper-parameter in [0,1]. 4.3.3 Results on Compas Dataset Recall that PFR aims to preserve local neighborhoods in Synthetic Low Dim. Synthetic Low Dim. 1.0 Protected Group (S) 1.0 0.9 0 1 0.8 Protected Group (S) 0.6 0.8 Original + iFair + LFR + PFR Hardt + 0 0.4 0.7 1.0 1.0 1.0 1.0 1.0 0.2 0.8 0.8 0.8 0.8 0.8 0.6 0.0 0.6 0.6 0.6 0.6 0.6 0.5 0.0 0.5 1.0 0.4 0.4 0.4 0.4 0.4 0.0 0.5 1.0 0.2 0.2 0.2 0.2 0.2 0.0 0.0 0.0 0.0 0.0 FNR FPR FNR FPR FNR FPR FNR FPR FNR FPR Crime Crime 0.70 0.70 (a) Di erence in per-group error rates (FPR and FNR) 0.65 0.65 Protected Group (S) 0.60 0.60 Protected Group (S) 0.55 0.55 0 1 0.50 0.50 Original + iFair + LFR + PFR Hardt + 0.0 0.5 1.0 0.0 0.5 1.0 1.0 1.0 1.0 1.0 1.0 0.8 0.8 0.8 0.8 0.8 0.6 0.6 0.6 0.6 0.6 Compas Compas 0.4 0.4 0.4 0.4 0.4 0.70 0.70 0.2 0.2 0.2 0.2 0.2 0.0 0.0 0.0 0.0 0.0 0.65 0.65 Protected Group (S) P(Y = 1) P(Y = 1) P(Y = 1) P(Y = 1) P(Y = 1) 0.60 0.60 (b) Di erence in per-group rates of positive prediction 0.55 0.55 0.50 0.50 Figure 9: Compas data: (a) error rates and (b) positive 0.0 0.5 1.0 0.0 0.5 1.0 predictions rates. Figure 10: In uence of on individual fairness and utility. the input space X (given by W ), as well as the similarity given by the fairness graph W , where the hyper-parameter ments can help to improve algorithmic decision making X F controls the relative in uence of W and W . Figure for historically disadvantaged groups. 10 shows the in uence of on individual fairness and utility In contrast, if W pairs of equal deservingness are com- for (a) low-dimensional synthetic, (b) Crime and (c) Compas patible with the classi er's ground-truth labels, there is data, respectively. We make the following key observations. no trade-o between utility and individual fairness. In Individual Fairness: We observe that with increasing such cases, W may even help to improve the utility by the consistency with regard to W increases. This is in line better learning a similarity manifold in the input space. with our expectation: as increases the in uence of W We observe this case for the synthetic data where W is on the objective function, the performance of the model on consistent with the ground-truth labels. As increases, individual fairness (consistency W ) improves. This trend the AUC of a classi er trained on PFR is enhanced. The holds for all the datasets. It is worth highlighting that the improvement in AUC holds for both protected and non- improvement in individual fairness is for newly seen test protected groups. samples that were unknown at the time when the fairness graph W was constructed and the PFR model was learned. 4.5 Sensitivity to Sparseness of Fairness Graph This demonstrates the ability of PFR to generalize individ- In this section we study the sensitivity of PFR to the ual fairness to unseen data. sparseness of the labeled pairs in the fairness graph W . Utility: The in uence of on the utility is more nuanced. We x all hyper-parameters to their best values in the main We observe that the extent of the trade-o between individ- experiments, and systematically vary the fraction of data- ual fairness in W and utility depends on the degree of con- points for which we use pairwise fairness labels. The results ict between the pairwise W , and the classi er's ground- are shown in Figure 11. All results reported are on out-of- truth labels. sample withheld test set of fairness graph W . Recall that If W indicates equal deservingness for data points that PFR accesses fairness labels only for training data. For test have di erent ground-truth labels, there is a natural con- data, it solely has the data attributes available in X . ict between individual fairness and utility. We observe Setup: For the synthetic data, we uniformly at random this case for the real-world datasets Crime and Compas sampled fractions of [log N , , , N , N log N , N ] pairs F 2 2 where W is in tension with ground-truth labels { pre- from the training data, which for this data translates into sumably due to implicit anti-subordination embedded in [9; 120; 600; 5537; 360000] pairs. For the Crime data, we graph or equivalently, due to historic discrimination in varied the percentage of training samples for which use equiv- the classi cation ground-truth. With increasing , there alence labels, in steps of 10% from 10% to 100%. For the is a slight drop in the utility AUC for the non-protected Compas data, we varied the percentage of training data group. However, there is an improvement in AUC for points for which we elicit per-group rankings, in steps of the protected group. The overall AUC drops by a few 10% from 10% to 100%. percentage points, but stays at a high level even for very Results: We observe the following trends. high . So we trade o a substantial gain in individual fairness for a small loss in utility. This is a clear case Increasing the fraction of fairness labels improve the re- of how incorporating side-information on pairwise judg- sults on individual fairness (consistency for W ), while F Consistency (W ) Consistency (W ) Consistency (W ) AUC AUC AUC in the input dataset, as leveraged by the augmented base- Synthetic Low Dim. Crime Compas lines (LFR+, iFair+), perform considerably worse than F F Consistency (W ) Consistency (W ) Consistency (W ) PFR on consistency (W ). 1.0 AUC AUC AUC The W input is needed solely for the training data; pre- 0.70 0.70 0.9 viously unseen test data (at deployment of the learned 0.65 0.65 0.8 representation and downstream classi er) does not have any pairwise judgments at all. This underlines the prac- 0.60 0.60 0.7 tical viability of PFR. 0.55 0.55 0 20 40 60 80 100 0 20 40 60 80 100 Graph Sparseness: Even a small amount of pairwise fair- % of fairness labels % of fairness labels ness judgments helps PFR in improving fairness. At % of pairs with labels some point of extreme sparseness, PFR loses this ad- Figure 11: In uence of fairness-graph sparseness. vantage, but its performance degrades quite gracefully. Robustness: PFR is fairly robust to the dimensionality hurting utility (AUC ) only mildly (or even improving it of the dataset. As the dimensionality of the input data in certain cases). increases, the performance of PFR drops a bit, but still For the synthetic data, even with as little as 0.17% of the outperforms other approaches in terms of balancing fair- fairness labels, the results are already fairly close to the ness and utility. Furthermore, PFR is quite insensitive to best possible: consistency for W is already 90%, and the choice of hyper-parameters. Its performance remains AUC reaches 95%. stable across a wide range of values. For the Crime data, we need about 30 to 40% to get Limitations: When the data exhibits a strong con ict close to the best results for the full fairness graph. How- between fairness and utility goals, even PFR will fail to ever, even with sparseness as low as 10%, PFR degrades counter such tension and will have to prioritize either smoothly: consistency W is 59% compared to 68% for one of the two criteria while degrading on the other. the full graph, and AUC is a ected only mildly by the The human judgements serve to mitigate exactly such sparseness. cases of historical subordination and discrimination, but For the Compas data, we observe similar trends: even if they are too sparse or too noisy, their in uence will be with very sparse W we stay within a few percent of the marginal. For the datasets in our experiments, we as- best possible consistency, and AUC varies only mildly sumed that the information on equally deserving indidi- with changing sparseness of the fairness graph. vuals would re ect high consensus among human judges. These observations indicate that the PFR model yields When this assumption is invalid for certain datasets, bene ts already with a small amount of human judgements PFR will lose its advantages and perform as poorly as of equally deserving individuals. (but no worse than) other methods. 4.6 Discussion and Lessons 5. CONCLUSIONS The experimental results suggest several key ndings. This paper proposes a new departure for the hot topic of Individual Fairness - Utility Trade-o : The extent of how to incorporate fairness in algorithmic decision making. this trade-o depends on the degree of con ict between Building on the paradigm of individual fairness, we devised the fairness graph and the classi er's ground-truth la- a new method, called PFR, for operationalizing this line of bels. When edges in the fairness graph connect data models, by eliciting and leveraging side-information on pairs points (for equally deserving individuals) that have dif- of individuals who are equally deserving and, thus, should ferent ground-truth labels, there is an inherent tension be treated similarly for a given task. We developed a rep- between individual fairness and utility. resentation learning model to learn Pairwise Fair Represen- For datasets where some compromise is unavoidable, PFR tations (PFR), as a fairness-enhanced input to downstream turns out to perform best in balancing the di erent goals. machine-learning tasks. Comprehensive experiments, with It is consistently best with regard to individual fairness, synthetic and real-life datasets, indicate that the pairwise by a substantial margin over the other methods. On judgements are bene cial for members of the protected group, utility, its AUC is competitive and always close to the resulting in high individual fairness and high group fairness best performing method on this metric, typically within (near-equal error rates across groups) with reasonably low 2 percentage points of the best AUC result. loss in utility. Balancing Individual Fairness and Group Fairness: The human judgements cast into the fairness graph help PFR 6. ACKNOWLEDGMENT to perform well also on group fairness criteria. On these This research was supported by the ERC Synergy Grant measures, PFR is almost as good as the method by Hardt \imPACT" (No. 610150) and ERC Advanced Grant \Foun- et al., which is speci cally geared for group fairness (but dations for Fair Social Computing" (No. 789373). disregards individual fairness). To a large extent, this is because the pairwise fairness judgments address his- torical subordination of groups. Eliciting human judge- 7. REFERENCES ments is a crucial asset for fair machine learning in a [1] E. Amid and A. Ukkonen. Multiview triplet wider sense. embedding: Learning attributes in multiple maps. In Data Representation: The graph-embedding method used ICML, 2015. by PFR appears to the best way of incorporating the [2] E. Anderson, Z. Bai, J. Dongarra, A. Greenbaum, pairwise human judgements. Alternative representations A. McKenney, J. Du Croz, S. Hammarling, of the same raw information such as additional features 2.6e 3.3e 6.7e 1.0e 1.3e 1.7e 100 J. Demmel, C. Bischof, and D. Sorensen. Lapack: A and Z. S. Wu. Eliciting and enforcing subjective portable linear algebra library for high-performance individual fairness. arXiv preprint arXiv:1905.10660, computers. In ICS, 1990. 2019. [3] J. Angwin, J. Larson, S. Mattu, and L. Kirchner. [21] F. Kamiran, T. Calders, and M. Pechenizkiy. Machine bias: There's software used across the Discrimination aware decision tree learning. In ICDM, country to predict future criminals and it's biased 2010. against blacks. In ProPublica 2016. [22] T. Kamishima, S. Akaho, H. Asoh, and J. Sakuma. [4] A. Asudeh, H. V. Jagadish, J. Stoyanovich, and Considerations on fairness-aware data mining. In ICDM, 2012. G. Das. Designing fair ranking schemes. In SIGMOD, 2019. [23] T. Kamishima, S. Akaho, and J. Sakuma. Fairness-aware learning through regularization [5] J. Biega, K. P. Gummadi, and G. Weikum. Equity of approach. In ICDMW, 2011. attention: Amortizing individual fairness in rankings. [24] M. Kearns, A. Roth, and Z. S. Wu. Meritocratic In SIGIR, 2018. fairness for cross-population selection. In ICML, 2017. [6] T. Brennan, W. Dieterich, and B. Ehret. Evaluating [25] J. M. Kleinberg, S. Mullainathan, and M. Raghavan. the predictive validity of the compas risk and needs Inherent trade-o s in the fair determination of risk assessment system. CJB, 2009. scores. In ITCS, 2017. [7] R. L. Brooks. Rethinking the American race problem. [26] P. Lahoti, K. P. Gummadi, and G. Weikum. ifair: Univ of California Press, 1992. Learning individually fair data representations for [8] T. Calders, F. Kamiran, and M. Pechenizkiy. Building algorithmic decision making. In ICDE, 2019. classi ers with independency constraints. In ICDM, 2009. [27] Y. Lin, T. Liu, and H. Chen. Semantic manifold learning for image retrieval. In ACM Multimedia, 2005. [9] F. Chierichetti, R. Kumar, S. Lattanzi, and S. Vassilvitskii. Fair clustering through fairlets. In [28] R. M. Communities and crime dataset, uci machine learning repository, 2009. NIPS, 2017. [10] A. Chouldechova. Fair prediction with disparate [29] D. Pedreschi, S. Ruggieri, and F. Turini. Discrimination-aware data mining. In KDD, 2008. impact: A study of bias in recidivism prediction instruments. Big data, 2017. [30] B. Salimi, L. Rodriguez, B. Howe, and D. Suciu. Interventional fairness: Causal database repair for [11] S. Corbett-Davies, E. Pierson, A. Feller, S. Goel, and algorithmic fairness. In SIGMOD, 2019. A. Huq. Algorithmic decision making and the cost of fairness. In KDD, 2017. [31] T. Speicher, H. Heidari, H. Grgic-Hlaca, K. Gummadi, A. Singla, A. Weller, and M. B. Zafar. A uni ed [12] K. Crawford. Arti cial intelligence^ aAZs white guy approach to quantifying algorithmic unfairness: problem. The New York Times 2016, 2016. Measuring individual &group unfairness via inequality [13] C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and indices. In KDD, 2018. R. S. Zemel. Fairness through awareness. In ITCS, [32] J. Stoyanovich, K. Yang, and H. V. Jagadish. Online set selection with fairness and diversity constraints. In [14] S. Elbassuoni, S. Amer-Yahia, C. E. Atie, EDBT, 2018. A. Ghizzawi, and B. Oualha. Exploring fairness of ranking in online job marketplaces. In EDBT, 2019. [33] M. Zafar, I. Valera, M. Rodriguez, K. Gummadi, and A. Weller. From parity to preference-based notions of [15] M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, fairness in classi cation. In NIPS, 2017. and S. Venkatasubramanian. Certifying and removing [34] M. B. Zafar, I. Valera, M. Gomez-Rodriguez, and disparate impact. In KDD, 2015. K. P. Gummadi. Fairness beyond disparate treatment [16] S. A. Friedler, C. Scheidegger, and & disparate impact: Learning classi cation without S. Venkatasubramanian. On the (im)possibility of disparate mistreatment. In WWW, 2017. fairness. CoRR, abs/1609.07236, 2016. [35] M. B. Zafar, I. Valera, M. Gomez-Rodriguez, and [17] S. Gillen, C. Jung, M. Kearns, and A. Roth. Online K. P. Gummadi. Fairness constraints: Mechanisms for learning with an unknown fairness metric. In fair classi cation. In AISTATS, 2017. NeurIPS, 2018. [36] M. Zehlike, F. Bonchi, C. Castillo, S. Hajian, [18] W. L. Hamilton, R. Ying, and J. Leskovec. M. Megahed, and R. A. Baeza-Yates. Fa*ir: A fair Representation learning on graphs: Methods and top-k ranking algorithm. In CIKM, 2017. applications. IEEE Data Eng. Bull., 2017. [37] R. S. Zemel, Y. Wu, K. Swersky, T. Pitassi, and [19] M. Hardt, E. Price, and N. Srebro. Equality of C. Dwork. Learning fair representations. In ICML, opportunity in supervised learning. In In NIPS 2016. [20] C. Jung, M. Kearns, S. Neel, A. Roth, L. Stapleton,
Journal
Statistics – arXiv (Cornell University)
Published: Jul 2, 2019