Fairness (machine learning)

We say the random variables ${\textstyle (R,A)}$ satisfy independence if the sensitive characteristics ${\textstyle A}$ are statistically independent to the prediction ${\textstyle R}$, and we write ${\textstyle R\bot A}$.

We can also express this notion with the following formula:

$${\displaystyle P(R=r|A=a)=P(R=r|A=b)\quad \forall r\in R\quad \forall a,b\in A}$$

This means that the probability of being classified by the algorithm in each of the groups is equal for two individuals with different sensitive characteristics.

Yet another equivalent expression for independence can be given using the concept of mutual information between random variables, defined as

$${\displaystyle I(X,Y)=H(X)+H(Y)-H(X,Y)}$$

In this formula, ${\textstyle H}$ of the random variable. Then ${\textstyle (R,A)}$ satisfy independence if ${\textstyle H(R,A)=0}$.

A possible relaxation of the indepence definition include introducing a positive slack ${\textstyle \epsilon >0}$ and is given by the formula: ${\displaystyle P(R=r|A=a)\geq P(R=r|A=b)-\epsilon \quad \forall r\in R\quad \forall a,b\in A}$

Finally, another possible relaxation is to require ${\textstyle I(R,A)\leq \epsilon }$.

We say the random variables ${\textstyle (R,A,Y)}$ satisfy separation if the sensitive characteristics ${\textstyle A}$ are statistically independent to the prediction ${\textstyle R}$ given the target value ${\textstyle Y}$, and we write ${\textstyle R\bot A|Y}$.

We can also express this notion with the following formula:

$${\displaystyle P(R=r|Y=q,A=a)=P(R=r|Y=q,A=b)\quad \forall r\in R\quad q\in Y\quad \forall a,b\in A}$$

This means that the probability of being classified by the algorithm in each of the groups is equal for two individuals with different sensitive characteristics given that they actually belong in the same group (have the same target variable).

Another equivalent expression, in the case of a binary target rate, is that the true positive rate and the false positive rate are equal (and therefore the false negative rate and the true negative rate are equal) for every value of the sensitive characteristics:

$${\displaystyle P(R=1|Y=1,A=a)=P(R=1|Y=1,A=b)\quad \forall a,b\in A}$$

$${\displaystyle P(R=1|Y=0,A=a)=P(R=1|Y=0,A=b)\quad \forall a,b\in A}$$

Finally, a possible relaxation of the given definitions is the difference between rates to be a positive number lower than a given slack ${\textstyle \epsilon >0}$, instead of equals to zero.

We say the random variables ${\textstyle (R,A,Y)}$ satisfy sufficiency if the sensitive characteristics ${\textstyle A}$ are statistically independent to the target value ${\textstyle Y}$ given the prediction ${\textstyle R}$, and we write ${\textstyle Y\bot A|R}$.

We can also express this notion with the following formula:

$${\displaystyle P(Y=q|R=r,A=a)=P(Y=q|R=r,A=b)\quad \forall q\in Y\quad r\in R\quad \forall a,b\in A}$$

This means that the probability of actually being in each of the groups is equal for two individuals with different sensitive characteristics given that they were predicted to belong to the same group.

Finally, we sum up some of the main results that relate the three definitions given above:

• If ${\textstyle A}$ and ${\textstyle Y}$ are not statistically independent, then sufficiency and independence cannot both hold.
• Assuming ${\textstyle Y}$ is binary, if ${\textstyle A}$ and ${\textstyle Y}$ are not statistically independent, and ${\textstyle R}$ and ${\textstyle Y}$ are not statistically independent either, then independence and separation cannot both hold.
• If ${\textstyle (R,A,Y)}$ as a joint distribution has positive probability for all its possible values and ${\textstyle A}$ and ${\textstyle Y}$ are not statistically independent, then separation and sufficiency cannot both hold.

The definitions in this section focus on a predicted outcome ${\textstyle R}$ for various distributions of subjects. They are the simplest and most intuitive notions of fairness.

• Group fairness, also referred to as statistical parity, demographic parity, acceptance rate and benchmarking. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal probability of being assigned to the positive predicted class. This is, if the following formula is satisfied:

$${\displaystyle P(R=+|A=a)=P(R=+|A=b)\quad \forall a,b\in A}$$

• Conditional statistical parity. Basically consists in the definition above, but restricted only to a subset of the attributes. With mathematical notation this would be:

$${\displaystyle P(R=+|L=l,A=a)=P(R=+|L=l,A=b)\quad \forall a,b\in A\quad \forall l\in L}$$

This definitions not only consider de predicted outcome ${\textstyle R}$ but also compare it to the actual outcome ${\textstyle Y}$.

• Predictive parity, also referred to as outcome test. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal PPV. This is, if the following formula is satisfied:

$${\displaystyle P(Y=+|R=+,A=a)=P(Y=+|R=+,A=b)\quad \forall a,b\in A}$$

Mathematically, if a classifier has equal PPV for both groups, it will also have equal FDR, satisfying the formula:

$${\displaystyle P(Y=-|R=+,A=a)=P(Y=-|R=+,A=b)\quad \forall a,b\in A}$$

• False positive error rate balance, also referred to as predictive equality. A classifier satisfies this definition if the subjects in the protected and unprotected groups have aqual FPR. This is, if the following formula is satisfied:

$${\displaystyle P(R=+|Y=-,A=a)=P(R=+|Y=-,A=b)\quad \forall a,b\in A}$$

Mathematically, if a classifier has equal FPR for both groups, it will also have equal TNR, satisfying the formula:

$${\displaystyle P(R=-|Y=-,A=a)=P(R=-|Y=-,A=b)\quad \forall a,b\in A}$$

• False negative error rate balance, also referred to as equal opportunity. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal FNR. This is, if the following formula is satisfied:

$${\displaystyle P(R=-|Y=+,A=a)=P(R=-|Y=+,A=b)\quad \forall a,b\in A}$$

Mathematically, if a classifier has equal FNR for both groups, ti will also have equal TPR, satisfying the formula:

$${\displaystyle P(R=+|Y=+,A=a)=P(R=+|Y=+,A=b)\quad \forall a,b\in A}$$

• Equalized odds, also referred to as conditional procedure accuracy equality and disparate mistreatment. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal TPR and equal FPR, satisfying the formula:

$${\displaystyle P(R=+|Y=y,A=a)=P(R=+|Y=y,A=b)\quad y\in \{+,-\}\quad \forall a,b\in A}$$

• Conditional use accuracy equality. A classifier satisfies this definition if the subjects in the protected and unprotected groups have equal PPV and equal NPV, satisfying the formula:

$${\displaystyle P(Y=y|R=y,A=a)=P(Y=y|R=y,A=b)\quad y\in \{+,-\}\quad \forall a,b\in A}$$

• Overall accuracy equality. A classifier satisfies this definition if the subject in the protected and unprotected groups have equal prediction accuracy, that is, the probability of a subject from one class to be assigned to it. This is, if it satisfies the following formula:

$${\displaystyle P(R=Y,A=a)=P(R=Y|A=b)\quad \forall a,b\in A}$$

• Treatment equality. A classifier satisfies this definition if the subjects in the protected and unprotected groups have an equal ratio of FN and FP, satisfying the formula:

$${\displaystyle {\frac {FN_{A=a}}{FP_{A=a}}}={\frac {FN_{A=b}}{FP_{A=b}}}}$$

These definitions are based in the actual outcome ${\textstyle Y}$ and the predicted probability score ${\textstyle S}$.

• Test-fairness, also known as calibration or matching conditional frequencies. A classifier satisfies this definition if individuals with the same predicted probability score ${\textstyle S}$ have the same probability to be classified in the positive class when they belong to either the protected or the unprotected group:

$${\displaystyle P(Y=+|S=s,A=a)=P(Y=+|S=s,A=b)\quad \forall s\in S\quad \forall a,b\in A}$$

• Well-calibration is an extension of the previous definition. It states that when individuals inside or outside the protected group have the same predicted probability score ${\textstyle S}$ they must have the same probability of being classified in the positive class, and this probability must be equal to ${\textstyle S}$:

$${\displaystyle P(Y=+|S=s,A=a)=P(Y=+|S=s,A=b)=s\quad \forall s\in S\quad \forall a,b\in A}$$

• Balance for positive class. A classifier satisfies this definition if the subjects constituting the positive class from both protected and unprotected groups have equal average predicted probability score ${\textstyle S}$. This means that the expected value of probability score for the protected and unprotected groups with positive actual outcome ${\textstyle Y}$ is the same, satisfying the formula:

$${\displaystyle E(S|Y=+,A=a)=E(S|Y=+,A=b)\quad \forall a,b\in A}$$

• Balance for negative class. A classifier satisfies this definition if the subjects constituting the negative class from both protected and unprotected groups have equal average predicted probability score ${\textstyle S}$. This means that the expected value of probability score for the protected and unprotected groups with negative actual outcome ${\textstyle Y}$ is the same, satisfying the formula:

$${\displaystyle E(S|Y=-,A=a)=E(S|Y=-,A=b)\quad \forall a,b\in A}$$

Usually, the classifier is not the only problem, the dataset is also biased. The discrimination of a dataset ${\textstyle D}$ with respect to the group ${\textstyle A=a}$ can be defined as follows:

$${\displaystyle disc_{A=a}(D)={\frac {|\{X\in D|X(A)\neq a,X(Y)=+\}|}{|\{X\in D|X(A)\neq a\}|}}-{\frac {|\{X\in D|X(A)=a,X(Y)=+\}|}{|\{X\in D|X(A)=a\}|}}}$$

That is, an approximation to the difference between the probabilities of belonging in the positive class given that the subject has a protected characteristic different from ${\textstyle a}$ and equal to ${\textstyle a}$.

Algorithms correcting bias at preprocessing remove information concerning variables in the dataset which can result in unfair decisions of the AI, while trying to alter just the bare minimum of this data. This is not as easy as just removing the sensitive variable, because other attributes can be related to the protected one.

A way to do this is by mapping each individual in the initial dataset into an intermediate representation in which it is impossible to identify whether it belongs to a particular protected group while maintaining as much information as possible. Then, the new representation of the data is adjusted to get the maximum accuracy in the algorithm.

This way, individuals are mapped into a new multivariable representation where the probability of any member of a protected group to be mapped to a certain value in the new representation is the same as the probability of an individual which doesn't belong to the protected group. Then, this representation is used to obtain the prediction for the individual, instead of the initial data. As the intermediate representation is constructed giving the same probability to individuals inside or outside the protected group, this attribute is hidden to the classificator.

An example is explained in Zemel et al.[7] where a multinomial random variable is used as an intermediate representation. In the process, the system is encouraged to preserve all the information except those that can lead to biased decisions and to obtain a prediction as accurate as possible.

On the one hand, this procedure has the advantage that the preprocessed data can be used for any machine learning task. Furthermore, the classifier does not need to be modified, as the correction is applied to the dataset before processing. On the other hand, the other methods obtain better results in accuracy and fairness.[8]

Reweighing is an example of a preprocessing algorithm. The idea is to assign a weight to each dataset point such that the weighted discrimination is 0 with respect to the designated group.

If the dataset ${\textstyle D}$ was unbiased the sensitive variable ${\textstyle A}$ and the target variable ${\textstyle Y}$ would be statistically independent and the probability of the joint distribution would be the product of the probabilities as follows:

$${\displaystyle P_{exp}(A=a\wedge Y=+)=P(A=a)\times P(Y=+)={\frac {|\{X\in D|X(A)=a\}|}{|D|}}\times {\frac {|\{X\in D|X(Y)=+\}|}{|D|}}}$$

In reality, however, the dataset is not unbiased and the variables are not statistically independent so the observed probability is:

$${\displaystyle P_{obs}(A=a\wedge Y=+)={\frac {|\{X\in D|X(A)=a\wedge X(Y)=+\}|}{|D|}}}$$

To compensate for the bias, lower weights to favored objects and higher weights to unfavored objects will be assigned. For each ${\textstyle X\in D}$ we get:

$${\displaystyle W(X)={\frac {P_{exp}(A=X(A)\wedge Y=X(Y))}{P_{obs}(A=X(A)\wedge Y=X(Y))}}}$$

When we have for each ${\textstyle X}$ a weight associated ${\textstyle W(X)}$ we compute the weighted discrimination with respect to group ${\textstyle A=a}$ as follows:

$${\displaystyle disc_{A=a}(D)={\frac {\sum W(X)X\in \{X\in D|X(A)\neq a,X(Y)=+\}}{\sum W(X)X\in \{X\in D|X(A)\neq a\}}}-{\frac {\sum W(X)X\in \{X\in D|X(A)=a,X(Y)=+\}}{\sum W(X)X\in \{X\in D|X(A)=a\}}}}$$

It can be shown that after reweighting this weighted discrimination is 0.

Another approach is correcting the bias at training time. This can be done by adding constraints to the optimization objective of the algorithm.[10] These constraints force the algorithm to improve fairness, by keeping the same rates of certain measures for the protected group and the rest of individuals. For example, we can add to the objective of the algorithm the condition that the false positive rate is the same for individuals in the protected group and the ones outside the protected group.

The main measures used in this approach are false positive rate, false negative rate, and overall misclassification rate. It is possible to add just one or several of these constraints to the objective of the algorithm. Note that the equality of false negative rates implies the equality of true positive rates so this implies the equality of opportunity. After adding the restrictions to the problem it may turn intractable, so a relaxation on them may be needed.

This technique obtains good results in improving fairness while keeping high accuracy and lets the programmer choose the fairness measures to improve. However, each machine learning task may need a different method to be applied and the code in the classifier needs to be modified, which is not always possible.[8]

We train two classifiers at the same time through some gradient-based method (f.e.: gradient descent). The first one, the predictor tries to accomplish the task of predicting ${\textstyle Y}$, the target variable, given ${\textstyle X}$, the input, by modifying its weights ${\textstyle W}$ to minimize some loss function ${\textstyle L_{P}({\hat {y}},y)}$. The second one, the adversary tries to accomplish the task of predicting ${\textstyle A}$, the sensitive variable, given ${\textstyle {\hat {Y}}}$ by modifying its weights ${\textstyle U}$ to minimize some loss function ${\textstyle L_{A}({\hat {a}},a)}$.

An important point here is that, in order to propagate correctly, ${\textstyle {\hat {Y}}}$ above must refer to the raw output of the classifier, not the discrete prediction; for example, with an artificial neural network and a classification problem, ${\textstyle {\hat {Y}}}$ could refer to the output of the softmax layer.

Then we update ${\textstyle U}$ to minimize ${\textstyle L_{A}}$ at each training step according to the gradient ${\textstyle \nabla _{U}L_{A}}$ and we modify ${\textstyle W}$ according to the expression:

$${\displaystyle \nabla _{W}L_{P}-proj_{\nabla _{W}L_{A}}\nabla _{W}L_{P}-\alpha \nabla _{W}L_{A}}$$

where $\alpha$ is a tuneable hyperparameter that can vary at each time step.

Graphic representation of the vectors used in adversarial debiasing as shown in Zhan et al.[11]

The intuitive idea is that we want the predictor to try to minimize ${\textstyle L_{P}}$ (therefore the term ${\textstyle \nabla _{W}L_{P}}$) while, at the same time, maximize ${\textstyle L_{A}}$ (therefore the term ${\textstyle -\alpha \nabla _{W}L_{A}}$), so that the adversary fails at predicting the sensitive variable from ${\textstyle {\hat {Y}}}$.

The term ${\textstyle -proj_{\nabla _{W}L_{A}}\nabla _{W}L_{P}}$ prevents the predictor from moving in a direction that helps the adversary decrease its loss function.

It can be shown that training a predictor classification model with this algorithm improves demographic parity with respect to training it without the adversary.

The final method tries to correct the results of a classifier to achieve fairness. In this method, we have a classifier that returns a score for each individual and we need to do a binary prediction for them. High scores are likely to get a positive answer, while low scores are likely to get a negative answer, but we need to adjust the threshold to determine when to answer yes or no depending on our needs. Note that variations in the threshold affect the trade-off between true positive rate and true negative rate.

If the score function is fair in the sense that it is independent of the protected attribute, then any choice of the threshold will also be fair, but this type of classifiers tend to be biased, so we may need to set a different threshold for each protected group to achieve fairness. A way to do this is plotting the true positive rate against the false negative rate at various threshold settings (this is called ROC curve) and check which threshold satisfies that the rates are equal for the protected group and the rest of the individuals.[13]

The advantages of postprocessing include that the technique can be applied after any classifiers, without modifying it, and has a good performance in fairness measures. The cons are the need to access to the protected attribute in test time and the lack of choice in the balance between accuracy and fairness.[8]

Given a classifier let ${\textstyle P(+|X)}$ be the probability computed by the classifiers as the probability that the instance ${\textstyle X}$ belongs to the positive class +. When ${\textstyle P(+|X)}$ is close to 1 or to 0, the instance ${\textstyle X}$ is specified with high degree of certainty to belong to class + or - respectively. However, when ${\textstyle P(+|X)}$ is closer to 0.5 the classification is more unclear.

We say ${\textstyle X}$ is a "rejected instance" if ${\textstyle max(P(+|X),1-P(+|X))\leq \theta }$ with a certain ${\textstyle \theta }$ such that ${\textstyle 0.5<\theta <1}$.

The algorithm of "ROC" consists on classifying the non-rejected instances following the rule above and the rejected instances as follows: if the instance is an example of a deprived group (${\displaystyle X(A)=a}$) then label it as positive, otherwise, label it as negative.

We can optimize different measures of discrimination (link) as functions of ${\textstyle \theta }$ to find the optimal ${\textstyle \theta }$ for each problem and avoid becoming discriminatory against the privileged group.