Date of Award
1-1-2024
Document Type
Dissertation
Degree Name
Ph.D. in Engineering Science
First Advisor
Yixin Chen
Second Advisor
Kris Belden-Adams
Third Advisor
Timothy Holston
School
University of Mississippi
Relational Format
dissertation/thesis
Abstract
Artificial intelligence and machine learning are becoming more present in everyday life. Training data distributions, amongst other variables, can cause these systems to be unfair. Users, as well as other individuals, may be negatively impacted. This work studies using modified loss functions to mitigate race and sex bias caused by data and models.
In the first experiment, the trade-off between fairness and accuracy when data is biased and unbiased is explored. A group risk function is introduced and used to create two versions of a modified loss function: Max Group Equity and Max Group Equality Binary Cross-Entropy (BCE). Both functions add the maximum group loss to the standard BCE loss and use a hyperparameter to adjust its significance. Max Group Equity BCE is more suited for unbiased data, while Max Group Equality BCE is useful for heavily biased data. Both are tested by training logistic regression models on Home Mortgage Disclosure Act (HMDA) data. The data and models are evaluated using multiple fairness metrics, including p-value and statistical parity difference. Race and race-sex group bias is observed. Results show that Max Group Equity BCE maintains high accuracy while slightly improving group fairness. Max Group Equality BCE increases group fairness substantially. However, there is usually a steep decline in accuracy.
In the second experiment, another group risk function is introduced, and the modified loss approach is expanded to categorical cross-entropy (CCE). Loss functions that use two different group risk functions are introduced as well. Face images from the FairFace dataset are used for binary and categorical classification tasks. The Max Group Equity and Max Group Equality BCE loss functions work moderately well with image classification. In some cases, combining the two yields better results than using one alone. There is some concern that both loss functions inject/invert bias instead of mitigating it. The new group risk function does not work well with binary or categorical image classification and overpowers any loss function it is a part of. The Max Group Equity CCE loss function does not yield as promising results as its BCE counterpart but may still have potential.
Recommended Citation
Trotter, Christina Joy, "Modified Loss Functions for Algorithmic Bias Mitigation" (2024). Electronic Theses and Dissertations. 2984.
https://egrove.olemiss.edu/etd/2984