Towards Fair Face Verification: An In-depth Analysis of Demographic Biases

Towards Fair Face Verification: An In-depth Analysis of Demographic Biases

연구 분야: Strategies

학회: Joint European Conference on Machine Learning and Knowledge Discovery in Databases

Deep learning-based person identification and verification systems have remarkably improved in terms of accuracy in recent years; however, such systems, including popular cloud services, have been found to exhibit significant biases related to race, age, and gender, that call for in-depth exploration and mitigation. This paper presents an in-depth analysis, with a particular emphasis on the intersectionality of these demographic factors. Intersectional bias refers to performance discrepancies w.r.t. the different combinations of race, age, and gender groups, an under-explored area in current literature. Furthermore, the reliance of most state-of-the-art approaches on accuracy as the principal evaluation metric often masks significant demographic disparities. To address this limitation, we incorporate five additional metrics in our quantitative analysis, including disparate impact and mistreatment metrics, which are typically ignored by relevant fairness-aware approaches. Results on the Racial Faces in-the-Wild (RFW) benchmark indicate pervasive biases in face recognition systems, extending beyond race, with different demographic factors yielding significantly disparate outcomes. In particular, Africans demonstrate an 11.25% lower True Positive Rate (TPR) compared to Caucasians, while only a 3.51% accuracy drop is observed. Even more concerning, the intersections of multiple protected groups, such as African females over 60 years old, demonstrate a +39.89% disparate mistreatment rate compared to the highest Caucasians rate. By shedding light on these biases and their implications, this paper aims to stimulate further research towards developing fairer face recognition and verification systems.