journal article·2024-10-02Open access

Bias in Machine Learning: A Literature Review

Konstantinos Mavrogiorgos, Athanasios Kiourtis, Argyro Mavrogiorgou, Andreas Menychtas, Dimosthenis Kyriazis

Applied Sciences· Volume 14· Issue 19

Abstract

Bias could be defined as the tendency to be in favor or against a person or a group, thus promoting unfairness. In computer science, bias is called algorithmic or artificial intelligence (i.e., AI) and can be described as the tendency to showcase recurrent errors in a computer system, which result in “unfair” outcomes. Bias in the “outside world” and algorithmic bias are interconnected since many types of algorithmic bias originate from external factors. The enormous variety of different types of AI biases that have been identified in diverse domains highlights the need for classifying the said types of AI bias and providing a detailed overview of ways to identify and mitigate them. The different types of algorithmic bias that exist could be divided into categories based on the origin of the bias, since bias can occur during the different stages of the Machine Learning (i.e., ML) lifecycle. This manuscript is a literature study that provides a detailed survey regarding the different categories of bias and the corresponding approaches that have been proposed to identify and mitigate them. This study not only provides ready-to-use algorithms for identifying and mitigating bias, but also enhances the empirical knowledge of ML engineers to identify bias based on the similarity that their use cases have to other approaches that are presented in this manuscript. Based on the findings of this study, it is observed that some types of AI bias are better covered in the literature, both in terms of identification and mitigation, whilst others need to be studied more. The overall contribution of this research work is to provide a useful guideline for the identification and mitigation of bias that can be utilized by ML engineers and everyone who is interested in developing, evaluating and/or utilizing ML models.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Imbalanced Data Classification Techniques · Machine Learning and Data Classification · Explainable Artificial Intelligence (XAI)

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Digital Systems, University of Piraeus, 185 34 Piraeus, Greece

Funding

European Union’s funded Projects AI4Gov · XR5.0

References (100+)

Show source-provided references
1. Algorithmic bias: Senses, sources, solutions10.1111/phc3.12760 ↗
3. (2024, February 15). Statista—“Market Size and Revenue Comparison for Artificial Intelligence Worldwide from 2018 to 2030”. Available online: https://www.statista.com/statistics/941835/artificial-intelligence-market-size-revenue-comparisons.
4. (2024, February 15). Statista—“Share of Adults in the United States Who Were Concerned about Issues Related to Artificial Intelligence (AI) as of February 2023”. Available online: https://www.statista.com/statistics/1378220/us-adults-concerns-about-artificial-intelligence-related-issues.
5. ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope10.1016/j.iotcps.2023.04.003 ↗
6. Meyer, J.G., Urbanowicz, R.J., Martin, P.C.N., O’connor, K., Li, R., Peng, P.-C., Bright, T.J., Tatonetti, N., Won, K.J., and Gonzalez-Hernandez, G. (2023). ChatGPT and large language models in academia: Opportunities and challenges. BioData Min., 16.10.1186/s13040-023-00339-9 ↗
7. Yee, K., Tantipongpipat, U., and Mishra, S. (2021, January 23). Image cropping on twitter: Fairness metrics, their limitations, and the importance of representation, design, and agency. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2), Virtual.10.1145/3479594 ↗
8. Birhane, A., Prabhu, V.U., and Whaley, J. (2022, January 5). Auditing saliency cropping algorithms. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV51458.2022.00158 ↗
9. Dressel, J.J. (2017). Accuracy and Racial Biases of Recidivism Prediction Instruments. [Undergraduate Thesis, Dartmouth College].
10. The limits of human predictions of recidivism10.1126/sciadv.aaz0652 ↗
11. Code is law: How COMPAS affects the way the judiciary handles the risk of recidivism
12. Roselli, D., Matthews, J., and Talagala, N. (2019, January 13). Managing bias in AI. Proceedings of the 2019 World Wide Web Conference, San Fransisco, CA, USA.10.1145/3308560.3317590 ↗
13. Algorithmic bias: Review, synthesis, and future research directions10.1080/0960085X.2021.1927212 ↗
14. Taming technical bias in machine learning pipelines
15. Improving Trust in AI with Mitigating Confirmation Bias: Effects of Explanation Type and Debiasing Strategy for Decision-Making with Explainable AI10.1080/10447318.2023.2285640 ↗
16. Handling imbalanced datasets: A review
17. Under-sampling approaches for improving prediction of the minority class in an imbalanced dataset
18. Cluster-based under-sampling approaches for imbalanced data distributions10.1016/j.eswa.2008.06.108 ↗
19. Inverse random under sampling for class imbalance problem and its application to multi-label classification10.1016/j.patcog.2012.03.014 ↗
20. Classification of imbalance data using tomek link (t-link) combined with random under-sampling (rus) as a data reduction method
21. SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary10.1613/jair.1.11192 ↗
22. He, H., Bai, Y., Garcia, E.A., and Li, S. (2008, January 1). ADASYN: Adaptive synthetic sampling approach for imbalanced learning. Proceedings of the 2008 IEEE International Joint Conference on Neural Netw, (IEEE World Congress on Computational Intelligence), Hong Kong, China.
23. AMDO: An over-sampling technique for multi-class imbalanced problems10.1109/TKDE.2017.2761347 ↗
24. Azaria, A. (2023, January 6). ChatGPT: More Human-Like Than Computer-Like, but Not Necessarily in a Good Way. Proceedings of the 2023 IEEE 35th International Conference on Tools with Artificial Intelligence (ICTAI), Atlanta, GA, USA.10.1109/ICTAI59109.2023.00074 ↗
25. Atreides, K., and Kelley, D. (2024, February 15). Cognitive Biases in Natural Language: Automatically Detecting, Differentiating, and Measuring Bias in Text. Differentiating, and Measuring Bias in Text. Available online: https://www.researchgate.net/profile/Kyrtin-Atreides/publication/372078491_Cognitive_Biases_in_Natural_Language_Automatically_Detecting_Differentiating_and_Measuring_Bias_in_Text/links/64a3e11195bbbe0c6e0f149c/Cognitive-Biases-in-Natural-Language-Automatically-Detecting-Differentiating-and-Measuring-Bias-in-Text.pdf.10.2139/ssrn.4927380 ↗
26. Blawatt, K.R. (2016). Appendix A: List of cognitive biases. Marconomics, Emerald Group Publishing Limited.10.1108/9781786355652 ↗
27. Sayão, L.F., and Baião, F.A. (September, January 28). An Ontology-based Data-driven Architecture for Analyzing Cognitive Biases in Decision-making. Proceedings of the XVI Seminar on Ontology Research in Brazil (ONTOBRAS 2023) and VII Doctoral and Masters Consortium on Ontologies, (WTDO 2023), Brasilia, Brazil.
28. Harris, G. (2020, January 20–24). Methods to Evaluate Temporal Cognitive Biases in Machine Learning Prediction Models. Proceedings of the Companion Proceedings of the Web Conference 2020, Taipei, Taiwan.
29. Liu, Q., Jiang, H., Pan, Z., Han, Q., Peng, Z., and Li, Q. (2024, January 18–21). BiasEye: A Bias-Aware Real-time Interactive Material Screening System for Impartial Candidate Assessment. Proceedings of the IUI ‘24: 29th International Conference on Intelligent User Interfaces, Greenville, SC, USA.10.1145/3640543.3645166 ↗
30. Harris, G. (2020, January 20–24). Mitigating cognitive biases in machine learning algorithms for decision making. Proceedings of the Companion Proceedings of the Web Conference 2020, Taipei, Taiwan.
31. Using causal inference to avoid fallouts in data-driven parametric analysis: A case study in the architecture, engineering, and construction industry10.1016/j.dibe.2023.100296 ↗
32. Kavitha, J., Kiran, J., Prasad, S.D.V., Soma, K., Babu, G.C., and Sivakumar, S. (2022, January 16). Prediction and Its Impact on Its Attributes While Biasing MachineLearning Training Data. Proceedings of the 2022 Third International Conference on Smart Technologies in Computing, Electrical and Electronics (ICSTCEE), Bengaluru, India.10.1109/ICSTCEE56972.2022.10100010 ↗
33. Schmidgall, S. (2024). Addressing cognitive bias in medical language models. arXiv.
34. Bareinboim, E., Tian, J., and Pearl, J. (2022). Recovering from selection bias in causal and statistical inference. Probabilistic and Causal Inference: The Works of Judea Pearl, Association for Computing Machinery.10.1145/3501714.3501740 ↗
35. Selection bias and information bias in clinical research10.1159/000312871 ↗
36. Selection mechanisms and their consequences: Understanding and addressing selection bias10.1007/s40471-020-00241-6 ↗
37. Mendez, M., Maathuis, B., Hein-Griggs, D., and Alvarado-Gamboa, L.-F. (2020). Performance evaluation of bias correction methods for climate change monthly precipitation projections over costa rica. Water, 12.10.3390/w12020482 ↗
38. Heo, J.-H., Ahn, H., Shin, J.-Y., Kjeldsen, T.R., and Jeong, C. (2019). Probability distributions for a quantile mapping technique for a bias correction of precipitation data: A case study to precipitation data under climate change. Water, 11.10.3390/w11071475 ↗
39. Soriano, E., Mediero, L., and Garijo, C. (2019). Selection of bias correction methods to assess the impact of climate change on flood frequency curves. Water, 11.10.3390/w11112266 ↗
40. Kaltenpoth, D., and Vreeken, J. (2023, January 7). Identifying selection bias from observational data. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.10.1609/aaai.v37i7.25987 ↗
41. A workflow to address pitfalls and challenges in applying machine learning models to hydrology10.1016/j.advwatres.2021.103920 ↗
42. Shen, Z., Cui, P., Kuang, K., Li, B., and Chen, P. (2018, January 22). Causally regularized learning with agnostic data selection bias. Proceedings of the 26th ACM International Conference on Multimedia, Seoul, Republic of Korea.10.1145/3240508.3240577 ↗
43. Bibi, S., and Shin, J. (2022, January 21). Detection of Face Features using Adapted Triplet Loss with Biased data. Proceedings of the 2022 IEEE International Conference on Imaging Systems and Techniques (IST), Virtual.10.1109/IST55454.2022.9827674 ↗
44. Hierarchical bias mitigation for semi-supervised medical image classification10.1109/TMI.2023.3247440 ↗
45. Wu, P., Xu, T., and Wang, Y. (2019, January 5). Learning Personalized Treatment Rules from Electronic Health Records Using Topic Modeling Feature Extraction. Proceedings of the 2019 IEEE International Conference on Data Science and Advanced Analytics (DSAA), Washington, DC, USA.10.1109/DSAA.2019.00054 ↗
46. VAP risk index: Early prediction and hospital phenotyping of ventilator-associated pneumonia using machine learning10.1016/j.artmed.2023.102715 ↗
47. Out-of-distribution generalization with causal feature separation10.1109/TKDE.2023.3312255 ↗
48. Counterfactual can be strong in medical question and answering10.1016/j.ipm.2023.103408 ↗
49. Using conventional and machine learning propensity score methods to examine the effectiveness of 12-step group involvement following inpatient addiction treatment10.1016/j.drugalcdep.2021.108943 ↗
50. Deriving user preferences of mobile apps from their management activities
51. Minatel, D., Parmezan, A.R., Cúri, M., and Lopes, A.D.A. (2023, January 15). Fairness-Aware Model Selection Using Differential Item Functioning. Proceedings of the 2023 International Conference on Machine Learning and Applications (ICMLA), Jacksonville, FL, USA.10.1109/ICMLA58977.2023.00298 ↗
52. Dost, K., Taskova, K., Riddle, P., and Wicker, J. (2020, January 17). Your best guess when you know nothing: Identification and mitigation of selection bias. Proceedings of the 2020 IEEE International Conference on Data Mining (ICDM), Sorrento, Italy.10.1109/ICDM50108.2020.00115 ↗
53. (2024, February 15). GitHub—Imitate. Available online: https://github.com/KatDost/Imitate.
54. Dost, K., Duncanson, H., Ziogas, I., Riddle, P., and Wicker, J. (2022, January 11). Divide and imitate: Multi-cluster identification and mitigation of selection bias. Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining, Chengdu, China.10.1007/978-3-031-05936-0_12 ↗
55. Selection bias mitigation in recommender system using uninteresting items based on temporal visibility10.1016/j.eswa.2022.118932 ↗
56. Liu, H. (2022, January 25). Rating distribution calibration for selection bias mitigation in recommendations. Proceedings of the ACM Web Conference, Lyon, France.10.1145/3485447.3512078 ↗
57. Liu, F., Cole, J., Eisenschlos, J.M., and Collier, N. (2024, February 15). Are Ever Larger Octopi Still Influenced by Reporting Biases?. Available online: https://research.google/pubs/are-ever-larger-octopi-still-influenced-by-reporting-biases/.
58. Shwartz, V., and Choi, Y. (2020, January 8). Do neural language models overcome reporting bias?. Proceedings of the 28th International Conference on Computational Linguistics, Virtual.10.18653/v1/2020.coling-main.605 ↗
59. Mitigating Reporting Bias in Semi-supervised Temporal Commonsense Inference with Probabilistic Soft Logic
60. Wu, Q., Zhao, M., He, Y., Huang, L., Ono, J., Wakaki, H., and Mitsufuji, Y. (2023). Towards reporting bias in visual-language datasets: Bimodal augmentation by decoupling object-attribute association. arXiv.
61. Chiou, M.J., Ding, H., Yan, H., Wang, C., Zimmermann, R., and Feng, J. (2021, January 17). Recovering the unbiased scene graphs from the biased ones. Proceedings of the 29th ACM International Conference on Multimedia, Virtual.10.1145/3474085.3475297 ↗
62. Misra, I., Lawrence Zitnick, C., Mitchell, M., and Girshick, R. (2016, January 27). Seeing through the human reporting bias: Visual classifiers from noisy human-centric labels. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.320 ↗
63. Atay, M., Gipson, H., Gwyn, T., and Roy, K. (2021, January 5). Evaluation of gender bias in facial recognition with traditional machine learning algorithms. Proceedings of the 2021 IEEE Symposium Series on Computational Intelligence (SSCI), Orlando, FL, USA.10.1109/SSCI50451.2021.9660186 ↗
64. Ayoade, G., Chandra, S., Khan, L., Hamlen, K., and Thuraisingham, B. (2018, January 12). Automated threat report classification over multi-source data. Proceedings of the 2018 IEEE 4th International Conference on Collaboration and Internet Computing (CIC), Vancouver, BC, Canada.10.1109/CIC.2018.00040 ↗
65. Robust intelligent malware detection using deep learning10.1109/ACCESS.2019.2906934 ↗
66. Hinchliffe, C., Rehman, R.Z.U., Branco, D., Jackson, D., Ahmaniemi, T., Guerreiro, T., Chatterjee, M., Manyakov, N.V., Pandis, I., and Davies, K. (2023, January 24). Identification of Fatigue and Sleepiness in Immune and Neurodegenerative Disorders from Measures of Real-World Gait Variability. Proceedings of the 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Sydney, Australia.10.1109/EMBC40787.2023.10339956 ↗
67. Make it or break it: On-time vaccination intent at the time of Covid-1910.1016/j.vaccine.2023.02.014 ↗
68. Predicting opioid misuse at the population level is different from identifying opioid misuse in individual patients10.1016/j.ypmed.2019.105969 ↗
69. Catania, B., Guerrini, G., and Janpih, Z. (2023, January 15). Mitigating Representation Bias in Data Transformations: A Constraint-based Optimization Approach. Proceedings of the 2023 IEEE International Conference on Big Data (BigData), Sorrento, Italy.10.1109/BigData59044.2023.10386181 ↗
70. Hu, Q., and Rangwala, H. (2020, January 17). Metric-free individual fairness with cooperative contextual bandits. Proceedings of the 2020 IEEE International Conference on Data Mining (ICDM), Sorrento, Italy.10.1109/ICDM50108.2020.00027 ↗
71. Rengasamy, D., Mase, J.M., Rothwell, B., and Figueredo, G.P. (2019, January 27). An intelligent toolkit for benchmarking data-driven aerospace prognostics. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.10.1109/ITSC.2019.8917115 ↗
72. Bosco: Boosting corrections for genome-wide association studies with imbalanced samples10.1109/TNB.2017.2660498 ↗
73. Tiwari, V., and Verma, M. (2023, January 23). Prediction Of Groundwater Level Using Advance Machine Learning Techniques. Proceedings of the 2023 3rd International Conference on Intelligent Technologies (CONIT), Hubali, India.10.1109/CONIT59222.2023.10205583 ↗
74. Behfar, S.K. (2023, January 13). Decentralized intelligence and big data analytics reciprocal relationship. Proceedings of the 2023 Fifth International Conference on Blockchain Computing and Applications (BCCA), Bristol, UK.10.1109/BCCA58897.2023.10338888 ↗
75. Designing a Model for Financial Reporting Bias
76. Al-Sarraj, W.F., and Lubbad, H.M. (2018, January 28). Bias detection of Palestinian/Israeli conflict in western media: A sentiment analysis experimental study. Proceedings of the 2018 International Conference on Promising Electronic Technologies (ICPET), Hyderabad, India.10.1109/ICPET.2018.00024 ↗
77. Shumway, R.H., Stoffer, D.S., Shumway, R.H., and Stoffer, D.S. (2024, February 24). ARIMA Models. Time Series Analysis and Its Applications: With R Examples. Available online: https://link.springer.com/book/9783031705830.
78. Salleh, M.N.M., Talpur, N., and Hussain, K. (2017, January 27). Adaptive neuro-fuzzy inference system: Overview, strengths, limitations, and solutions. Proceedings of the Data Mining and Big Data: Second International Conference, Fukuoka, Japan.10.1007/978-3-319-61845-6_52 ↗
79. Teodorović, D. (2009). Bee colony optimization (BCO). Innovations in Swarm Intelligence, Springer.10.1007/978-3-642-04225-6_3 ↗
80. Siami-Namini, S., Tavakoli, N., and Namin, A.S. (2019, January 9). The performance of LSTM and BiLSTM in forecasting time series. Proceedings of the 2019 IEEE International Conference on Big Data (Big Data), Los Angeles, CA, USA.10.1109/BigData47090.2019.9005997 ↗
81. Kramer, O. (2015). Cascade support vector machines with dimensionality reduction. Applied Computational Intelligence and Soft Computing, Wiley Online Library.10.1155/2015/216132 ↗
82. Efficient C4. 5 [classification algorithm]10.1109/69.991727 ↗
83. Lewis, R.J. (2002). An introduction to classification and regression tree (CART) analysis. Annual Meeting of the Society for Academic Emergency Medicine, Department of Emergency Medicine Harbor-UCLA Medical Center Torrance.
84. A CNN-BiLSTM-AM method for stock price prediction10.1007/s00521-020-05532-z ↗
85. Wallach, H.M. (2004). Conditional Random Fields: An Introduction, CIS.
86. Mustaqeem, K.S. (2020). CLSTM: Deep feature-based speech emotion recognition using the hierarchical ConvLSTM network. Mathematics, 8.10.3390/math8122133 ↗
87. A survey of convolutional neural networks: Analysis, applications, and prospects10.1109/TNNLS.2021.3084827 ↗
88. Proportional hazards model: A review10.1016/0951-8320(94)90010-8 ↗
89. Kuhn, M., Weston, S., Keefer, C., and Coulter, N. (2024, February 20). Cubist Models for Regression. R Package Vignette R Package Version 0.0. Available online: https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=fd880d2b4482fc9b383435d51f6d730c02e0be36.
90. Decision tree methods: Applications for classification and prediction
91. Canziani, A., Paszke, A., and Culurciello, E. (2016). An analysis of deep neural network models for practical applications. arXiv.
92. Brim, A. (2020, January 6). Deep reinforcement learning pairs trading with a double deep Q-network. Proceedings of the 2020 10th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA.10.1109/CCWC47524.2020.9031159 ↗
93. Exponential smoothing: The state of the art—Part II10.1016/j.ijforecast.2006.03.005 ↗
94. Extremely randomized trees10.1007/s10994-006-6226-1 ↗
95. Chen, T., and Guestrin, C. (2016, January 13). Xgboost: A scalable tree boosting system. Proceedings of the 22nd acm Sigkdd International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939785 ↗
96. Gene expression programming: A survey10.1109/MCI.2017.2708618 ↗
97. Prettenhofer, P., and Louppe, G. (2014, January 21–23). Gradient boosted regression trees in scikit-learn. Proceedings of the PyData, London, UK.
98. Dey, R., and Salem, F.M. (2017). Gate-variants of gated recurrent unit (GRU) neural networks. 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS), IEEE.10.1109/MWSCAS.2017.8053243 ↗
99. Guo, G., Wang, H., Bell, D., Bi, Y., and Greer, K. (2003, January 3). KNN model-based approach in classification. Proceedings of the On the Move to Meaningful Internet Systems 2003: CoopIS, DOA, and ODBASE: OTM Confederated International Conferences, CoopIS, DOA, and ODBASE 2003, Catania, Italy.10.1007/978-3-540-39964-3_62 ↗
100. Light Gradient Boosting Machine: An efficient soft computing model for estimating daily reference evapotranspiration with local and external meteorological data10.1016/j.agwat.2019.105758 ↗