A Methodology for Controlling Bias and Fairness in Synthetic Data Generation
Source abstract
Abstract
The development of algorithms, based on machine learning techniques, supporting (or even replacing) human judgment must take into account concepts such as data bias and fairness. Though scientific literature proposes numerous techniques to detect and evaluate these problems, less attention has been dedicated to methods generating intentionally biased datasets, which could be used by data scientists to develop and validate unbiased and fair decision-making algorithms. To this end, this paper presents a novel method to generate a synthetic dataset, where bias can be modeled by using a probabilistic network exploiting structural equation modeling. The proposed methodology has been validated on a simple dataset to highlight the impact of tuning parameters on bias and fairness, as well as on a more realistic example based on a loan approval status dataset. In particular, this methodology requires a limited number of parameters compared to other techniques for generating datasets with a controlled amount of bias and fairness.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Explainable Artificial Intelligence (XAI) · Ethics and Social Impacts of AI · Adversarial Robustness in Machine Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Mathematical, Physical and Natural Sciences, Catholic University of the Sacred Heart, 25121 Brescia, Italy
Department of Management, Information and Production Engineering, University of Bergamo, 24129 Bergamo, Italy
Funding
Catholic University of the Sacred Heart
References (24)
Show source-provided references
1. Roskill, J. (2022, April 13). Why Digital Transformation Is an Ongoing Journey. Available online: https://www.forbes.com/sites/forbestechcouncil/2021/11/10/why-digital-transformation-is-an-ongoing-journey/?sh=672f83985bb9.
2. Su, B. (2022, April 13). 5 Key Factors Holding Small Businesses Back from Joining the “Data Revolution”. Available online: https://medium.com/analytics-for-humans/5-key-factors-holding-small-businesses-back-from-joining-the-data-revolution-6b95618deb7f.
3. (2022, April 13). Expert Panel, 13 Strategies For Collecting High-Quality Data. Available online: https://www.forbes.com/sites/forbescommunicationscouncil/2020/11/17/13-strategies-for-collecting-high-quality-data/?sh=6a6a5b763f1d.
4. Bias in Computer Systems10.1145/230538.230561 ↗
5. Xu, D., Yuan, S., Zhang, L., and Wu, X. (2018, January 10–14). FairGAN: Fairness-aware Generative Adversarial Networks. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA.10.1109/BigData.2018.8622525 ↗
6. A Survey on Bias and Fairness in Machine Learning10.1145/3457607 ↗
7. Explainable Deep Learning: A Field Guide for the Uninitiated
8. Data Pre-Processing Techniques for Classification without Discrimination10.1007/s10115-011-0463-8 ↗
9. Dahmen, J., and Cook, D. (2019). SynSys: A Synthetic Data Generation System for Healthcare Applications. Sensors, 19.10.3390/s19051181 ↗
10. Wang, T., Wu, D.J., Coates, A., and Ng, A.Y. (2012, January 11–15). End-to-end text recognition with convolutional neural networks. Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), Tsukuba, Japan.
11. Jaderberg, M., Simonyan, K., Vedaldi, A., and Zisserman, A. (2014, January 8–13). Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition. Proceedings of the Workshop on Deep Learning, NIPS, Montreal, QC, Canada.
12. Frid-Adar, M., Klang, E., Amitai, M., Goldberger, J., and Greenspan, H. (2018, January 4–7). Synthetic data augmentation using GAN for improved liver lesion classification. Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington, DC, USA.10.1109/ISBI.2018.8363576 ↗
13. Jaipuria, N., Zhang, X., Bhasin, R., Arafa, M., Chakravarty, P., Shrivastava, S., Manglani, S., and Murali, V.N. (2020, January 14–19). Deflating Dataset Bias Using Synthetic Data Augmentation. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA.10.1109/CVPRW50498.2020.00394 ↗
14. Arvanitis, T.N., White, S., Harrison, S., Chaplin, R., and Despotou, G. (2021). A Method for Machine Learning Generation of Realistic Synthetic Datasets for Validating Healthcare Applications. medRxiv.10.1101/2021.02.11.21250741 ↗
15. Lou, Y., Yuen, S.Y., and Chen, G. Evolving Benchmark Functions Using Kruskal-Wallis Test. Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO’18), Kyoto, Japan, 15–19 July 2018.10.1145/3205651.3208257 ↗
16. On constructing alternative benchmark suite for evolutionary algorithms10.1016/j.swevo.2018.04.005 ↗
17. Shand, C., Allmendinger, R., Handl, J., Webb, A., and Keane, J. Evolving Controllably Difficult Datasets for Clustering. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’19), Prague, Czech Republic, 13–17 July 2019.10.1145/3321707.3321761 ↗
18. MacKay, D.J.C. (2003). Information Theory, Inference, and Learning Algorithms, Cambridge University Press.
19. Kline, R. (2005). Principles and Practice of Structural Equation Modeling, Guilford Publications. [2nd ed.]. Methodology in the Social Sciences.
20. Gentle, J.E. (2002). Elements of Computational Statistics, Springer. Statistics and Computing.
21. Murphy, K.P. (2012). Machine Learning: A Probabilistic Perspective, The MIT Press.
22. Nicholson, A., and Jitnah, N. (1998, January 7–11). Using mutual information to determine relevance in Bayesian networks. Proceedings of the Pacific Rim International Conference on Artificial Intelligence, Guilin, China.10.1007/BFb0095287 ↗
23. Array programming with NumPy10.1038/s41586-020-2649-2 ↗
24. Scikit-Learn: Machine Learning in Python