FairRAG: A Privacy-Preserving Framework for Fair Financial Decision-Making
Source abstract
Abstract
Customer churn prediction has become crucial for businesses, yet it poses significant challenges regarding privacy preservation and prediction accuracy. In this paper, we address two fundamental questions: (1) How can customer churn be effectively predicted while ensuring robust privacy protection of sensitive data? (2) How can large language models enhance churn prediction accuracy while maintaining data privacy? To address these questions, we propose FairRAG, a robust architecture that combines differential privacy, retrieval-augmented generation, and LLMs. Our approach leverages OPT-125M as the core language model along with a sentence transformer for semantic similarity matching while incorporating differential privacy mechanisms to generate synthetic training data. We evaluate FairRAG on two diverse datasets: Bank Churn and Telco Churn. The results demonstrate significant improvements over both traditional machine learning approaches and standalone LLMs, achieving accuracy improvements of up to 11% on the Bank Churn dataset and 12% on the Telco Churn dataset. These improvements were maintained when using differentially private synthetic data, thus indicating robust privacy and accuracy trade-offs.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Privacy-Preserving Technologies in Data · Privacy, Security, and Data Protection · Customer churn and segmentation
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 32 Vassar St, Cambridge, MA 02139, USA
Cybersecurity at MIT Sloan (CAMS), Massachusetts Institute of Technology, 77 Massachusetts Ave, Cambridge, MA 02139, USA
Institute for Research in Technology, Universidad Pontificia Comillas, Alberto Aguilera 23, 28015 Madrid, Spain
AI Institute for Community-Engaged Research (AI-ICER), The University of Texas at El Paso, 500 West University Avenue, El Paso, TX 79968, USA
Funding
MIT Internal
References (62)
Show source-provided references
1. The algorithmic foundations of differential privacy10.1561/0400000042 ↗
2. A Survey on Bias and Fairness in Machine Learning10.1145/3457607 ↗
3. Cummings, R., Gupta, V., Kimpara, D., and Morgenstern, J. (2019, January 9–12). On the Compatibility of Privacy and Fairness. Proceedings of the Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization, UMAP’19 Adjunct, Larnaca, Cyprus.10.1145/3314183.3323847 ↗
4. Kearns, M., and Roth, A. (2020). Ethical Algorithm Design: Guiding Principles for Technology Regulation, Brookings Institution. Technical Report.10.1145/3440959.3440966 ↗
5. Zhang, H., Hartvigsen, T., and Ghassemi, M. (2025, July 03). Algorithmic Fairness in Chest X-ray Diagnosis: A Case Study. MIT Case Studies in Social and Ethical Responsibilities of Computing. Available online: https://mit-serc.pubpub.org/pub/algorithmic-chest.
6. The European Union general data protection regulation: What it is and what it means10.1080/13600834.2019.1573501 ↗
7. Understanding the scope and impact of the california consumer privacy act of 2018
8. Barocas, S., Hardt, M., and Narayanan, A. (2023). Fairness and Machine Learning: Limitations and Opportunities, MIT Press.
9. How much does racial bias affect mortgage lending? Evidence from human and algorithmic credit decisions10.1111/jofi.13444 ↗
10. Utilizing data sampling techniques on algorithmic fairness for customer churn prediction with data imbalance problems10.12688/f1000research.72929.2 ↗
11. Bourou, S., El Saer, A., Velivassaki, T.H., Voulkidis, A., and Zahariadis, T. (2021). A review of tabular data synthesis using GANs on an IDS dataset. Information, 12.10.3390/info12090375 ↗
12. When machine learning meets privacy: A survey and outlook
13. Liu, H., Wu, Z., Zhou, Y., Peng, C., Tian, F., and Lu, L. (2018). Privacy-Preserving Monotonicity of Differential Privacy Mechanisms. Appl. Sci., 8.10.3390/app8112081 ↗
14. Ghoukasian, H., and Asoodeh, S. (2024, January 7–12). Differentially private fair binary classifications. Proceedings of the 2024 IEEE International Symposium on Information Theory (ISIT), Athens, Greece.10.1109/ISIT57864.2024.10619147 ↗
15. Kurakin, A., Ponomareva, N., Syed, U., MacDermed, L., and Terzis, A. (2024). Harnessing large-language models to generate private synthetic text. arXiv.
16. Kalodanis, K., Papadopoulos, S., Feretzakis, G., Rizomiliotis, P., and Anagnostopoulos, D. (2025). SecureLLM: A Unified Framework for Privacy-Focused Large Language Models. Appl. Sci., 15.10.3390/app15084180 ↗
17. Chen, Y., Liu, Y., Yan, J., Bai, X., Zhong, M., Yang, Y., Yang, Z., Zhu, C., and Zhang, Y. (2024). See what llms cannot answer: A self-challenge framework for uncovering llm weaknesses. arXiv.
18. Gu, J., Jiang, X., Shi, Z., Tan, H., Zhai, X., Xu, C., Li, W., Shen, Y., Ma, S., and Liu, H. (2024). A survey on llm-as-a-judge. arXiv.
19. Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, H., and Wang, H. (2023). Retrieval-augmented generation for large language models: A survey. arXiv.
20. Repository, U.M.L. (2025, July 03). Bank Customer Churn Prediction Dataset. Available online: https://www.kaggle.com/datasets/mathchi/churn-for-bank-customers.
21. Corporation, I. (2025, June 05). Telco Customer Churn Dataset. Available online: https://www.ibm.com/docs/en/cognos-analytics/11.1.0?topic=samples-telco-customer-churn.
22. Veselovsky, V., Ribeiro, M.H., Arora, A., Josifoski, M., Anderson, A., and West, R. (2023). Generating faithful synthetic data with large language models: A case study in computational social science. arXiv.
23. Xie, C., Lin, Z., Backurs, A., Gopi, S., Yu, D., Inan, H.A., Nori, H., Jiang, H., Zhang, H., and Lee, Y.T. (2024). Differentially private synthetic data via foundation model apis 2: Text. arXiv.
24. Lin, Z., Gopi, S., Kulkarni, J., Nori, H., and Yekhanin, S. (2023). Differentially private synthetic data via foundation model apis 1: Images. arXiv.
25. Nasr, M., Shokri, R., and Houmansadr, A. (2020). Improving deep learning with differential privacy using gradient encoding and denoising. arXiv.
26. Li, J., Zhang, F., Guo, Y., Li, S., Wu, G., Li, D., and Zhu, H. (2023, January 24–26). A privacy-preserving online deep learning algorithm based on differential privacy. Proceedings of the 2023 26th International Conference on Computer Supported Cooperative Work in Design (CSCWD), Rio de Janeiro, Brazil.10.1109/CSCWD57460.2023.10152847 ↗
27. Dwork, C., McSherry, F., Nissim, K., and Smith, A. (2006, January 4–7). Calibrating noise to sensitivity in private data analysis. Proceedings of the Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA. Proceedings 3.10.1007/11681878_14 ↗
28. Triastcyn, A., and Faltings, B. (2020, January 13–18). Bayesian differential privacy for machine learning. Proceedings of the International Conference on Machine Learning, PMLR, Virtual.
29. Feedback-based architecture for reading courtesy amounts on checks10.1117/1.1526105 ↗
30. Mo, R., Liu, J., Yu, W., Jiang, F., Gu, X., Zhao, X., Liu, W., and Peng, J. (2019, January 5–8). A differential privacy-based protecting data preprocessing method for big data mining. Proceedings of the 2019 18th IEEE International Conference on Trust, Security and Privacy in Computing and Communications/13th IEEE International Conference on Big Data Science and Engineering (TrustCom/BigDataSE), Rotorua, New Zealand.10.1109/TrustCom/BigDataSE.2019.00098 ↗
31. Cummings, R., and Durfee, D. (2020, January 5–8). Individual sensitivity preprocessing for data privacy. Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, Salt Lake City, UT, USA.10.1137/1.9781611975994.32 ↗
32. Song, S., Chaudhuri, K., and Sarwate, A.D. (2013, January 3–5). Stochastic gradient descent with differentially private updates. Proceedings of the 2013 IEEE Global Conference on Signal and Information Processing, Austin, TX, USA.10.1109/GlobalSIP.2013.6736861 ↗
33. Abadi, M., Chu, A., Goodfellow, I., McMahan, H.B., Mironov, I., Talwar, K., and Zhang, L. (2016, January 24–28). Deep learning with differential privacy. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria.10.1145/2976749.2978318 ↗
34. Attack-Aware Noise Calibration for Differential Privacy
35. Privacy and Fairness Analysis in the Post-Processed Differential Privacy Framework10.1109/TIFS.2025.3528222 ↗
36. Generative adversarial networks for synthetic data generation in finance: Evaluating statistical similarities and quality assessment10.3390/ai5020035 ↗
37. Mukherjee, M., and Khushi, M. (2021). SMOTE-ENC: A novel SMOTE-based method to generate synthetic data for nominal and continuous features. Appl. Syst. Innov., 4.10.3390/asi4010018 ↗
38. Charitou, C., Dragicevic, S., and d’Avila Garcez, A. (2021). Synthetic Data Generation for Fraud Detection using GANs. arXiv.
39. Miletic, M., and Sariyar, M. (2024). Assessing the potentials of LLMs and GANs as state-of-the-art tabular synthetic data generation methods. International Conference on Privacy in Statistical Databases, Springer.10.1007/978-3-031-69651-0_25 ↗
40. Pawade, P., Kulkarni, M., Naik, S., Raut, A., and Wagh, K. (2024, January 5–7). Efficiency Comparison of Dataset Generated by LLMs using Machine Learning Algorithms. Proceedings of the 2024 International Conference on Emerging Smart Computing and Informatics (ESCI), Pune, India.10.1109/ESCI59607.2024.10497340 ↗
41. Privacy issues in large language models: A survey10.1016/j.compeleceng.2024.109698 ↗
42. Unveiling security, privacy, and ethical concerns of ChatGPT
43. Brown, H., Lee, K., Mireshghallah, F., Shokri, R., and Tramèr, F. (2022, January 21–24). What does it mean for a language model to preserve privacy?. Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, Seoul, Republic of Korea.10.1145/3531146.3534642 ↗
44. Chouldechova, A., and Roth, A. (2018). The Frontiers of Fairness in Machine Learning. arXiv.
45. Vajpayee, A.S., and Khobragade, D. (2024, January 29–30). The Problem of Data Bias in Healthcare AI. Proceedings of the 2024 2nd DMIHER International Conference on Artificial Intelligence in Healthcare, Education and Industry (IDICAIEI), Wardha, India.10.1109/IDICAIEI61867.2024.10842779 ↗
46. A Multi-Objective Framework for Balancing Fairness and Accuracy in Debiasing Machine Learning Models10.3390/make6030105 ↗
47. Bird, S., Dudík, M., Edgar, R., Horn, B., Lutz, R., Milan, V., Sameki, M., Wallach, H., and Walker, K. (2020). Fairlearn: A Toolkit for Assessing and Improving Fairness in AI, Microsoft. Technical Report MSR-TR-2020-32.
48. Friedler, S.A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E.P., and Roth, D. (2019, January 29–31). A comparative study of fairness-enhancing interventions in machine learning. Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* ’19, Atlanta, GA, USA.10.1145/3287560.3287589 ↗
49. Fairness in Criminal Justice Risk Assessments: The State of the Art10.1177/0049124118782533 ↗
50. A Reductions Approach to Fair Classification
51. Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., and Chen, Y. (2023). Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models. arXiv.
52. Bouamor, H., Pino, J., and Bali, K. (2023, January 6–10). Query Rewriting in Retrieval-Augmented Large Language Models. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore.
53. Auquan (2025, July 03). The Advantages of Retrieval-Augmented Generation (RAG) AI Over Generative AI for Financial Services. Available online: https://insights.auquan.com/the-advantages-of-retrieval-augmented-generation-rag-ai-over-generative-ai-for-financial-services-white-paper.
54. Huang, Y., Huang, B., and Kechadi, M.T. (2011, January 24–27). A rule-based method for customer churn prediction in telecommunication services. Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining, Shenzhen, China.10.1007/978-3-642-20841-6_34 ↗
55. An interpretable machine learning framework for customer churn prediction: A case study in the telecommunications industry
56. Fahmy, Y.G., Fakhr, M.W., and Kholief, M. (2024, January 14–16). Privacy Preserving Customer Churn Prediction using Deep and Federated Learning. Proceedings of the 2024 34th International Conference on Computer Theory and Applications (ICCTA), Alexandria, Egypt.10.1109/ICCTA64612.2024.10974881 ↗
57. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods10.1007/s10994-021-05946-3 ↗
58. Neal, R.M. (2012). Bayesian Learning for Neural Networks, Springer Science & Business Media.
59. Lakshminarayanan, B., Pritzel, A., and Blundell, C. (2017, January 4–9). Simple and scalable predictive uncertainty estimation using deep ensembles. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
60. Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., Schiefer, N., Jones, A., Joseph, N., and Mann, B. (2022). Language Models (Mostly) Know What They Know. arXiv.
61. Tang, Y., Fei, Z., Huang, L., Zhang, W., Zhao, B., Guan, H., and Huang, Y. (2025). Failure Mode and Effects Analysis Method on the Air System of an Aircraft Turbofan Engine in Multi-Criteria Open Group Decision-Making Environment. Cybern. Syst., 1–32.10.1080/01969722.2025.2468189 ↗
62. Lahitani, A.R., Permanasari, A.E., and Setiawan, N.A. (2016, January 26–27). Cosine similarity to determine similarity measure: Study case in online essay assessment. Proceedings of the 2016 4th International Conference on Cyber and IT Service Management, Bandung, Indonesia.10.1109/CITSM.2016.7577578 ↗