Addressing Memorization and Aggregation Risks in AI: A Knowledge Graph Approach to Privacy
Source abstract
Abstract
Recent studies have shown that AI models can memorize specific data records, resulting in sensitive data exposure through model access. Current privacy-enhancing technologies often overlook the crucial, context-dependent nature of privacy risk as they largely fail to account for the inherent relationships and complex interactions between data records, leading to high risks associated with memorization and potential data aggregation. Our research first investigates two key factors influencing AI privacy risks: implicit connections and data redundancy. These experiments have shown that AI models learn subtle links between private data, even when they are discretely distributed. To address the privacy issue, we introduce PrivGraph, a hierarchically structured knowledge graph for modeling and aggregating private information. Based on PrivGraph, we introduce the Sensitivity Level Factor (SLF) to quantify the degree to which an individual’s private information is embedded in the data. In addition, we propose a PrivGraph-based knowledge probing method to facilitate post-training privacy assessments. Our experiments demonstrated that PrivGraph achieves comparable performance to existing models in the Personally Identifiable Information (PII) detection task, while effectively modeling the aggregation of private information even with lengthy texts and data obtained from multiple origins. Finally, we discuss PrivGraph’s integration into the AI engineering lifecycle for full-spectrum, full-lifecycle, and traceable privacy protection.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Privacy-Preserving Technologies in Data · Advanced Graph Neural Networks · Privacy, Security, and Data Protection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Artificial Intelligence, Ajou University, Suwon-si 16499, Republic of Korea
Department of Software and Computer Engineering, Ajou University, Suwon-si 16499, Republic of Korea
Funding
BK21 FOUR program of the National Research Foundation of Korea · Institute of Information & Communications Technology Planning & Evaluation (IITP) under the Artificial Intelligence Convergence Innovation Human Resources Development
References (46)
Show source-provided references
1. Deep Learning10.1038/nature14539 ↗
2. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics.
3. Wu, S., Fei, H., Qu, L., Ji, W., and Chua, T.-S. (2024, January 21–27). Next-GPT: Any-to-Any Multimodal LLM. Proceedings of the 41st International Conference on Machine Learning, Vienna, Austria.
4. Islam, S.B., Rahman, M.A., Hossain, K.T., Hoque, E., Joty, S., and Parvez, M.R. (2024). Open-RAG: Enhanced Retrieval Augmented Reasoning with Open-Source Large Language Models. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024), Miami, FL, USA, 12–16 November 2024, Association for Computational Linguistics.10.18653/v1/2024.findings-emnlp.831 ↗
5. Targ, S., Almeida, D., and Lyman, K. (2016). Resnet in Resnet: Generalizing Residual Architectures. arXiv.
6. Carlini, N., Tramèr, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., and Erlingsson, Ú. (2021). Extracting Training Data from Large Language Models. Proceedings of the 30th USENIX Security Symposium, Vancouver, BC, Canada, 11–13 August 2021, USENIX Association.
7. Shokri, R., Stronati, M., Song, C., and Shmatikov, V. (2017, January 22–24). Membership Inference Attacks Against Machine Learning Models. Proceedings of the 2017 IEEE Symposium on Security and Privacy (SP), San Jose, CA, USA.10.1109/SP.2017.41 ↗
8. Rahman, M.M., Arshi, A.S., Hasan, M.M., Mishu, S.F., Shahriar, H., and Wu, F. (2023, January 26–30). Security Risk and Attacks in AI: A Survey of Security and Privacy. Proceedings of the 2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC), Torino, Italy.10.1109/COMPSAC57700.2023.00284 ↗
9. Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., and Miller, A. (2019, January 3–7). Language Models as Knowledge Bases?. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.10.18653/v1/D19-1250 ↗
10. OECD.AI (2025, March 13). AI Principles. Available online: https://oecd.ai/en/ai-principles.
11. Privacy-preserving explainable AI: A survey10.1007/s11432-024-4123-4 ↗
12. Tesfay, W.B., Serna, J., and Rannenberg, K. (2019, January 22–25). Privacybot: Detecting Privacy Sensitive Information in Unstructured Texts. Proceedings of the 6th International Conference on Social Networks Analysis, Management and Security (SNAMS), Granada, Spain.10.1109/SNAMS.2019.8931855 ↗
13. Yermilov, O., Raheja, V., and Chernodub, A. (2023). Privacy- and Utility-Preserving NLP with Anonymized Data: A Case Study of Pseudonymization. arXiv.10.18653/v1/2023.trustnlp-1.20 ↗
14. Hassanpour, A., Utsash, M.M., and Yang, B. (2020). The Impact of Linkability on Privacy Leakage. Proceedings of the International Conference on Advances in Social Networks Analysis and Mining (ASONAM), Istanbul, Turkey, 10–12 November 2022, ACM.
15. Lukas, N., Ahmed, S.C., Li, B., Salem, K., and Kerschbaum, F. (2023, January 21–25). Analyzing Leakage of Personally Identifiable Information in Language Models. Proceedings of the 2023 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA.10.1109/SP46215.2023.10179300 ↗
16. The Privacy Issue of Counterfactual Explanations: Explanation Linkage Attacks10.1145/3608482 ↗
17. A Survey of Privacy Attacks in Machine Learning10.1145/3624010 ↗
18. Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., and Song, D. (2019). The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks. Proceedings of the 28th USENIX Security Symposium, Santa Clara, CA, USA, 14–16 August 2019, USENIX Association.
19. Knowledge Editing for Large Language Models: A Survey
20. Liu, Z., Huang, Y., Wen, Y., Zhao, J., and Lin, J. (2024). Breaking the Trilemma of Privacy, Utility, and Efficiency via Controllable Machine Unlearning. Proceedings of the ACM Web Conference 2024 (WWW ’24), Singapore, 13–17 May 2024, Association for Computing Machinery.10.1145/3589334.3645669 ↗
21. More Than Privacy: Applying Differential Privacy in Key Areas of Artificial Intelligence10.1109/TKDE.2020.3014246 ↗
22. Shumailov, I., Shumaylov, Z., Zhao, Y., Gal, Y., Papernot, N., and Anderson, R. (2024). Ununlearning: Unlearning Is Not Sufficient for Content Regulation in Advanced Generative AI. arXiv.
23. Zhang, Z., Wang, F., Li, X., Wu, Z., Tang, X., Liu, H., He, Q., Yin, W., and Wang, S. (2025, January 24–28). Catastrophic failure of LLM unlearning via quantization. Proceedings of the 13th International Conference on Learning Representations (ICLR), Singapore.
24. Privacy BERT-LSTM: A Novel NLP Algorithm for Sensitive Information Detection in Textual Documents10.1007/s00521-024-09707-w ↗
25. Brown, C., and Morisset, C. (2022, January 17–20). Simple and Efficient Identification of Personally Identifiable Information on a Public Website. Proceedings of the 2022 IEEE International Conference on Big Data (Big Data), Osaka, Japan.10.1109/BigData55660.2022.10020584 ↗
26. PII-Codex: A Python Library for PII Detection, Categorization, and Severity Assessment10.21105/joss.05402 ↗
27. Gambarelli, G., and Gangemi, A. (2022). PRIVAFRAME: A Frame-Based Knowledge Graph for Sensitive Personal Data. Big Data Cogn. Comput., 6.10.3390/bdcc6030090 ↗
28. Huo, L., and Jiang, J. (2023). Research on Intelligent Perception Algorithm for Sensitive Information. Appl. Sci., 13.10.3390/app13063383 ↗
29. Petroni, F., Rocktäschel, T., Lewis, P., Bakhtin, A., Wu, Y., Miller, A.H., and Riedel, S. (2019, January 3–7). Language Models as Knowledge Bases?. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), Hong Kong, China.10.18653/v1/D19-1250 ↗
30. Propile: Probing Privacy Leakage in Large Language Models
31. Zuo, J., and Lee, S.-W. (2025, January 16–20). PrivGraph: Modeling personal privacy information using knowledge graph. Proceedings of the 2025 25th International Conference on Software Quality, Reliability, and Security (QRS), Hangzhou, China.10.1109/QRS65678.2025.00080 ↗
32. The PII Problem: Privacy and a New Concept of Personally Identifiable Information
33. What Are Data? A Categorization of the Data Sensitivity Spectrum10.1016/j.bdr.2017.11.001 ↗
34. European Commission (2016). Regulation
35. (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data (General Data Protection Regulation). Off. J. Eur. Union, L119, 1-88.
36. U.S. Government (1996). Health Insurance Portability and Accountability Act of 1996 (HIPAA).
37. European Commission (2024). Regulation
38. (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act). Off. J. Eur. Union, L, 1-81.
39. A Survey on Differential Privacy for Unstructured Data Content10.1145/3490237 ↗
40. Stadler, T., Oprisanu, B., and Troncoso, C. (2022). Synthetic Data–Anonymisation Groundhog Day. Proceedings of the 31st USENIX Security Symposium, Boston, MA, USA, 10–12 August 2022, USENIX Association.
41. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter. arXiv.
42. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. (2019). Language Models Are Unsupervised Multitask Learners, OpenAI. OpenAI Technical Report.
43. Explosion (2025, March 13). spaCy: Industrial-Strength Natural Language Processing. Available online: https://spacy.io/.
44. Otmazgin, S., Cattan, A., and Goldberg, Y. (2022, January 20–23). F-coref: Fast, Accurate and Easy to Use Coreference Resolution. Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing: System Demonstrations, Taipei, Taiwan.10.18653/v1/2022.aacl-demo.6 ↗
45. Zhong, Z., and Chen, D. (2021). A Frustratingly Easy Approach for Entity and Relation Extraction. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2021), Online, 6–11 June 2021, Association for Computational Linguistics.10.18653/v1/2021.naacl-main.5 ↗
46. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks