Diagnosing and Mitigating LLM Failures in Recognizing Culturally Specific Korean Names: An Error-Driven Prompting Framework
Source abstract
Abstract
As large language models (LLMs) improve in understanding and reasoning, they are increasingly used in privacy protection tasks such as de-identification, privacy-sensitive text generation, and entity obfuscation. However, these applications depend on an essential requirement: the accurate identification of personally identifiable information (PII). Compared with template-based PII that follows clear structural patterns, name-related PII depends much more on cultural and pragmatic context, which makes it harder for models to detect and raises higher privacy risks. Although recent studies begin to address this issue, existing work remains limited in language coverage, evaluation granularity, and the depth of error analysis. To address these gaps, this study proposes an error-driven framework that integrates diagnosis and intervention. Specifically, the framework introduces a method called Error-Driven Prompt (EDP), which transforms common failure patterns into executable prompting strategies. It further explores the integration of EDP with general advanced prompting techniques such as Chain-of-Thought (CoT), few-shot learning, and role-playing. In addition, the study constructed K-NameDiag, the first fine-grained evaluation benchmark for Korean name-related PII, which includes twelve culturally sensitive subtypes designed to examine model weaknesses in real-world contexts. The experimental results showed that EDP improved F1-scores in the range of 6 to 9 points across three widely used commercial LLMs, namely Claude Sonnet 4.5, GPT-5, and Gemini 2.5 Pro, while the Combined Enhanced Prompt (CEP), which integrates EDP with advanced prompting strategies, resulted in different shifts in precision and recall rather than consistent improvements. Further subtype-level analysis suggests that subtypes reliant on implicit cultural context remain resistant to correction, which shows the limitations of prompt engineering in addressing a model’s lack of internalized cultural knowledge.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Authorship Attribution and Profiling · Computational and Text Analysis Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Interdisciplinary Graduate Program of Linguistics and Informatics, Yonsei University, Seoul 03722, Republic of Korea
Tscientific Company Ltd., Seoul 06900, Republic of Korea
Funding
Korea Internet & Security Agency
References (52)
Show source-provided references
1. Li, Y., Pei, Q., Sun, M., Lin, H., Ming, C., Gao, X., Wu, J., He, C., and Wu, L. (August, January 27). CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenge. Proceedings of the Findings of the Association for Computational Linguistics: ACL 2025, Vienna, Austria.10.18653/v1/2025.findings-acl.309 ↗
2. Sheikhaei, M.S., Tian, Y., Wang, S., and Xu, B. (2024). An Empirical Study on the Effectiveness of Large Language Models for SATD Identification and Classification. arXiv.10.1007/s10664-024-10548-3 ↗
3. Deußer, T., Sparrenberg, L., Berger, A., Hahnbück, M., Bauckhage, C., and Sifa, R. (2025). A Survey on Current Trends and Recent Advances in Text Anonymization. arXiv.10.1109/DSAA65442.2025.11247969 ↗
4. Hui, B.S.H., Miao, X., and Wang, X. (2025). SecureSpeech: Prompt-based Speaker and Content Protection. arXiv.
5. KDPII: A new korean dialogic dataset for the deidentification of personally identifiable information10.1109/ACCESS.2024.3461804 ↗
6. Pham, D., Kairouz, P., Mireshghallah, N., Bagdasarian, E., Pham, C.M., and Houmansadr, A. (2025). Can Large Language Models Really Recognize Your Name?. arXiv.
7. Sang, E.F.T.K., and De Meulder, F. (June, January 31). Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, Edmonton, AB, Canada.
8. Chinchor, N., and Robinson, P. (–1, January 29). MUC-7 named entity task definition. Proceedings of the 7th Conference on Message Understanding, Fairfax, VA, USA.
9. Annotating longitudinal clinical narratives for de-identification: The 2014 i2b2/UTHealth corpus10.1016/j.jbi.2015.07.020 ↗
10. Singh, D., and Narayanan, S. (2025). Unmasking the Reality of PII Masking Models: Performance Gaps and the Call for Accountability. arXiv.
11. Shen, H., Gu, Z., Hong, H., and Han, W. (2025). PII-Bench: Evaluating Query-Aware Privacy Protection Systems. arXiv.
12. Kocetkov, D., Li, R., Allal, L.B., Li, J., Mou, C., Ferrandis, C.M., Jernite, Y., Mitchell, M., Hughes, S., and Wolf, T. (2022). The stack: 3 tb of permissively licensed source code. arXiv.
13. Asthana, S., Mahindru, R., Zhang, B., and Sanz, J. (2025). Adaptive PII Mitigation Framework for Large Language Models. arXiv.
14. Savkin, M., Ionov, T., and Konovalov, V. (May, January 30). SPY: Enhancing Privacy with Synthetic PII Detection Dataset. Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Student Research Workshop), Albuquerque, NM, USA.10.18653/v1/2025.naacl-srw.23 ↗
15. Bikel, D.M., Miller, S., Schwartz, R., and Weischedel, R. (1998). Nymble: A high-performance learning name-finder. arXiv.10.3115/974557.974586 ↗
16. Isozaki, H., and Kazawa, H. (September, January 24). Efficient support vector classifiers for named entity recognition. Proceedings of the COLING 2002: The 19th International Conference on Computational Linguistics, Taipei, Taiwan.10.3115/1072228.1072282 ↗
17. Lafferty, J., McCallum, A., and Pereira, F.C. (July, January 28). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. Proceedings of the 18th International Conference on Machine Learning (ICML), Williamstown, MA, USA.
18. McCallum, A., and Li, W. (2003, January 31). Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons. Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, Edmonton, AB, Canada.10.3115/1119176.1119206 ↗
19. Settles, B. (2004, January 28–29). Biomedical Named Entity Recognition using Conditional Random Fields and Rich Feature Sets. Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and Its Applications (NLPBA/BioNLP), Geneva, Switzerland.10.3115/1567594.1567618 ↗
20. Tomás, J., Rasteiro, D., and Bernardino, J. (2022). Data anonymization: An experimental evaluation using open-source tools. Future Internet, 14.10.3390/fi14060167 ↗
21. Yang, J., Zhang, X., Liang, K., and Liu, Y. (2023, January 15–18). Exploring the Application of Large Language Models in Detecting and Protecting Personally Identifiable Information in Archival Data: A Comprehensive Study*. Proceedings of the 2023 IEEE International Conference on Big Data (BigData), Sorrento, Italy.10.1109/BigData59044.2023.10386949 ↗
22. Ratinov, L., and Roth, D. (2009, January 4). Design Challenges and Misconceptions in Named Entity Recognition. Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL-2009), Boulder, CO, USA.10.3115/1596374.1596399 ↗
23. Huang, Z., Xu, W., and Yu, K. (2015). Bidirectional LSTM-CRF models for sequence tagging. arXiv.
24. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., and Dyer, C. (2016). Neural architectures for named entity recognition. arXiv.10.18653/v1/N16-1030 ↗
25. Ma, X., and Hovy, E. (2016). End-to-end sequence labeling via bi-directional lstm-cnns-crf. arXiv.10.18653/v1/P16-1101 ↗
26. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
27. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2–7). Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.
28. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv.
29. He, P., Liu, X., Gao, J., and Chen, W. (2020). Deberta: Decoding-enhanced bert with disentangled attention. arXiv.
30. A survey on deep learning for named entity recognition10.1109/TKDE.2020.2981314 ↗
31. Language models are few-shot learners
32. Wei, X., Cui, X., Cheng, N., Wang, X., Zhang, X., Huang, S., Xie, P., Xu, J., Chen, Y., and Zhang, M. (2023). Chatie: Zero-shot information extraction via chatting with chatgpt. arXiv.
33. Mainetti, L., and Elia, A. (2025). Detecting Personally Identifiable Information Through Natural Language Processing: A Step Forward. Appl. Syst. Innov., 8.10.3390/asi8020055 ↗
34. Nakka, K.K., Frikha, A., Mendes, R., Jiang, X., and Zhou, X. (2024). PII-Scope: A Comprehensive Study on Training Data PII Extraction Attacks in LLMs. arXiv.
35. M3exam: A multilingual, multimodal, multilevel benchmark for examining large language models
36. Chiu, Y.Y., Jiang, L., Lin, B.Y., Park, C.Y., Li, S.S., Ravi, S., Bhatia, M., Antoniak, M., Tsvetkov, Y., and Shwartz, V. (2024). CulturalBench: A Robust, Diverse and Challenging Benchmark on Measuring (the Lack of) Cultural Knowledge of LLMs. arXiv.10.18653/v1/2025.acl-long.1247 ↗
37. Shi, F., Suzgun, M., Freitag, M., Wang, X., Srivats, S., Vosoughi, S., Chung, H.W., Tay, Y., Ruder, S., and Zhou, D. (2022). Language models are multilingual chain-of-thought reasoners. arXiv.
38. AI Hub (2025, October 10). Korean SNS Multi-Turn Dialogue Data. Available online: https://www.aihub.or.kr/aihubdata/data/view.do?pageIndex=1&currMenu=115&topMenu=100&srchOptnCnd=OPTNCND001&searchKeyword=SNS&srchDetailCnd=DETAILCND001&srchOrder=ORDER001&srchPagePer=20&aihubDataSe=data&dataSetSn=71694.
39. Center for Science and Technology Security Research (2022). Network Intrusion Detection Dataset, Korea Institute of Science and Technology Security Research Center.
40. Park, S., Moon, J., Kim, S., Cho, W.I., Han, J., Park, J., Song, C., Kim, J., Song, Y., and Oh, T. (2021). Klue: Korean language understanding evaluation. arXiv.
41. Unleashing the potential of prompt engineering for large language models10.1016/j.patter.2025.101260 ↗
42. Prompting AI art: An investigation into the creative skill of prompt engineering10.1080/10447318.2024.2431761 ↗
43. Prompting meaning: A hermeneutic approach to optimising prompt engineering with ChatGPT10.1007/s00146-023-01752-8 ↗
44. Cost, Complexity, and Efficacy of Prompt Engineering Techniques for Large Language Models
45. Abdallah, A.E.M., Mozafari, J., Piryani, B., Abdelgwad, M.M., and Jatowt, A. (2025, January 19–24). DynRank: Improve passage retrieval with dynamic zero-shot prompting based on question classification. Proceedings of the 31st International Conference on Computational Linguistics, Abu Dhabi, United Arab Emirates.
46. Wu, J., Wang, X., and Jia, W. (2025, January 6–11). Enhancing text annotation through rationale-driven collaborative few-shot prompting. Proceedings of the ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Hyderabad, India.10.1109/ICASSP49660.2025.10889661 ↗
47. Zhang, Y., Wang, X., Wu, L., and Wang, J. (25–4, January 25). Enhancing chain of thought prompting in large language models via reasoning patterns. Proceedings of the AAAI Conference on Artificial Intelligence, Philadelphia, PA, USA.
48. Chen, Y.C., Lee, S.H., Sheu, H., Lin, S.H., Hu, C.C., Fu, S.C., Yang, C.P., and Lin, Y.C. (2025). Enhancing responses from large language models with role-playing prompts: A comparative study on answering frequently asked questions about total knee arthroplasty. BMC Med. Inform. Decis. Mak., 25.10.1186/s12911-025-03024-5 ↗
49. Li, J., Yraola, L., Zhu, K., and O’brien, S. (2025). Error Reflection Prompting: Can Large Language Models Successfully Understand Errors?. The Sixth Workshop on Insights from Negative Results in NLP, Association for Computational Linguistics.10.18653/v1/2025.insights-1.15 ↗
50. Self-refine: Iterative refinement with self-feedback
51. Reflexion: Language agents with verbal reinforcement learning
52. Islam, R., and Moushi, O.M. (2025). Gpt-4o: The cutting-edge advancement in multimodal llm. Intelligent Computing, Proceedings of the Computing Conference, London, UK, 19–20 June 2025, Springer.10.36227/techrxiv.171986596.65533294/v1 ↗