Automatic recognition of cross-language classic entities based on large language models

Abstract Large language models (LLMs) hold immense potential for the intelligent processing of classical texts. They offer new approaches for digital research on classical literature resources, cross-linguistic understanding, text knowledge mining, and the promotion and preservation of cultural heritage. To explore the performance of named entity recognition (NER) tasks supported by LLMs, this study first fine-tuned four LLMs—Xunzi-Baichuan, Baichuan2-7B-Base, Xunzi-GLM, and ChatGLM3-6B—using supervised fine-tuning methods based on open-source models. Zero-shot, one-shot, and few-shot prompting methods were then employed to validate the performance of these models in the NER tasks. Finally, the applicability of fine-tuning LLMs in specific domains for NER tasks was examined using BLEU-4, ROUGE-1, ROUGE-2, ROUGE-L, precision, recall, and F1 scores as evaluation metrics for model performance and applicability. The experimental results indicated that fine-tuned LLMs achieved high scores across multiple metrics, demonstrating strong performance in text generation. In entity extraction, the Xunzi-Baichuan model performed optimally across several metrics and also exhibited generalization capabilities. In addition, we have open-sourced our models for community research. https://github.com/Xunzi-LLM-of-Chinese-classics/XunziALLM.