A Chinese Few-Shot Named-Entity Recognition Model Based on Multi-Label Prompts and Boundary Information
Source abstract
Abstract
Currently, few-shot setting and entity nesting are two major challenges in named-entity recognition (NER). Compared to English, Chinese NER not only has issues such as complex grammatical structures, polysemy, and entity nesting but also faces low-resource scenarios in specific domains due to difficulties in sample annotation. To address these two issues, we propose a Chinese few-shot named-entity recognition model that integrates multi-label prompts and boundary information (MPBCNER). This model is an improvement based on a pre-trained language model (PLM) combined with a pointer network. First, the model uses multiple entity label words and position slots as prompt information in the entity recognition training task. Activating the relevant parameters in PLM associated with the corresponding entity labels through the prompt information improved the model’s performance in entity recognition under small-sample data. Secondly, by using a Graph Attention Network (GAT) to construct the boundary information extraction module, we integrated boundary information with text features, allowing the model to pay more attention to features near the boundaries when recognizing entities, thereby improving the accuracy of entity boundary recognition. Experiments on multiple public small-sample datasets and our own annotated datasets in the field of government auditing demonstrated the effectiveness of this model.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Natural Language Processing Techniques · Text and Document Classification Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer, Electronic and Information, Guangxi University, Nanning 530004, China
Funding
Open Project Program of Guangxi Key Laboratory of Digital Infrastructure
References (48)
Show source-provided references
1. A Review of Chinese Named Entity Recognition
2. A Survey of Named Entity Recognition and Classification10.1075/li.30.1.03nad ↗
3. A Survey on Deep Learning for Named Entity Recognition10.1109/TKDE.2020.2981314 ↗
4. Lafferty, J., Mccallum, A., and Pereira, F. (July, January 28). Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. Proceedings of the 18th International Conference on Machine Learning, Williamstown, MA, USA.
5. Huang, Z., Xu, W., and Yu, K. (2015). Bidirectional LSTM-CRF Models for Sequence Tagging. arXiv.
6. Ma, X., and Hovy, E. (2016). End-to-End Sequence Labeling via Bi-Directional Lstm-Cnns-Crf. arXiv.10.18653/v1/P16-1101 ↗
7. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). Bert: Pre-Training of Deep Bidirectional Transformers for Language Understanding. arXiv.
8. Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2018). Improving Language Understanding by Generative Pre-Training, OpenAI.
9. Language Models Are Unsupervised Multitask Learners
10. Language Models Are Few-Shot Learners
11. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv.
12. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
13. Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., and Liu, Q. (August, January 28). ERNIE: Enhanced Language Representation with Informative Entities. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
14. Ernie 2.0: A Continual Pre-Training Framework for Language Understanding
15. Sun, Y., Wang, S., Feng, S., Ding, S., Pang, C., Shang, J., Liu, J., Chen, X., Zhao, Y., and Lu, Y. (2021). ERNIE 3.0: Large-Scale Knowledge Enhanced Pre-Training for Language Understanding and Generation. arXiv.
16. Souza, F., Nogueira, R., and Lotufo, R. (2019). Portuguese Named Entity Recognition Using BERT-CRF. arXiv.
17. Deep Learning for Named Entity Recognition: A Survey10.1007/s00521-024-09646-6 ↗
18. A Comprehensive Survey on Transfer Learning10.1109/JPROC.2020.3004555 ↗
19. Das, S.S.S., Katiyar, A., Passonneau, R.J., and Zhang, R. (2022). CONTaiNER: Few-Shot Named Entity Recognition via Contrastive Learning. arXiv.10.18653/v1/2022.acl-long.439 ↗
20. Named Entity Extraction for Knowledge Graphs: A Literature Overview10.1109/ACCESS.2020.2973928 ↗
21. Pre-Train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing10.1145/3560815 ↗
22. Cui, L., Wu, Y., Liu, J., Yang, S., and Zhang, Y. (2021). Template-Based Named Entity Recognition Using BART, ACL Anthology.10.18653/v1/2021.findings-acl.161 ↗
23. Chen, X., Li, L., Deng, S., Tan, C., Xu, C., Huang, F., Si, L., Chen, H., and Zhang, N. (2022). LightNER: A Lightweight Tuning Paradigm for Low-Resource NER via Pluggable Prompting. arXiv.
24. PromptNER: Prompt Locating and Typing for Named Entity Recognition
25. Chinese Named Entity Recognition: The State of the Art10.1016/j.neucom.2021.10.101 ↗
26. Zhang, S., Li, G., Shi, X., Liu, L., and Chen, M. (2024, January 20–22). Chinese Named Entity Recognition with Integrated Boundary Features and Location Information. Proceedings of the 2024 IEEE 7th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), Chongqing, China.10.1109/ITNEC60942.2024.10732932 ↗
27. A Boundary Regression Model for Nested Named Entity Recognition10.1007/s12559-022-10058-8 ↗
28. Yuan, L., Yang, S., Li, R., Kanoulas, E., and Zhao, X. (2023, January 15–17). Chinese Nested Named Entity Recognition Based on Boundary Prompt. Proceedings of the Web Information Systems and Applications, Chengdu, China.
29. Wang, Y., Tong, H., Zhu, Z., Hou, F., and Li, Y. (2025). Enhancing Biomedical Named Entity Recognition with Parallel Boundary Detection and Category Classification. BMC Bioinform., 26.10.1186/s12859-025-06086-4 ↗
30. Chen, C., and Kong, F. (2021, January 1–6). Enhancing Entity Boundary Detection for Better Chinese Named Entity Recognition. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Virtual Event.10.18653/v1/2021.acl-short.4 ↗
31. Natural Language Processing (Almost) from Scratch
32. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
33. A Review on the Long Short-Term Memory Model10.1007/s10462-020-09838-1 ↗
34. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention Is All You Need. Adv. Neural Inf. Process. Syst., 30.
35. Pre-Trained Models: Past, Present and Future10.1016/j.aiopen.2021.08.002 ↗
36. Ma, R., Zhou, X., Gui, T., Tan, Y., Zhang, Q., and Huang, X. (2021). Template-Free Prompt Tuning for Few-Shot NER. arXiv.10.18653/v1/2022.naacl-main.420 ↗
37. Gu, Y., Han, X., Liu, Z., and Huang, M. (2022). PPT: Pre-Trained Prompt Tuning for Few-Shot Learning. arXiv.10.18653/v1/2022.acl-long.576 ↗
38. Han, X., Zhao, W., Ding, N., Liu, Z., and Sun, M. (2021). PTR: Prompt Tuning with Rules for Text Classification. arXiv.10.1016/j.aiopen.2022.11.003 ↗
39. Chinese NER Using Dynamic Meta-Embeddings10.1109/ACCESS.2019.2916816 ↗
40. Zhang, Y., and Yang, J. (2018, January 15–20). Chinese NER Using Lattice LSTM. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia.10.18653/v1/P18-1144 ↗
41. Wang, C., Zhao, S., Yan, T., Song, S., Ma, W., Liu, K., and Wang, M. (2025). Hierarchical Label-Enhanced Contrastive Learning for Chinese NER. IEEE Transactions on Neural Networks and Learning Systems, IEEE.10.1109/TNNLS.2025.3528416 ↗
42. Zhang, S., Wang, L., Sun, K., and Xiao, X. (2020). A Practical Chinese Dependency Parser Based on A Large-Scale Dataset. arXiv.
43. Pradhan, S., Moschitti, A., Xue, N., Ng, H.T., Björkelund, A., Uryupina, O., Zhang, Y., and Zhong, Z. (2013, January 8–9). Towards Robust Linguistic Analysis Using Ontonotes. Proceedings of the Seventeenth Conference on Computational Natural Language Learning, Sofia, Bulgaria.
44. Peng, N., and Dredze, M. (2015, January 17–21). Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.10.18653/v1/D15-1064 ↗
45. Ng, H.T., and Kwong, O.O.Y. (2006, January 22–23). The Third International Chinese Language Processing Bakeoff: Word Segmentation and Named Entity Recognition. Proceedings of the Fifth SIGHAN Workshop on Chinese Language Processing, Sydney, Australia.
46. Eberts, M., and Ulges, A. (September, January 29). Span-Based Joint Entity and Relation Extraction with Transformer Pre-Training. Proceedings of the ECAI 2020, Santiago de Compostela, Spain.
47. Li, X., Feng, J., Meng, Y., Han, Q., Wu, F., and Li, J. (2020, January 5–10). A Unified MRC Framework for Named Entity Recognition. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.10.18653/v1/2020.acl-main.519 ↗
48. Yan, H., Gui, T., Dai, J., Guo, Q., Zhang, Z., and Qiu, X. (2021). A Unified Generative Framework for Various NER Subtasks. arXiv.10.18653/v1/2021.acl-long.451 ↗