CJE-PCHF: Chinese Joint Entity and Relation Extraction Model Based on Progressive Contrastive Learning and Heterogeneous Feature Fusion
Source abstract
Abstract
The joint extraction of entities and relations is a critical task in information extraction, and its performance directly affects the performance of downstream tasks. However, existing joint extraction models based on deep learning exhibit weak processing capabilities for the phenomenon of multiple pronunciations of one character and multiple characters of one pronunciation when processing Chinese texts, resulting in a performance loss. To address these issues, this paper introduces part-of-speech (POS) and pinyin features to aid the model in learning semantic features that are more contextually appropriate. We propose a Chinese Joint Entity and Relation Extraction Model based on progressive contrastive learning and heterogeneous feature fusion (CJE-PCHF). During model training, an interactive fusion network based on progressive contrastive learning is employed to learn the dependencies between pinyin, POS, and semantic features. This guides the model in heterogeneous feature fusion, capturing higher-order semantic associations between heterogeneous features. On the commonly used DuIE evaluation dataset for joint extraction, our model achieved a significant improvement, with the F1 score increasing by 5.4% compared to the benchmark model CasRel.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Natural Language Processing Techniques · Text and Document Classification Technologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer and Information Engineering, Inner Mongolia Agricultural University, Hohhot 010018, China
Funding
Inner Mongolia Autonomous Region Science and Technology Plan Project · Basic Business Funding Project for Universities Directly under Inner Mongolia Autonomous Region
References (31)
Show source-provided references
1. A survey of knowledge mapping construction techniques
2. Sun, C., Gong, Y., Wu, Y., Gong, M., Jiang, D., Lan, M., Sun, S., and Duan, N. (August, January 28). Joint type inference on entities and relations via graph convolutional networks. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
3. Chinese entity and relation extraction model based on parallel heterogeneous graph and sequential attention mechanism
4. Lu, X., Sun, L., Ling, C., Tong, Z., Liu, J., and Tang, Q. (2024). Named Entity Recognition of Chinese Electronic Health Records Incorporating Phonetic and Part-of-speech Features. J. Chin. Comput. Syst., 1–12. Available online: https://link.cnki.net/urlid/21.1106.TP.20240228.1116.013.
5. Saunshi, N., Plevrakis, O., Arora, S., Khodak, M., and Khandeparkar, H. (2019, January 9–15). A theoretical analysis of contrastive unsupervised representation learning. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA.
6. Shao, Y. (2024, October 08). Research on Chinese Entity Relationship Extraction Based on Deep Learning. Available online: https://link.cnki.net/doi/10.27162/d.cnki.gjlin.2020.002647.
7. Miwa, M., and Bansal, M. (2016). End-to-end relation extraction using lstms on sequences and tree structures. arXiv.10.18653/v1/P16-1105 ↗
8. Katiyar, A., and Cardie, C. (August, January 30). Going out on a limb: Joint extraction of entity mentions and relations without dependency trees. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, BC, Canada.
9. Zheng, S., Wang, F., Bao, H., Hao, Y., Zhou, P., and Xu, B. (2017). Joint extraction of entities and relations based on a novel tagging scheme. arXiv.10.18653/v1/P17-1113 ↗
10. Zeng, X., Zeng, D., He, S., Liu, K., and Zhao, J. (2018, January 15–20). Extracting relational facts by an end-to-end neural model with copy mechanism. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia.10.18653/v1/P18-1047 ↗
11. Fu, T., Li, P., and Ma, W. (August, January 28). Graphrel: Modeling text as relational graphs for joint entity and relation extraction. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
12. Wei, Z., Su, J., Wang, Y., Tian, Y., and Chang, Y. (2019). A novel cascade binary tagging framework for relational triple extraction. arXiv.10.18653/v1/2020.acl-main.136 ↗
13. Review of Chinese named entity recognition research
14. Zhang, Y., Liu, Y., Zhu, J., Zheng, Z., Liu, X., Wang, W., Chen, Z., and Zhai, S. (2019, January 3–7). Learning Chinese word embeddings from stroke, structure and pinyin of characters. Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, China.10.1145/3357384.3358005 ↗
15. Named Entity Recognition of Microblog Texts Based on POS Features
16. Zhu, W., Jin, X., Ni, J., Wei, B., and Lu, Z. (2018). Improve word embedding using both writing and pronunciation. PLoS ONE, 13.10.1371/journal.pone.0208785 ↗
17. Wu, S., Song, X., and Feng, Z. (2021). MECT: Multi-metadata embedding based cross-transformer for Chinese named entity recognition. arXiv.10.18653/v1/2021.acl-long.121 ↗
18. AMFRel: A method for joint extraction of enity relations in Chinese electronic medical records
19. Li, S., Chang, Z., and Liu, Y. (2024). Joint Extraction of Entities and Relations Based on Multi-feature Fusion. Int. J. Emerg. Technol. Adv. Appl., 1.10.62677/IJETAA.2401101 ↗
20. Joint extraction method of Chinese entity relationship based on mixture of characters and words
21. Exploring the intelligent extraction of entity relations in Chinese text: A new method based on data and model collaborative optimization
22. Song, W., Zhou, X., Yang, P., Sun, Y., Yang, L., and Lin, H. (2021, January 13–15). Joint Entity and Relation Extraction for Legal Texts. Proceedings of the 20th Chinese National Conference on Computational Linguistics, Hohhot, China.
23. Joint entity and relation extraction based on contextual semantic enhancement
24. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., and Dean, J. (2013). Distributed representations of words and phrases and their compositionality. Adv. Neural Inf. Process. Syst., 26.
25. Fang, H., Wang, S., Zhou, M., Ding, J., and Xie, P. (2020). CERT: Contrastive Self-supervised Learning for Language Understanding. arXiv.10.36227/techrxiv.12308378.v1 ↗
26. Li, S., He, W., Shi, Y., Jiang, W., Liang, H., Jiang, Y., Zhang, Y., Lyu, Y., and Zhu, Y. (2019, January 9–14). Duie: A large-scale chinese dataset for information extraction. Proceedings of the Natural Language Processing and Chinese Computing: 8th CCF International Conference, NLPCC 2019, Dunhuang, China. Proceedings, Part II 8.10.1007/978-3-030-32236-6_72 ↗
27. Hoffmann, R., Zhang, C., Ling, X., Zettlemoyer, L., and Weld, D.S. (2011, January 19–24). Knowledge-based weak supervision for information extraction of overlapping relations. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, OR, USA.
28. Ren, X., Wu, Z., He, W., Qu, M., Voss, C.R., Ji, H., Abdelzaher, T.F., and Han, J. (2017, January 3–7). Cotype: Joint extraction of typed entities and relations with knowledge bases. Proceedings of the 26th International Conference on World Wide Web, Perth, Australia.10.1145/3038912.3052708 ↗
29. Joint extraction method of entity and relationship based on pointer network
30. Joint entity and relation extraction fusing entity type information
31. Yu, K. (2022, January 21–23). Joint Extraction of Chinese Entity Relationship Based on Bidirectional Semantic Learning Model. Proceedings of the 2022 6th International Conference on Electronic Information Technology and Computer Engineering, Xiamen, China.