Generating Chinese Event Extraction Method Based on ChatGPT and Prompt Learning
Source abstract
Abstract
Regarding the scarcity of annotated data for existing event extraction tasks and the insufficient semantic mining of event extraction models in the Chinese domain, this paper proposes a generative joint event extraction model to improve existing models in two aspects. Firstly, it utilizes the content generation capability of ChatGPT to generate annotated data corpora for event extraction tasks and trains the model using supervised learning methods adapted to downstream tasks. Secondly, explicit entity markers and event knowledge are added to the text to construct generative input templates, enhancing the performance of event extraction. To validate the performance of this model, experiments are conducted on DuEE1.0 and Title2Event public datasets, and the results show that both data enhancement and prompt learning based on ChatGPT effectively improve the performance of the event extraction model, and the F1 values of the events extracted by the CPEE model proposed in this paper reach 85.1% and 59.9% on the two datasets, respectively, which are comparable to the existing models’ values of 1.3% and 10%, respectively; moreover, on the Title2Event dataset, the performance of different models on the event extraction task can be gradually improved as the data size of the annotated corpus of event extraction generated using ChatGPT increases.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Computational and Text Analysis Methods · Data Quality and Management
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Network Security, People’s Public Security University of China, Beijing 100038, China
Funding
2022 Post Graduate Course Construction Project of People’s Public Security University of China · Fundamental Research Funds for the Central Universities
References (38)
Show source-provided references
1. A Survey of Event Extraction Methods from Text for Decision Support Systems10.1016/j.dss.2016.02.006 ↗
2. Garofolo, J. (2021, October 03). Automatic Content Extraction (ACE), Available online: http://itl.gov/iad/mig/-tests/ace/2005.
3. Walker, C., Strassel, S., Medero, J., and Maeda, K. (2006). ACE 2005 Multilingual Training Corpus LDC2006T06, Linguistic Data Consortium. Web Download.
4. Satyapanich, T., Ferraro, F., and Finin, T. (2020, January 7–12). CASIE: Extracting Cybersecurity Event Information from Text. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i05.6401 ↗
5. Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., and Zettlemoyer, L. (2020, January 5–10). BART: Denoising Sequence-to-Sequence Pre-Training for Natural Language Generation, Translation, and Comprehension. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.10.18653/v1/2020.acl-main.703 ↗
6. Riloff, E. (1993, January 11). Automatically Constructing a Dictionary for Information Extraction Tasks. Proceedings of the Eleventh National Conference on Artificial Intelligence, Washington, DC, USA.
7. Acquisition of Linguistic Patterns for Knowledge-Based Information Extraction10.1109/69.469825 ↗
8. Riloff, E., and Shoen, J. (1995, January 30). Automatically Acquiring Conceptual Patterns without an Annotated Corpus. Proceedings of the Third Workshop on Very Large Corpora, Boston, MA, USA.
9. Yangarber, R. (2000). Scenario Customization for Information Extraction. [Ph.D. Thesis, New York University].
10. An Event IE Pattern Acquisition Method
11. Chieu, H.L., and Ng, H.T. (2002, January 28). A Maximum Entropy Approach to Information Extraction from Semi-Structured and Free Text. Proceedings of the Eighteenth National Conference on Artificial Intelligence, Edmonton, AB, Canada.
12. Llorens, H., Saquete, E., and Navarro-Colorado, B. (2010, January 23–27). TimeML Events Recognition and Classification: Learning CRF Models with Semantic Roles. Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), Coling 2010 Organizing Committee, Beijing, China.
13. Li, Q., Ji, H., and Huang, L. (2013, January 4–9). Joint Event Extraction via Structured Prediction with Global Features. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, Long Papers, Sofia, Bulgaria.
14. Ahn, D. (2006, January 23). The Stages of Event Extraction. Proceedings of the Workshop on Annotating and Reasoning about Time and Events, Sydney, Australia.10.3115/1629235.1629236 ↗
15. Chen, Y., Xu, L., Liu, K., Zeng, D., and Zhao, J. (2015, January 26–31). Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Beijing, China.10.3115/v1/P15-1017 ↗
16. Zeng, Y., Yang, H., Feng, Y., Wang, Z., and Zhao, D. (2016, January 2–6). A Convolution BiLSTM Neural Network Model for Chinese Event Extraction. Proceedings of the Natural Language Understanding and Intelligent Applications: 5th CCF Conference on Natural Language Processing and Chinese Computing, NLPCC 2016, and 24th International Conference on Computer Processing of Oriental Languages, ICCPOL 2016, Kunming, China.
17. Nguyen, T.H., and Grishman, R. (2018, January 2). Graph Convolutional Networks with Argument-Aware Pooling for Event Detection. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.12039 ↗
18. Li, F., Peng, W., Chen, Y., Wang, Q., Pan, L., Lyu, Y., and Zhu, Y. (2020, January 16–20). Event Extraction as Multi-Turn Question Answering. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2020, Online.10.18653/v1/2020.findings-emnlp.73 ↗
19. Du, X., and Cardie, C. (2020, January 16–20). Event Extraction by Answering (Almost) Natural Questions. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.10.18653/v1/2020.emnlp-main.49 ↗
20. Multi-turn Event Argument Extraction Based on Role Information Guidance
21. Ding, N., Li, Z., Liu, Z., Zheng, H., and Lin, Z. (2019, January 3–7). Event Detection with Trigger-Aware Lattice Neural Network. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.10.18653/v1/D19-1033 ↗
22. Lai, V.D., Nguyen, T.N., and Nguyen, T.H. (2020, January 16–20). Event Detection: Gate Diversity and Syntactic Importance Scores for Graph Convolution Neural Networks. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.10.18653/v1/2020.emnlp-main.435 ↗
23. Nguyen, T.H., Cho, K., and Grishman, R. (2016, January 12–17). Joint Event Extraction via Recurrent Neural Networks. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA.10.18653/v1/N16-1034 ↗
24. Sha, L., Qian, F., Chang, B., and Sui, Z. (2018, January 2–7). Jointly Extracting Event Triggers and Arguments by Dependency-Bridge RNN and Tensor-Based Argument Interaction. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.12034 ↗
25. Sheng, J., Guo, S., Yu, B., Li, Q., Hei, Y., Wang, L., Liu, T., and Xu, H. (2021, January 1–6). CasEE: A Joint Learning Framework with Cascade Decoding for Overlapping Event Extraction. Proceedings of the Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Online.10.18653/v1/2021.findings-acl.14 ↗
26. One for All: Neural Joint Modeling of Entities and Events
27. Lu, Y., Lin, H., Xu, J., Han, X., Tang, J., Li, A., Sun, L., Liao, M., and Chen, S. (2021, January 1–6). Text2Event: Controllable Sequence-to-Structure Generation for End-to-End Event Extraction. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online.10.18653/v1/2021.acl-long.217 ↗
28. Lu, Y., Liu, Q., Dai, D., Xiao, X., Lin, H., Han, X., Sun, L., and Wu, H. (2022, January 22–27). Unified Structure Generation for Universal Information Extraction. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Dublin, Ireland.10.18653/v1/2022.acl-long.395 ↗
29. Huang, K.-H., Hsu, I.-H., Natarajan, P., Chang, K.-W., and Peng, N. (2022, January 22–27). Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Dublin, Ireland.10.18653/v1/2022.acl-long.317 ↗
30. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019, January 7). BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA.
31. Radford, A., and Narasimhan, K. (2018). Improving Language Understanding by Generative Pre-Training, OpenAI.
32. Winata, G.I., Madotto, A., Lin, Z., Liu, R., Yosinski, J., and Fung, P. (2021, January 11). Language Models Are Few-Shot Multilingual Learners. Proceedings of the 1st Workshop on Multilingual Representation Learning; Association for Computational Linguistics, Punta Cana, Dominican Republic.10.18653/v1/2021.mrl-1.1 ↗
33. Li, X., Li, F., Pan, L., Chen, Y., Peng, W., Wang, Q., Lyu, Y., and Zhu, Y. (2020, January 14–18). DuEE: A Large-Scale Dataset for Chinese Event Extraction in Real-World Scenarios. Proceedings of the Natural Language Processing and Chinese Computing: 9th CCF International Conference, NLPCC 2020, Zhengzhou, China.10.1007/978-3-030-60457-8_44 ↗
34. Deng, H., Zhang, Y., Zhang, Y., Ying, W., Yu, C., Gao, J., Wang, W., Bai, X., Yang, N., and Ma, J. (2022, January 7–11). Title2Event: Benchmarking Open Event Extraction with a Large-Scale Chinese Title Dataset. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Abu Dhabi, United Arab Emirates.10.18653/v1/2022.emnlp-main.437 ↗
35. Shao, Y., Geng, Z., Liu, Y., Dai, J., Yan, H., Yang, F., Zhe, L., Bao, H., and Qiu, X. (2021). CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation. arXiv.
36. Zhang, Z.Y., Han, X., Liu, Z.Y., Jiang, X., Sun, M.S., and Liu, Q. (August, January 28). ERNIE: Enhanced Language Representation with Informative Entities. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
37. Yang, S., Feng, D., Qiao, L., Kan, Z., and Li, D. (August, January 28). Exploring Pre-Trained Language Models for Event Extraction and Generation. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
38. Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., and Raffel, C. (2021, January 6–11). MT5: A Massively Multilingual Pre-Trained Text-to-Text Transformer. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online.10.18653/v1/2021.naacl-main.41 ↗