Document-Level Event Role Filler Extraction Using Key-Value Memory Network
Source abstract
Abstract
Previous work has demonstrated that end-to-end neural sequence models work well for document-level event role filler extraction. However, the end-to-end neural network model suffers from the problem of not being able to utilize global information, resulting in incomplete extraction of document-level event arguments. This is because the inputs to BiLSTM are all single-word vectors with no input of contextual information. This phenomenon is particularly pronounced at the document level. To address this problem, we propose key-value memory networks to enhance document-level contextual information, and the overall model is represented at two levels: the sentence-level and document-level. At the sentence-level, we use BiLSTM to obtain key sentence information. At the document-level, we use a key-value memory network to enhance document-level representations by recording information about those words in articles that are sensitive to contextual similarity. We fuse two levels of contextual information by means of a fusion formula. We perform various experimental validations on the MUC-4 dataset, and the results show that the model using key-value memory networks works better than the other models.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Natural Language Processing Techniques · Advanced Text Analysis Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Science and Technology, North China University of Technology, Beijing 100144, China
CNONIX National Standard Application and Promotion Lab, Beijing 100144, China
Beijing Urban Governance Research Center, Beijing 102206, China
Funding
RD Program of the Beijing Municipal Education Commission · National Natural Science Foundation of China · Beijing Urban Governance Research Center Fund
References (25)
Show source-provided references
1. Patwardhan, S., and Riloff, E. (2009, January 6–7). A unified model of phrasal and sentential evidence for information extraction. Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, Singapore.10.3115/1699510.1699530 ↗
2. Huang, R., and Riloff, E. (2011, January 19–24). Peeling back the layers: Detecting event role fillers in secondary contexts. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, OR, USA.
3. Named entity recognition with bidirectional LSTM-CNNs10.1162/tacl_a_00104 ↗
4. Wadden, D., Wennberg, U., Luan, Y., and Hajishirzi, H. (2019). Entity, relation, and event extraction with contextualized span representations. arXiv.10.18653/v1/D19-1585 ↗
5. Trinh, T., Dai, A., Luong, T., and Le, Q. (2018, January 10–15). Learning longer-term dependencies in rnns with auxiliary losses. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden.
6. Du, X., and Cardie, C. (2020). Document-level event role filler extraction using multi-granularity contextualized encoding. arXiv.10.18653/v1/2020.acl-main.714 ↗
7. Akbik, A., Bergmann, T., and Vollgraf, R. (2019, January 2–7). Pooled contextualized embeddings for named entity recognition. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.10.18653/v1/N19-1078 ↗
8. Surdeanu, M., Harabagiu, S., Williams, J., and Aarseth, P. (2003, January 7–12). Using predicate-argument structures for information extraction. Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, Sapporo, Japan.10.3115/1075096.1075098 ↗
9. Chen, Y., Xu, L., Liu, K., Zeng, D., and Zhao, J. (2015, January 26–31). Event extraction via dynamic multi-pooling convolutional neural networks. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the Seventh International Joint Conference on Natural Language Processing, Beijing, China.10.3115/v1/P15-1017 ↗
10. Nguyen, T., and Grishman, R. (2018, January 2–7). Graph convolutional networks with argument-aware pooling for event detection. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.12039 ↗
11. Yao, L., Mao, C., and Luo, Y. (February, January 27). Graph convolutional networks for text classification. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
12. Yang, S., Feng, D., Qiao, L., Kan, Z., and Li, D. (August, January 28). Exploring pre-trained language models for event extraction and generation. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
13. Yang, H., Chen, Y., Liu, K., Xiao, Y., and Zhao, J. (2018, January 15–20). Dcfee: A document-level chinese financial event extraction system based on automatically labeled training data. Proceedings of the ACL 2018, System Demonstrations, Melbourne, Australia.10.18653/v1/P18-4009 ↗
14. Yang, B., and Mitchell, T. (2016). Joint extraction of events and entities within a document context. arXiv.10.18653/v1/N16-1033 ↗
15. (2019). Doc2EDAG: An end-to-end document-level framework for Chinese financial event extraction. arXiv.
16. (2021). Document-level event extraction via heterogeneous graph-based interaction model with a tracker. arXiv.
17. (2021, January 1–6). Document-level event extraction via parallel prediction networks. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Bangkok, Thailand.
18. Zhang, Y., Liu, Q., and Song, L. (2018). Sentence-state lstm for text representation. arXiv.10.18653/v1/P18-1030 ↗
19. Zhou, J., and Zhao, H. (2019). Head-driven phrase structure grammar parsing on Penn treebank. arXiv.10.18653/v1/P19-1230 ↗
20. Pennington, J., Socher, R., and Manning, C.D. (2014, January 25–29). Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1162 ↗
21. Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L. (2018). Deep Contextualized Word Representations, Association for Computational Linguistics.10.18653/v1/N18-1202 ↗
22. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2–7). Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the NAACL-HLT, Minneapolis, MN, USA.
23. Luo, Y., Xiao, F., and Zhao, H. (2020, January 7–12). Hierarchical contextualized representation for named entity recognition. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i05.6363 ↗
24. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Advances in Neural Information Processing Systems, MIT Press.
25. Huang, R., and Riloff, E. (2012, January 22–26). Modeling textual cohesion for event extraction. Proceedings of the AAAI Conference on Artificial Intelligence, Toronto, ON, Canada.10.1609/aaai.v26i1.8354 ↗