Korean Semantic Role Labeling with Bidirectional Encoder Representations from Transformers and Simple Semantic Information
Source abstract
Abstract
State-of-the-art semantic role labeling (SRL) performance has been achieved using neural network models by incorporating syntactic feature information such as dependency trees. In recent years, breakthroughs achieved using end-to-end neural network models have resulted in a state-of-the-art SRL performance even without syntactic features. With the advent of a language model called bidirectional encoder representations from transformers (BERT), another breakthrough was witnessed. Even though the semantic information of each word constituting a sentence is important in determining the meaning of a word, previous studies regarding the end-to-end neural network method did not utilize semantic information. In this study, we propose a BERT-based SRL model that uses simple semantic information without syntactic feature information. To obtain the latter, we used PropBank, which described the relational information between predicates and arguments. In addition, text-originated feature information obtained from the training text data was utilized. Our proposed model achieved state-of-the-art results on both Korean PropBank and CoNLL-2009 English benchmarks.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Natural Language Processing Techniques · Topic Modeling · Speech and dialogue systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Language AI Lab, LG CNS, Seoul 07795, Korea
Department of Computer Science and Engineering, Kangwon National University, Chuncheon 24341, Korea
Funding
Institute for Information & Communications Technology Promotion (IITP)
References (29)
Show source-provided references
1. Chen, Y.N., Wang, W.Y., and Rudnicky, A.I. (2013, January 8–12). Unsupervised induction and filling of semantic slots for spoken dialogue systems using frame-semantic parsing. Proceedings of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, Olomouc, Czech Republic.10.1109/ASRU.2013.6707716 ↗
2. Shen, D., and Lapata, M. (2007, January 28–30). Using semantic roles to improve question answering. Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), Prague, Czech Republic.
3. Yih, W., Richardson, M., Meek, C., Chang, M., and Suh, J. (2016, January 7–12). The value of semantic parse labeling for knowledge base question answering. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), Berlin, Germany.10.18653/v1/P16-2033 ↗
4. Fader, A., Soderland, S., and Etzioni, O. (2011, January 27–31). Identifying relations for open information extraction. Proceedings of the Conference on Empirical Methods in Natural Language Processing, Edinburgh, UK.
5. Bastianelli, E., Castellucci, G., Croce, D., and Basili, R. (2013, January 20–23). Textual inference and meaning representation in human robot interaction. Proceedings of the Joint Symposium on Semantic Processing, Textual Inference and Structures in Corpora, Trento, Italy.
6. Zhou, J., and Xu, W. (2015, January 27–31). End-to-end learning of semantic role labeling using recurrent neural networks. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China.10.3115/v1/P15-1109 ↗
7. He, L., Lee, K., Lewis, M., and Zettlemoyer, L. (August, January 30). Deep semantic role labeling: What works and what’s next. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, BC, Canada.
8. Tan, Z., Wang, M., Xie, J., Chen, Y., and Shi, X. (2018, January 2–7). Deep semantic role labeling with self-attention. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18), New Orleans, LA, USA.10.1609/aaai.v32i1.11928 ↗
9. He, L., Lee, K., Levy, O., and Zettlemoyer, L. (2018, January 15–20). Jointly predicting predicates and arguments in neural semantic role labeling. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia.10.18653/v1/P18-2058 ↗
10. Levin, B. (1993). English Verb Classes and Alternations: A Preliminary Investigation, University of Chicago Press.
11. The importance of syntactic parsing and inference in semantic role labeling10.1162/coli.2008.34.2.257 ↗
12. Roth, M., and Lapata, M. (2016, January 7–12). Neural semantic role labeling with dependency path embeddings. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany.10.18653/v1/P16-1113 ↗
13. Strubell, E., Verga, P., Andor, D., Weiss, D., and McCallum, A. (2018, January 2–4). Linguistically-informed self-attention for semantic role labeling. Proceedings of the Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.10.18653/v1/D18-1548 ↗
14. He, S., Li, Z., Zhao, H., and Bai, H. (2018, January 15–20). Syntax for semantic role labeling, to be, or not to be. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia.10.18653/v1/P18-1192 ↗
15. Shi, P., and Zhang, Y. (2017, January 5–7). Joint bi-affine parsing and semantic role labeling. Proceedings of the 2017 International Conference on Asian Language Processing (IALP), Singapore.10.1109/IALP.2017.8300612 ↗
16. Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L. (2018, January 1–6). Deep contextualized word representations. Proceedings of the Conference of the North American Chapter of the Association or Computational Linguistics: Human Language Technologies, New Orleans, LA, USA.10.18653/v1/N18-1202 ↗
17. Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2018). Improving language understanding by generative pre-training. arXiv.
18. Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv.
19. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. arXiv.
20. Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., and Le, Q. (2020). XLNet: Generalized Autoregressive Pretraining for Language Understanding. arXiv.
21. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020). Language Models Are Few-Shot Learners. arXiv.
22. Shi, P., and Lin, J. (2019). Simple BERT Models for Relation Extraction and Semantic Role Labeling. arXiv.
23. Kingsbury, P., and Palmer, M. (2002, January 29–31). From treebank to propbank. Proceedings of the Third International Conference on Language Resources and Evaluation (LREC’02). European Language Resources Association (ELRA), Las Palmas, Spain.
24. Li, X., Bing, L., Zhang, W., and Lam, W. (2019, January 4). Exploiting BERT for End-to-End Aspect-based Sentiment Analysis. Proceedings of the 2019 EMNLP Workshop W-NUT: The 5th Workshop on Noisy User-Generated Text, Hong Kong, China.10.18653/v1/D19-5505 ↗
25. Huang, Z., Xu, W., and Yu, K. (2015). Bidirectional LSTM-CRF Models for Sequence Tagging. arXiv.
26. Palmer, M., Ryu, S., Choi, J., Yoon, S., and Jeon, Y. (2022, May 19). Korean Propbank LDC2006T03. Available online: https://catalog.ldc.upenn.edu/LDC2006T03.
27. Korean Semantic Role Labeling Using Stacked Bidirectional LSTM-CRFs10.5626/JOK.2017.44.1.36 ↗
28. Bae, J., and Lee, C. (2017, January 13–14). Korean Semantic Role Labeling with Highway BiLSTM-CRFs. Proceedings of the HCLT, Daegu, Korea.
29. Lim, S., and Kim, H. (2015, January 16–17). A Study of Korean Semantic Role Labeling using Word Sense. Proceedings of the HCLT, Jeonju, Korea.