An Effective Knowledgeable Label-Aware Approach for Sentential Relation Extraction
Source abstract
Abstract
In recent years, sentential relation extraction has made remarkable progress with text and knowledge graphs (KGs). However, existing architectures ignore the valuable information contained in relationship labels, which KGs provide and can complement the model with additional signals and prior knowledge. To address this limitation, we propose a neural architecture that leverages knowledgeable labels to enhance sentential relation extraction. We name our proposed method knowledge label-aware sensitive relation extraction (KLA-SRE). To achieve this, we combine pre-trained static knowledge graph embeddings with learned semantic embeddings from other tokens to efficiently represent relation labels. By combining static pre-trained graph embeddings with learned word embeddings, we mitigate the inconsistency between predicted relations and given entities. Experimental results on various relation extraction benchmarks in different fields show that knowledge labels improve the F1 score by 1.6% and 1.1% on average over the baseline on standard- and minority-shot benchmarks, respectively.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Topic Modeling · Natural Language Processing Techniques · Advanced Text Analysis Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Digital Media School, Hangzhou Dianzi University, Hangzhou 310000, China
Funding
Fundamental Research Funds for the Provincial Universities of Zhejiang
References (40)
Show source-provided references
1. Vrandečić, D. (2012, January 16–20). Wikidata: A new platform for collaborative data collection. Proceedings of the Web Conference, Lyon, France.10.1145/2187980.2188242 ↗
2. Suchanek, F.M., Kasneci, G., and Weikum, G. (2007, January 8–12). Yago: A Core of Semantic Knowledge Unifying WordNet and Wikipedia. Proceedings of the 16th International Conference on World Wide Web, Banff, AB, Canada.10.1145/1242572.1242667 ↗
3. The Unified Medical Language System (UMLS): Integrating biomedical terminology10.1093/nar/gkh061 ↗
4. Mintz, M.D., Bills, S., Snow, R., and Jurafsky, D. (2009, January 2–7). Distant supervision for relation extraction without labeled data. Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, Suntec, Singapore.10.3115/1690219.1690287 ↗
5. Bastos, A., Nadgeri, A., Singh, K., Mulang, I.O., Shekarpour, S., and Hoffart, J. (2021, January 19–23). RECON: Relation Extraction using Knowledge Graph Context in a Graph Neural Network. Proceedings of the Web Conference, Ljubljana, Slovenia.10.1145/3442381.3449917 ↗
6. Nadgeri, A., Bastos, A., Singh, K., Mulang, I.O., Hoffart, J., Shekarpour, S., and Saraswat, V.A. (2021, January 1–6). KGPool: Dynamic Knowledge Graph Context Selection for Relation Extraction. Proceedings of the Findings of the Association for Computational Linguistics: ACL-IJCNLP, Virtual Event.10.18653/v1/2021.findings-acl.48 ↗
7. Vashishth, S., Joshi, R., Prayaga, S.S., Bhattacharyya, C., and Talukdar, P.P. (November, January 31). RESIDE: Improving Distantly-Supervised Neural Relation Extraction using Side Information. Proceedings of the Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.
8. Papaluca, A., Krefl, D., Suominen, H., and Lenskiy, A. (2022, January 22–27). Pretrained Knowledge Base Embeddings for improved Sentential Relation Extraction. Proceedings of the Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, Dublin, Ireland.10.18653/v1/2022.acl-srw.29 ↗
9. Sorokin, D., and Gurevych, I. (2017, January 9–11). Context-Aware representations for knowledge base relation extraction. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP, Copenhagen, Denmark.10.18653/v1/D17-1188 ↗
10. Zhou, W., and Chen, M. (2022, January 20–23). An Improved Baseline for Sentence-level Relation Extraction. Proceedings of the Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, Online.
11. Xu, B., Liu, N., Cheng, L., Huang, S., Wei, S., Du, M., Song, H., and Wang, H. (2023, January 18–23). Knowledge Graph Enhanced Sentential Relation Extraction via Dual Heterogeneous Graph Context Selection. Proceedings of the International Joint Conference on Neural Networks, Gold Coast, Australia.10.1109/IJCNN54540.2023.10191797 ↗
12. Soares, L.B., FitzGerald, N., Ling, J., and Kwiatkowski, T. (August, January 28). Matching the blanks: Distributional similarity for relation learning. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
13. Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., and Liu, Q. (August, January 28). ERNIE: Enhanced language representation with informative entities. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy.
14. Peters, M.E., Neumann, M., Logan, R., Schwartz, R., Joshi, V., Singh, S., and Smith, N.A. (2019, January 3–7). Knowledge enhanced contextual word representations. Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China.10.18653/v1/D19-1005 ↗
15. Weston, J., Bordes, A., Yakhnenko, O., and Usunier, N. (2013, January 18–21). Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction. Proceedings of the Conference on Empirical Methods in Natural Language Processing, Grand Hyatt, Seattle, Seattle, WA, USA.
16. Han, X., Liu, Z., and Sun, M. (2018, January 2–7). Neural Knowledge Acquisition via Mutual Attention Between Knowledge Graph and Text. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11927 ↗
17. Zhang, N., Deng, S., Sun, Z., Wang, G., Chen, X., Zhang, W., and Chen, H. (2019, January 2–7). Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks. Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.10.18653/v1/N19-1306 ↗
18. Hu, L., Zhang, L., Shi, C., Nie, L., Guan, W., and Yang, C. (2019, January 3–7). Improving Distantly-Supervised Relation Extraction with Joint Label Embedding. Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China.10.18653/v1/D19-1395 ↗
19. Dai, Q., Inoue, N., Reisert, P., Takahashi, R., and Inui, K. (2019, January 13–15). Incorporating Chains of Reasoning over Knowledge Graph for Distantly Supervised Biomedical Knowledge Acquisition. Proceedings of the Pacific Asia Conference on Language, Information and Computation, Hakodate, Japan.10.18653/v1/W19-2601 ↗
20. Hu, Z., Cao, Y., Huang, L., and Chua, T. (2021, January 1–6). How Knowledge Graph and Attention Help? A Qualitative Analysis into Bag-level Relation Extraction. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Virtual Event.10.18653/v1/2021.acl-long.359 ↗
21. Angeli, G., Johnson, M., and Manning, C.D. (2015, January 26–31). Leveraging Linguistic Structure For Open Domain Information Extraction. Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Beijing, China.10.3115/v1/P15-1034 ↗
22. Bidirectional recurrent neural networks10.1109/78.650093 ↗
23. Bordes, A., Usunier, N., García-Durán, A., Weston, J., and Yakhnenko, O. (2013, January 5–8). Translating Embeddings for Modeling Multi-relational Data. Proceedings of the International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA.
24. Zhu, H., Lin, Y., Liu, Z., Fu, J., Chua, T.S., and Sun, M. (2019). Graph Neural Networks with Generated Parameters for Relation Extraction. arXiv.10.18653/v1/P19-1128 ↗
25. Kipf, T., and Welling, M. (2016). Semi-Supervised Classification with Graph Convolutional Networks. arXiv.
26. Nguyen, D.Q., and Verspoor, K.M. (2018, January 26–29). End-to-end neural relation extraction using deep biaffine attention. Proceedings of the European Conference on Information Retrieval, Grenoble, France.10.1007/978-3-030-15712-8_47 ↗
27. Giorgi, J., Wang, X., Sahar, N., Shin, W.Y., Bader, G.D., and Wang, B. (2019). End-to-end Named Entity Recognition and Relation Extraction using Pre-trained Language Models. arXiv.
28. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., and Funtowicz, M. (2019). HuggingFace’s Transformers: State-of-the-art Natural Language Processing. arXiv.10.18653/v1/2020.emnlp-demos.6 ↗
29. Araci, D. (2019). FinBERT: Financial Sentiment Analysis with Pre-trained Language Models. arXiv.
30. Lerer, A., Wu, L.Y., Shen, J., Lacroix, T., Wehrstedt, L., Bose, A., and Peysakhovich, A. (2019). PyTorch-BigGraph: A Large-scale Graph Embedding System. arXiv.
31. Dozat, T., and Manning, C.D. (2016). Deep Biaffine Attention for Neural Dependency Parsing. arXiv.
32. Riedel, S., Yao, L., and McCallum, A. (2010, January 9–11). Modeling Relations and Their Mentions without Labeled Text. Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases, MIT Stata Center, MA, USA.10.1007/978-3-642-15939-8_10 ↗
33. Han, X., Zhu, H., Yu, P., Wang, Z., Yao, Y., Liu, Z., and Sun, M. (November, January 31). FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.10.18653/v1/D18-1514 ↗
34. Gao, T., Han, X., Zhu, H., Liu, Z., Li, P., Sun, M., and Zhou, J. (2019, January 3–7). FewRel 2.0: Towards More Challenging Few-Shot Relation Classification. Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.10.18653/v1/D19-1649 ↗
35. KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation10.1162/tacl_a_00360 ↗
36. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. arXiv.
37. Bragg, J., Cohan, A., Lo, K., and Beltagy, I. (2021, January 6–14). FLEX: Unifying Evaluation for Few-Shot NLP. Proceedings of the Neural Information Processing Systems, virtual.
38. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
39. Nickel, M., Tresp, V., and Kriegel, H.P. (July, January 28). A Three-Way Model for Collective Learning on Multi-Relational Data. Proceedings of the 28th International Conference on International Conference on Machine Learning, Bellevue, WA, USA.
40. Yang, B., Yih, W.T., He, X., Gao, J., and Deng, L. (2014). Embedding Entities and Relations for Learning and Inference in Knowledge Bases. arXiv.