Commonsense-Guided Inductive Relation Prediction with Dual Attention Mechanism
Source abstract
Abstract
The inductive relation prediction of knowledge graphs, as an important research topic, aims at predicting the missing relation between unknown entities with many real-world applications. Existing approaches toward this problem mostly use enclosing subgraphs to extract the features of target nodes to make predictions; however, there is a tendency to ignore the neighboring relations outside the enclosing subgraph, thus leading to inaccurate predictions. In addition, they also neglect the rich commonsense information that can help filter out less convincing results. In order to address the above issues, this paper proposes a commonsense-guided inductive relation prediction method with a dual attention mechanism called CNIA. Specifically, in addition to the enclosing subgraph, we added the multi-hop neighboring relations of target nodes, thereby forming a neighbor-enriched subgraph where the initial embeddings are generated. Next, we obtained the subgraph representations with a dual attention (i.e., edge-aware and relation-aware) mechanism, as well as the neighboring relational path embeddings. Then, we concatenated the two embeddings before feeding them into the supervised learning model. A commonsense re-ranking mechanism was introduced to filter the results that conformed to commonsense. Extensive experiments on WN18RR, FB15k-237, and NELL995 showed that CNIA achieves better prediction results when compared to the state-of-the-art models. The results suggested that our proposed model can be considered as an effective and state-of-the-art solution for inductive relation prediction.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Graph Neural Networks · Domain Adaptation and Few-Shot Learning · Topic Modeling
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Laboratory for Big Data and Decision, National University of Defense Technology, Changsha 410072, China
Funding
NSFC
References (35)
Show source-provided references
1. Zhang, Y., Dai, H., Kozareva, Z., Smola, A.J., and Song, L. (2018, January 2–7). Variational Reasoning for Question Answering With Knowledge Graph. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI 2018), New Orleans, LA, USA.10.1609/aaai.v32i1.12057 ↗
2. Verlinden, S., Zaporojets, K., Deleu, J., Demeester, T., and Develder, C. (2021). Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Association for Computational Linguistics.
3. Wang, H., Zhao, M., Xie, X., Li, W., and Guo, M. (2019, January 13–17). Knowledge Graph Convolutional Networks for Recommender Systems. Proceedings of the WWW 2019, San Francisco, CA, USA.10.1145/3308558.3313417 ↗
4. Zhao, X., Zeng, W., and Tang, J. (2023). Entity Alignment—Concepts, Recent Advances and Novel Approaches, Springer.10.1007/978-981-99-4250-3 ↗
5. On entity alignment at scale10.1007/s00778-021-00703-3 ↗
6. Bollacker, K.D., Evans, C., Paritosh, P.K., Sturge, T., and Taylor, J. (2008, January 10–12). Freebase: A collaboratively created graph database for structuring human knowledge. Proceedings of the SIGMOD Conference 2008, Vancouver, BC, Canada.10.1145/1376616.1376746 ↗
7. Vrandecic, D. (2012, January 16–20). Wikidata: A new platform for collaborative data collection. Proceedings of the WWW 2012, Lyon, France.10.1145/2187980.2188242 ↗
8. Bordes, A., Usunier, N., García-Durán, A., Weston, J., and Yakhnenko, O. (2013, January 5–10). Translating Embeddings for Modeling Multi-relational Data. Proceedings of the NIPS 2013, Lake Tahoe, NV, USA.
9. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., and Bouchard, G. (2016, January 19–24). Complex Embeddings for Simple Link Prediction. Proceedings of the 33rd International Conference on International Conference on Machine Learning (ICML 2016), New York, NY, USA.
10. Modeling Relational Data with Graph Convolutional Networks10.1007/978-3-319-93417-4_38 ↗
11. Vashishth, S., Sanyal, S., Nitin, V., and Talukdar, P.P. (2020, January 26–30). Composition-based Multi-Relational Graph Convolutional Networks. Proceedings of the ICLR 2020, Addis Ababa, Ethiopia.
12. Complete feature learning and consistent relation modeling for few-shot knowledge graph completion10.1016/j.eswa.2023.121725 ↗
13. Lin, Q., Liu, J., Xu, F., Pan, Y., Zhu, Y., Zhang, L., and Zhao, T. (2022, January 11–15). Incorporating Context Graph with Logical Reasoning for Inductive Relation Prediction. Proceedings of the SIGIR 2022, Madrid, Spain.10.1145/3477495.3531996 ↗
14. Yang, F., Yang, Z., and Cohen, W.W. (2017, January 4–9). Differentiable Learning of Logical Rules for Knowledge Base Reasoning. Proceedings of the NIPS 2017, Long Beach, CA, USA.
15. Teru, K.K., Denis, E.G., and Hamilton, W.L. (2020, January 13–18). Inductive Relation Prediction by Subgraph Reasoning. Proceedings of the ICML 2020, Virtual.
16. Chen, J., He, H., Wu, F., and Wang, J. (2021, January 2–9). Topology-Aware Correlations Between Relations for Inductive Link Prediction in Knowledge Graphs. Proceedings of the AAAI 2021, Virtual.
17. Mai, S., Zheng, S., Yang, Y., and Hu, H. (2021, January 2–9). Communicative Message Passing for Inductive Relation Reasoning. Proceedings of the AAAI 2021, Virtual.
18. Galkin, M., Denis, E.G., Wu, J., and Hamilton, W.L. (2022, January 25–29). NodePiece: Compositional and Parameter-Efficient Representations of Large Knowledge Graphs. Proceedings of the ICLR 2022, Virtual.
19. Baek, J., Lee, D.B., and Hwang, S.J. (2020, January 6–12). Learning to Extrapolate Knowledge: Transductive Few-shot Out-of-Graph Link Prediction. Proceedings of the NeurIPS 2020, Virtual.
20. Zhang, Y., Wang, W., Chen, W., Xu, J., Liu, A., and Zhao, L. (2021, January 1–5). Meta-Learning Based Hyper-Relation Feature Modeling for Out-of-Knowledge-Base Embedding. Proceedings of the CIKM 2021, Gold Coast, QLD, Australia.10.1145/3459637.3482367 ↗
21. Xu, X., Zhang, P., He, Y., Chao, C., and Yan, C. (2022, January 23–29). Subgraph Neighboring Relations Infomax for Inductive Link Prediction on Knowledge Graphs. Proceedings of the IJCAI 2022, Vienna, Austria.10.24963/ijcai.2022/325 ↗
22. Zeng, W., Zhao, X., Tang, J., and Lin, X. (2020, January 20–24). Collective Entity Alignment via Adaptive Features. Proceedings of the 36th IEEE International Conference on Data Engineering, ICDE 2020, Dallas, TX, USA.10.1109/ICDE48307.2020.00191 ↗
23. Reinforcement Learning-based Collective Entity Alignment with Adaptive Features10.1145/3446428 ↗
24. GLANet: Temporal knowledge graph completion based on global and local information-aware network10.1007/s10489-023-04481-z ↗
25. Multi-hop path reasoning over sparse temporal knowledge graphs based on path completion and reward shaping10.1016/j.ipm.2023.103605 ↗
26. Multi-Level Interaction Based Knowledge Graph Completion10.1109/TASLP.2023.3331121 ↗
27. Sadeghian, A., Armandpour, M., Ding, P., and Wang, D.Z. (2019, January 8–14). DRUM: End-To-End Differentiable Rule Mining on Knowledge Graphs. Proceedings of the NeurIPS 2019, Vancouver, BC, Canada.
28. Li, J., Wang, Q., and Mao, Z. (2023, January 4–10). Inductive Relation Prediction from Relational Paths and Context with Hierarchical Transformers. Proceedings of the ICASSP 2023, Rhodes Island, Greece.10.1109/ICASSP49357.2023.10096502 ↗
29. Geng, Y., Chen, J., Pan, J.Z., Chen, M., Jiang, S., Zhang, W., and Chen, H. (2023, January 3–7). Relational Message Passing for Fully Inductive Knowledge Graph Completion. Proceedings of the ICDE 2023, Anaheim, CA, USA.10.1109/ICDE55515.2023.00098 ↗
30. CYC: A Large-Scale Investment in Knowledge Infrastructure10.1145/219717.219745 ↗
31. Suchanek, F.M., Kasneci, G., and Weikum, G. (2007, January 8–12). Yago: A core of semantic knowledge. Proceedings of the WWW 2007, Banff, AB, Canada.10.1145/1242572.1242667 ↗
32. Song, Y., Zheng, S., Niu, Z., Fu, Z., Lu, Y., and Yang, Y. (2021, January 7–15). Communicative Representation Learning on Attributed Molecular Graphs. Proceedings of the IJCAI 2020, Yokohama, Japan.10.24963/ijcai.2020/392 ↗
33. Dettmers, T., Minervini, P., Stenetorp, P., and Riedel, S. (2018, January 2–7). Convolutional 2D Knowledge Graph Embeddings. Proceedings of the AAAI 2018, New Orleans, LO, USA.10.1609/aaai.v32i1.11573 ↗
34. Toutanova, K., Chen, D., Pantel, P., Poon, H., Choudhury, P., and Gamon, M. (2015, January 17–21). Representing Text for Joint Embedding of Text and Knowledge Bases. Proceedings of the EMNLP 2015, Lisbon, Portugal.10.18653/v1/D15-1174 ↗
35. Xiong, W., Hoang, T., and Wang, W.Y. (2017, January 7–11). DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning. Proceedings of the EMNLP 2017, Copenhagen, Denmark.10.18653/v1/D17-1060 ↗