Adaptive Attentional Network for Few-Shot Relational Learning of Knowledge Graphs
Source abstract
Abstract
Few-shot knowledge graph reasoning is a research focus in the field of knowledge graph reasoning. At present, in order to expand the application scope of knowledge graphs, a large number of researchers are devoted to the study of the multi-shot knowledge graph model. However, as far as we know, the knowledge graph contains a large number of missing relations and entities, and there are not many reference examples at the time of training. In this paper, our goal is to be able to infer the correct entity given a few training instances, or even only one training instance is available. Therefore, we propose an adaptive attentional network for few-shot relational learning of knowledge graphs, extracting knowledge based on traditional embedding methods, using the Transformer mechanism and hierarchical attention mechanism to obtain hidden attributes of entities, and then using a noise checker to filter out unreasonable candidate entities. Our model produces large performance improvements on the NELL-One dataset.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Graph Neural Networks · Topic Modeling · Domain Adaptation and Few-Shot Learning
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Software Technology, Dalian University of Technology, Dalian 116024, China
The Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province, Dalian 116600, China
Funding
National Natural Science Foundation of China · Natural Science Foundation of Liaoning Province · Fundamental Research Funds for the Central Universities
References (36)
Show source-provided references
1. Auer, S., Bizer, C., Kobilarov, G., Lehmann, G., Lehmann, J., Cyganiak, R., and Ives, Z. (2007). DBpedia: A Nucleus for a Web of Open Data. The Semantic Web, Springer.10.1007/978-3-540-76298-0_52 ↗
2. Bollacker, K. (2008, January 9–12). Freebase: A collaboratively created graph database for structuring human knowledge. Proceedings of the 2008 ACM SIGMOD international conference on Management of data, Vancouver, BC, Canada.10.1145/1376616.1376746 ↗
3. Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka, E.R., and Mitchell, T.M. (2010, January 11–15). Toward an architecture for never-ending language learning. Proceedings of the Twenty-Fourth AAAI conference, Atlanta, GA, USA.10.1609/aaai.v24i1.7519 ↗
4. Wu, W., Li, H., Wang, H., and Zhu, K.Q. (2012, January 20–24). Probase: A probabilistic taxonomy for text understanding. Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, Scottsdale, AZ, USA.10.1145/2213836.2213891 ↗
5. Wikidata: A free collaborative knowledgebase10.1145/2629489 ↗
6. Min, B., Grishman, R., Wan, L., Wang, C., and Gondek, D. (2013, January 9–14). Distant supervision for relation extraction with an incomplete knowledge base. Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, GA, USA.
7. Xiong, W., Yu, M., Chang, S., Guo, X., and Wang, W.Y. (2018). One-shot relational learning for knowledge graphs. arXiv.10.18653/v1/D18-1223 ↗
8. Hybrid attention mechanism for few-shot relational learning of knowledge graphs10.1049/cvi2.12066 ↗
9. Chen, M., Zhang, W., Zhang, W., Chen, Q., and Chen, H. (2019). Meta relational learning for few-shot link prediction in knowledge graphs. arXiv.10.18653/v1/D19-1431 ↗
10. Zhang, C., Yao, H., Huang, C., Jiang, M., Li, Z., and Chawla, N.V. (2020, January 7–12). Few-Shot Knowledge Graph Completion. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i03.5698 ↗
11. Knowledge Graph Embedding via Graph Attenuated Attention Networks10.1109/ACCESS.2019.2963367 ↗
12. DT-LET: Deep transfer learning by exploring where to transfer10.1016/j.neucom.2020.01.042 ↗
13. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, 30, Long Beach, CA, USA.
14. Nickel, M., Tresp, V., and Kriegel, H.-P. (2011, January 2–28). A three-way model for collective learning on multi-relational data. Proceedings of the 28th International Conference on International Conference on Machine Learning, Bellevue, WA, USA.
15. Yang, B., Yih, W.-t., He, X., Gao, J., and Deng, L. (2014). Embedding entities and relations for learning and inference in knowledge bases. arXiv.
16. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., and Bouchard, G. (2016, January 19–24). Complex embeddings for simple link prediction. Proceedings of the International conference on machine learning, New York, NY, USA.
17. Sun, Z., Deng, Z.-H., Nie, J.-Y., and Tang, J. (2019). Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv.
18. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., and Yakhnenko, O. (2013, January 5–8). Translating embeddings for modeling multi-relational data. Proceedings of the Advances in Neural Information Processing Systems, 26, Lake Tahoe, NV, USA.
19. Dettmers, T., Minervini, P., Stenetorp, P., and Riedel, S. (2018, January 2–7). Convolutional 2d knowledge graph embeddings. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11573 ↗
20. Jiang, X., Wang, Q., and Wang, B. (2019, January 2–7). Adaptive Convolution for Multi-Relational Learning. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.10.18653/v1/N19-1103 ↗
21. Nguyen, D.Q., Nguyen, T.D., and Phung, D. (2017). A novel embedding model for knowledge base completion based on convolutional neural network. arXiv.10.18653/v1/N18-2053 ↗
22. Schlichtkrull, M., Kipf, T.N., Bloem, P., Van Den Berg, R., Titov, I., and Welling, M. (2018, January 3–7). Modeling relational data with graph convolutional networks. Proceedings of the European Semantic Web Conference, Heraklion, Greece.10.1007/978-3-319-93417-4_38 ↗
23. Joint learning based on multi-shaped filters for knowledge graph completion
24. Koch, G., Zemel, R., and Salakhutdinov, R. (2015). Siamese neural networks for one-shot image recognition. ICML Deep Learning Workshop, ICML.
25. Vinyals, O., Blundell, C., Lillicrap, T., and Wierstra, D. (2016, January 5–10). Matching networks for one shot learning. Proceedings of the Advances in Neural Information Processing Systems, 29, Barcelona, Spain.
26. Gao, T., Han, X., Liu, Z., and Sun, M. (February, January 27). Hybrid Attention-Based Prototypical Networks for Noisy Few-Shot Relation Classification. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
27. Li, W., Wang, L., Xu, J., Huo, J., Gao, Y., and Luo, J. (2019, January 15). Revisiting Local Descriptor Based Image-To-Class Measure for Few-Shot Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00743 ↗
28. Yang, L., Li, L., Zhang, Z., Zhou, X., Zhou, E., and Liu, Y. (2020, January 14–19). DPGN: Distribution Propagation Graph Network for Few-Shot Learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01340 ↗
29. Ravi, S., and Larochelle, H. (2017, January 24–26). Optimization as a model for few-shot learning. Proceedings of the5th International Conference on Learning Representations, Toulon, France.
30. Finn, C., Abbeel, P., and Levine, S. (2017, January 6–11). Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of the International conference on machine learning, Sydney, Australia.
31. Lee, Y., and Choi, S. (2018, January 10–15). Gradient-based meta-learning with learned layerwise metric and subspace. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden.
32. VSSA-NET: Vertical Spatial Sequence Attention Network for Traffic Sign Detection10.1109/TIP.2019.2896952 ↗
33. Lin, X.V., Socher, R., and Xiong, C. (2018). Multi-hop knowledge graph reasoning with reward shaping. arXiv.10.18653/v1/D18-1362 ↗
34. Han, K., Xiao, A., Wu, E., Guo, J., Xu, C., and Wang, Y. (2020, January 6–12). Transformer in transformer. Proceedings of the Advances in Neural Information Processing Systems, 34, Online Event, Vancouver, BC, Canada.
35. Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R.R. (2012). Improving neural networks by preventing co-adaptation of feature detectors. arXiv.
36. Never-ending learning10.1145/3191513 ↗