journal article·2023-07-06Open access

KTAT: A Complex Embedding Model of Knowledge Graph Integrating Type Information and Attention Mechanism

Ying Liu, Peng Wang, Di Yang

Applied Sciences· Volume 13· Issue 13

Abstract

Knowledge graph embedding learning aims to represent the entities and relationships of real-world knowledge as low-dimensional dense vectors. Existing knowledge representation learning methods mostly aggregate only the internal information of triplets and graph structure information. Recent research has proved that multi-source information of entities is conducive to more accurate knowledge embedding tasks. In this paper, we propose a model based on an attention mechanism and integrating the type information of entities, named KTAT. This model is based on the graph attention mechanism, to distribute corresponding attention mechanisms according to different weights between nodes. We introduce a type-specific hyperplane, which enables entities to have different embedding representations according to their type in the current triplet. Simultaneously, we also used textual description information of entities to improve the performance of the model. We conducted link prediction experiments on the FB15k and FB15k-237 datasets. The experimental results show that our model outperforms previous advanced methods compared to some baseline models, and demonstrate that combining type information can effectively improve the performance of link prediction.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Graph Neural Networks · Topic Modeling · Complex Network Analysis Techniques

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Computer Science and Technology, Changchun University of Science and Technology, Changchun 130022, China

School of Computer Science, Tonghua Normal University, Tonghua 134002, China

Changchun University of Science and Technology Chongqing Research Institute, Chongqing 401120, China

Funding

Jilin Provincial Science and Technology Innovation Center for Network Database Application

References (29)

Show source-provided references
1. Krompaβ, D., Baier, S., and Tresp, V. (2015, January 11–15). Type-Constrained Representation Learning in Knowledge Graphs. Proceedings of the International Semantic Web Conference: The Semantic Web, Bethlehem, PA, USA.10.1007/978-3-319-25007-6_37 ↗
2. Xie, R., Liu, Z., and Sun, M. (2016, January 12–17). Representation learning of knowledge graphs with hierarchical types. Proceedings of the National Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10329 ↗
3. Knowledge representation learning with entities, attributes and relations
4. Representation Learning of Knowledge Graphs with Entity Attributes10.1109/ACCESS.2020.2963990 ↗
5. Sun, Z., Hu, W., and Li, C. (2017). International Semantic Web Conference, Springer.
6. Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv.
7. Xie, R., Liu, Z., Jia, J., and Sun, M. (2016, January 12–17). Representation learning of knowledge graphs with entity descriptions. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10329 ↗
8. Translating Embeddings for Modeling Multi-Relational Data
9. Knowledge Graph Embedding by Translating on Hyperplanes
10. Ji, G., He, S., Xu, L., Liu, K., and Zhao, J. (2015, January 26–31). Knowledge Graph Embedding via Dynamic Mapping Matrix. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China.10.3115/v1/P15-1067 ↗
11. Dai, S., Liang, Y., Liu, S., Ying, W., and Shao, W. (2018, January 2–7). Learning Entity and Relation Embeddings with Entity Description for Knowledge Graph Completion. Proceedings of the AAAI Conference on Artificial Intelligence, ICAITA 2018, New Orleans, LA, USA.10.2991/icaita-18.2018.49 ↗
12. Fan, M., Zhou, Q., Chang, E., and Zheng, F. (2014, January 12–14). Transition-Based Knowledge Graph Embedding with Relational Mapping Properties. Proceedings of the 28th Pacific Asia Conference on Language, Information and Computing, Phuket, Thailand.
13. Feng, J., Zhou, M., Hao, Y., Huang, M., and Zhu, X. (2016, January 25–29). Knowledge Graph Embedding by Flexible Translation. Proceedings of the Fifteenth International Conference on the Principles of Knowledge Representation and Reasoning, Cape Town, South Africa.
14. Perozzi, B., Al-Rfou, R., and Skiena, S. (2014, January 24–27). DeepWalk: Online Learning of Social Representations. Proceedings of the KDD’14: 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA.10.1145/2623330.2623732 ↗
15. Tomas, M., Kai, C., Greg, C., and Jeffrey, D. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv.
16. Grover, A., and Leskovec, J. (2016, January 13–17). node2vec: Scalable Feature Learning for Networks. Proceedings of the KDD’16: 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939754 ↗
17. Kipf, T.N., and Welling, M. (2016). Semi-Supervised Classification with Graph Convolutional Networks. arXiv.
18. Schlichtkrull, M., Kipf, T.N., Bloem, P., Berg, R., Titov, I., and Welling, M. (2018). European Semantic Web Conference, Springer.
19. Inductive Representation Learning on Large Graphs
20. Graph Attention Networks
21. Learning knowledge graph embedding with heterogeneous relation attention networks10.1109/TNNLS.2021.3055147 ↗
22. Relational Graph Neural Network with Hierarchical Attention for Knowledge Graph Completion
23. Entity alignment via summary and attribute embeddings10.1093/jigpal/jzac021 ↗
24. Cheng, Y., Liu, Z., Zhao, D., Sun, M., and Chang, E. (2015, January 25–31). Network Representation Learning with Rich Text Information. Proceedings of the International Conference on Artificial Intelligence, Buenos Aires, Argentina.
25. Tu, C., Liu, H., Liu, Z., and Sun, M. (August, January 30). Cane: Context-aware network embedding for relation modeling. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, BC, Canada.
26. SSE: Semantically smooth embedding for knowledge graphs10.1109/TKDE.2016.2638425 ↗
27. Ma, S., Ding, J., Jia, W., Wang, K., and Guo, M. (2017, January 18–22). Transt: Type-based multiple embedding representations for knowledge graph completion. Proceedings of the Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2017, Skopje, North Macedonia.10.1007/978-3-319-71249-9_43 ↗
28. Toutanova, K., and Chen, D. (2015, January 31). Observed versus latent features for knowledge base and text inference. Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, Beijing, China.10.18653/v1/W15-4007 ↗
29. Sun, Z., Deng, Z., Nie, J., and Tang, J. (2019). RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. arXiv.