Translation-Based Embeddings with Octonion for Knowledge Graph Completion
Source abstract
Abstract
Knowledge representation learning achieves the automatic completion of knowledge graphs (KGs) by embedding entities into continuous low-dimensional vector space. In knowledge graph completion (KGC) tasks, the inter-dependencies and hierarchical information in KGs have gained attention. Existing methods do not well capture the latent dependencies between all components of entities and relations. To address this, we introduce the mathematical theories of octonion, a more expressive generalized form of complex number and quaternion, and propose a translation-based KGC model with octonion (TransO). TransO models entities as octonion coordinate vectors, relations as the combination of octonion component matrices and coordinate vectors, and uses specific grouping calculation rules to interact between entities and relations. In addition, since hyperbolic Poincaré space in non-Euclidean mathematics can represent hierarchical data more accurately and effectively than traditional Euclidean space, we propose a Poincaré-extended TransO model (PTransO). PTransO transforms octonion coordinate vectors into hyperbolic embeddings by exponential mapping, and integrates the Euclidean-based calculations into hyperbolic space by operations such as Möbius addition and hyperbolic distance. The experimental results of link prediction indicate that TransO outperforms other translation-based models on the WN18 benchmark, and PTransO further achieves state-of-the-art performance in low-dimensional space on the well-established WN18RR and FB15k-237 benchmarks.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Graph Neural Networks · Topic Modeling · Complex Network Analysis Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Intelligence and Computing, Tianjin University, Tianjin 300350, China
Tianjin Key Laboratory of Advanced Networking (TANKLab), Tianjin University, Tianjin 300350, China
Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin University, Tianjin 300350, China
Foreign Language, Literature and Culture Studies Center, Tianjin Foreign Studies University, Tianjin 300204, China
Funding
National Natural Science Foundation of China
References (46)
Show source-provided references
1. Yang, X., Huan, Z., Zhai, Y., and Lin, T. (2021). Research of Personalized Recommendation Technology Based on Knowledge Graphs. Appl. Sci., 11.10.3390/app11157104 ↗
2. (2012, May 16). Introducing the Knowledge Graph: Things, Not Strings. Available online: https://www.blog.google/products/search/introducing-knowledge-graph-things-not/.
3. Xiong, C., Power, R., and Callan, J. (2017, January 3–7). Explicit Semantic Ranking for Academic Search via Knowledge Graph Embedding. Proceedings of the 26th International Conference on World Wide Web, Perth, Australia.10.1145/3038912.3052558 ↗
4. Wang, X., He, X., Cao, Y., Liu, M., and Chua, T.-S. (2019, January 4–8). KGAT: Knowledge Graph Attention Network for Recommendation. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA.10.1145/3292500.3330989 ↗
5. Saha, A., Pahuja, V., Khapra, M.M., Sankaranarayanan, K., and Chandar, S. (2018, January 2–7). Complex Sequential Question Answering: Towards Learning to Converse Over Linked Question Answer Pairs with a Knowledge Graph. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11332 ↗
6. Song, H.-J., Kim, A.-Y., and Park, S.-B. (2020). Learning Translation-Based Knowledge Graph Embeddings by N-Pair Translation Loss. Appl. Sci., 10.10.3390/app10113964 ↗
7. Choi, S.J., Song, H.-J., and Park, S.-B. (2020). An Approach to Knowledge Base Completion by a Committee-Based Knowledge Graph Embedding. Appl. Sci., 10.10.3390/app10082651 ↗
8. A Survey on Knowledge Graphs: Representation, Acquisition, and Applications10.1109/TNNLS.2021.3070843 ↗
9. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., and Yakhnenko, O. (2013). Translating Embeddings for Modeling Multi-relational Data. Advances in Neural Information Processing Systems 26 (NIPS 2013), Curran Associates, Inc.
10. Nickel, M., Tresp, V., and Kriegel, H.-P. (July, January 28). A Three-Way Model for Collective Learning on Multi-Relational Data. Proceedings of the 28th International Conference on Machine Learning, Bellevue, WA, USA.
11. Dettmers, T., Minervini, P., Stenetorp, P., and Riedel, S. (2018, January 2–7). Convolutional 2D Knowledge Graph Embeddings. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11573 ↗
12. Nguyen, D.Q., Nguyen, T.D., Nguyen, D.Q., and Phung, D. (2018, January 1–6). A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, LA, USA.10.18653/v1/N18-2053 ↗
13. Popa, C.A. (2016, January 6–9). Octonion-Valued Neural Networks. Proceedings of the 25th International Conference on Artificial Neural Networks, Barcelona, Spain.10.1007/978-3-319-44778-0_51 ↗
14. Color Palmprint Feature Extraction and Recognition Algorithm Based on Octonion
15. Matrix representation of octonions and generalizations10.1063/1.532950 ↗
16. Algorithm for multiplying two octonions10.3103/S0735272712100056 ↗
17. The Real Representation of Octonion Vector and Matrix
18. Hyperbolic Trigonometry and its Application in the Poincaré Ball Model of Hyperbolic Geometry10.1016/S0898-1221(01)85012-4 ↗
19. Balažević, I., Allen, C., and Hospedales, T. (2019). Multi-relational Poincaré Graph Embeddings. Advances in Neural Information Processing Systems 32 (NeurIPS 2019), Curran Associates, Inc.
20. Chami, I., Wolf, A., Juan, D.-C., Sala, F., Ravi, S., and Ré, C. (2020, January 5–10). Low-Dimensional Hyperbolic Knowledge Graph Embeddings. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.10.18653/v1/2020.acl-main.617 ↗
21. Wang, Z., Zhang, J., Feng, J., and Chen, Z. (2014, January 27–31). Knowledge Graph Embedding by Translating on Hyperplanes. Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, Québec City, QC, Canada.10.1609/aaai.v28i1.8870 ↗
22. Lin, Y., Liu, Z., Sun, M., Liu, Y., and Zhu, X. (2015, January 25–30). Learning Entity and Relation Embeddings for Knowledge Graph Completion. Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, USA.10.1609/aaai.v29i1.9491 ↗
23. Ji, G., He, S., Xu, L., Liu, K., and Zhao, J. (2015, January 26–31). Knowledge Graph Embedding via Dynamic Mapping Matrix. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China.10.3115/v1/P15-1067 ↗
24. Nguyen, D.Q., Sirts, K., Qu, L., and Johnson, M. (2016, January 12–17). STransE: A novel embedding model of entities and relationships in knowledge bases. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA.10.18653/v1/N16-1054 ↗
25. Ji, G., Liu, K., He, S., and Zhao, J. (2016, January 12–17). Knowledge Graph Completion with Adaptive Sparse Transfer Matrix. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10089 ↗
26. Qian, W., Fu, C., Zhu, Y., Cai, D., and He, X. (2018, January 13–19). Translating Embeddings for Knowledge Graph Completion with Relation Attention Mechanism. Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, Stockholm, Sweden.10.24963/ijcai.2018/596 ↗
27. Yang, B., Yih, W.-T., He, X., Gao, J., and Deng, L. (2014, January 14–16). Embedding Entities and Relations for Learning and Inference in Knowledge Bases. Proceedings of the 2nd International Conference on Learning Representations, Banff, AB, Canada.
28. Kazemi, S.M., and Poole, D. (2018). SimplE Embedding for Link Prediction in Knowledge Graphs. Advances in Neural Information Processing Systems 31 (NeurIPS 2018), Curran Associates, Inc.
29. Balažević, I., Allen, C., and Hospedales, T.M. (2019, January 3–7). TuckER: Tensor Factorization for Knowledge Graph Completion. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China.10.18653/v1/D19-1522 ↗
30. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., and Bouchard, G. (2016, January 20–22). Complex Embeddings for Simple Link Prediction. Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA.
31. Sun, Z., Deng, Z., Nie, J., and Tang, J. (2019, January 6–9). RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. Proceedings of the 7th International Conference on Learning Representations, New Orleans, LA, USA.
32. Zhang, S., Tay, Y., Yao, L., and Liu, Q. (2019). Quaternion Knowledge Graph Embeddings. Advances in Neural Information Processing Systems 32 (NeurIPS 2019), Curran Associates, Inc.
33. Nguyen, D.Q., Vu, T., Nguyen, T.D., Nguyen, D.Q., and Phung, D. (2019, January 2–7). A Capsule Network-based Embedding Model for Knowledge Graph Completion and Search Personalization. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.10.18653/v1/N19-1226 ↗
34. Vashishth, S., Sanyal, S., Nitin, V., Agrawal, N., and Talukdar, P. (2020, January 7–12). InteractE: Improving Convolution-based Knowledge Graph Embeddings by Increasing Feature Interactions. Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i03.5694 ↗
35. Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., and Welling, M. (2018, January 3–7). Modeling Relational Data with Graph Convolutional Networks. Proceedings of the 15th European Semantic Web Conference, Heraklion, Greece.10.1007/978-3-319-93417-4_38 ↗
36. Subforms of norm forms of octonion fields10.1007/s00013-017-1129-x ↗
37. On quaternions and octonions: Their geometry, arithmetic, and symmetry10.1007/BF02985662 ↗
38. Quaternions and octonions in Mechanics
39. Loshchilov, I., and Hutter, F. (2019, January 6–9). Decoupled Weight Decay Regularization. Proceedings of the 7th International Conference on Learning Representations, New Orleans, LA, USA.
40. Bottou, L. (2010, January 22–27). Large-Scale Machine Learning with Stochastic Gradient Descent. Proceedings of the 19th International Conference on Computational Statistics, Paris, France.
41. Li, K., Gu, S., and Yan, D. (2021). A Link Prediction Method Based on Neural Networks. Appl. Sci., 11.10.3390/app11115186 ↗
42. Toutanova, K., and Chen, D. (2015, January 31). Observed versus latent features for knowledge base and text inference. Proceedings of the 3rd Workshop on Continuous Vector Space Models and their Compositionality, Beijing, China.10.18653/v1/W15-4007 ↗
43. Introduction to WordNet: An On-line Lexical Database10.1093/ijl/3.4.235 ↗
44. Bollacker, K.D., Evans, C., Paritosh, P.K., Sturge, T., and Taylor, J. (2008, January 10–12). Freebase: A collaboratively created graph database for structuring human knowledge. Proceedings of the ACM SIGMOD International Conference on Management of Data, Vancouver, BC, Canada.10.1145/1376616.1376746 ↗
45. Lacroix, T., Usunier, N., and Obozinski, G. (2018, January 10–15). Canonical Tensor Decomposition for Knowledge Base Completion. Proceedings of the 35th International Conference on Machine Learning, Stockholmsmässan, Stockholm, Sweden.
46. Zhang, Y., Yao, Q., Shao, Y., and Chen, L. (2019, January 8–11). NSCaching: Simple and Efficient Negative Sampling for Knowledge Graph Embedding. Proceedings of the 35th IEEE International Conference on Data Engineering, Macao, China.10.1109/ICDE.2019.00061 ↗