Cosine-Based Embedding for Completing Lightweight Schematic Knowledge in DL-Litecore
Source abstract
Abstract
Schematic knowledge, an important component of knowledge graphs (KGs), defines a rich set of logical axioms based on concepts and relations to support knowledge integration, reasoning, and heterogeneity elimination over KGs. Although several KGs consist of lots of factual knowledge, their schematic knowledge (e.g., subclassOf axioms, disjointWith axioms) is far from complete. Currently, existing KG embedding methods for completing schematic knowledge still suffer from two limitations. Firstly, existing embedding methods designed to encode factual knowledge pay little attention to the completion of schematic knowledge (e.g., axioms). Secondly, several methods try to preserve logical properties of relations for completing schematic knowledge, but they cannot simultaneously preserve the transitivity (e.g., subclassOf) and symmetry (e.g., disjointWith) of axioms well. To solve these issues, we propose a cosine-based embedding method named CosE tailored for completing lightweight schematic knowledge in DL-Litecore. Precisely, the concepts in axioms will be encoded into two semantic spaces defined in CosE. One is called angle-based semantic space, which is employed to preserve the transitivity or symmetry of relations in axioms. The other one is defined as translation-based semantic space that is used to measure the confidence of each axiom. We design two types of score functions for these two semantic spaces, so as to sufficiently learn the vector representations of concepts. Moreover, we propose a novel negative sampling strategy based on the mutual exclusion between subclassOf and disjointWith. In this way, concepts can obtain better vector representations for schematic knowledge completion. We implement our method and verify it on four standard datasets generated by real ontologies. Experiments show that CosE can obtain better results than existing models and keep the logical properties of relations for transitivity and symmetry simultaneously.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Graph Neural Networks · Topic Modeling · Semantic Web and Ontologies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Modern Posts, Nanjing University of Posts and Telecommunications, Nanjing 210003, China
Key Laboratory of Computer Network and Information Integration, Southeast University, Ministry of Education, Nanjing 211189, China
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210093, China
School of Computer Science, The University of Auckland, Auckland 1010, New Zealand
School of Computer Science and Engineering, Southeast University, Nanjing 211189, China
Intel Joint Research Institute on Intelligent Edge Computing, Nanjing 211135, China
Funding
Natural Science Foundation of China · Foundation of Jiangsu Provincial Double-Innovation Doctor Program · NUPTSF
References (57)
Show source-provided references
1. Hybrid reasoning in knowledge graphs: Combing symbolic reasoning and statistical reasoning10.3233/SW-190375 ↗
2. Fellbaum, C. (2010). WordNet, Springer.10.1007/978-90-481-8847-5_10 ↗
3. DBpedia—A large-scale, multilingual knowledge base extracted from Wikipedia10.3233/SW-140134 ↗
4. YAGO: A Large Ontology from Wikipedia and WordNet10.1016/j.websem.2008.06.001 ↗
5. Wang, M., Wang, R., Liu, J., Chen, Y., Zhang, L., and Qi, G. (2018, January 8–12). Towards Empty Answers in SPARQL: Approximating Querying with RDF Embedding. Proceedings of the 17th International Semantic Web Conference, Monterey, CA, USA.10.1007/978-3-030-00671-6_30 ↗
6. RDF2Vec: RDF graph embeddings and their applications10.3233/SW-180317 ↗
7. Sun, Z., Zhang, Q., Hu, W., Wang, C., Chen, M., Akrami, F., and Li, C. (September, January 31). A Benchmarking Study of Embedding-based Entity Alignment for Knowledge Graphs. Proceedings of the 46th International Conference on Very Large Data Bases, Tokyo, Japan.
8. Weston, J., Bordes, A., Yakhnenko, O., and Usunier, N. (2013, January 18–21). Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Washington, DC, USA.
9. Knowledge Graph Embedding: A Survey of Approaches and Applications10.1109/TKDE.2017.2754499 ↗
10. Dai, Y., Wang, S., Xiong, N.N., and Guo, W. (2020). A Survey on Knowledge Graph Embedding: Approaches, Applications and Benchmarks. Electronics, 9.10.3390/electronics9050750 ↗
11. A review: Knowledge reasoning over knowledge graph10.1016/j.eswa.2019.112948 ↗
12. Knowledge Graph Embedding for Link Prediction: A Comparative Analysis
13. Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., and Yakhnenko, O. (2013, January 5–8). Translating Embeddings for Modeling Multi-relational Data. Proceedings of the 27th Annual Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA.
14. Wang, Z., Zhang, J., Feng, J., and Chen, Z. (2014, January 27–31). Knowledge Graph Embedding by Translating on Hyperplanes. Proceedings of the 28th AAAI Conference on Artificial Intelligence, Québec City, QC, Canada.10.1609/aaai.v28i1.8870 ↗
15. Lin, Y., Liu, Z., Sun, M., Liu, Y., and Zhu, X. (2015, January 25–30). Learning Entity and Relation Embeddings for Knowledge Graph Completion. Proceedings of the 29th AAAI Conference on Artificial Intelligence, Austin, TX, USA.10.1609/aaai.v29i1.9491 ↗
16. Nickel, M., Tresp, V., and Kriegel, H.P. (July, January 28). A Three-Way Model for Collective Learning on Multi-Relational Data. Proceedings of the 28th International Conference on Machine Learning, Bellevue, WA, USA.
17. Yang, B., Yih, S.W.T., He, X., Gao, J., and Deng, L. (2015, January 7–9). Embedding Entities and Relations for Learning and Inference in Knowledge Bases. Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA.
18. Nickel, M., Rosasco, L., and Poggio, T. (2016, January 12–17). Holographic Embeddings of Knowledge Graphs. Proceedings of the 30th AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10314 ↗
19. Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., and Bouchard, G. (2016, January 19–24). Complex Embeddings for Simple Link Prediction. Proceedings of the 33nd International Conference on Machine Learning, New York, NY, USA.
20. Diaz, G.I., Fokoue, A., and Sadoghi, M. (2018, January 26–29). EmbedS: Scalable, Ontology-aware Graph Embeddings. Proceedings of the 21st International Conference on Extending Database Technology, Vienna, Austria.
21. Lv, X., Hou, L., Li, J., and Liu, Z. (November, January 31). Differentiating Concepts and Instances for Knowledge Graph Embedding. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.10.18653/v1/D18-1222 ↗
22. Zhang, Z., Cai, J., Zhang, Y., and Wang, J. (2020, January 7–12). Learning Hierarchy-Aware Knowledge Graph Embeddings for Link Prediction. Proceedings of the 34th AAAI Conference on Artificial Intelligence, New York, NY, USA.10.1609/aaai.v34i03.5701 ↗
23. Sun, Z., Deng, Z.H., Nie, J.Y., and Tang, J. (2019, January 6–9). RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. Proceedings of the 7th International Conference on Learning Representations, Los Angeles, CA, USA.
24. Kulmanov, M., Liu-Wei, W., Yan, Y., and Hoehndorf, R. (2019, January 10–16). EL Embeddings: Geometric Construction of Models for the Description Logic EL++. Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China.10.24963/ijcai.2019/845 ↗
25. OWL2Vec*: Embedding of OWL ontologies10.1007/s10994-021-05997-6 ↗
26. Calvanese, D., De Giacomo, G., Lembo, D., Lenzerini, M., and Rosati, R. (2005, January 9–13). DL-Lite: Tractable Description Logics for Ontologies. Proceedings of the 20th National Conference on Artificial Intelligence, Pittsburgh, PA, USA.
27. Graph-based approaches to debugging and revision of terminologies in DL-Lite10.1016/j.knosys.2016.01.039 ↗
28. Ji, G., He, S., Xu, L., Liu, K., and Zhao, J. (2015, January 26–31). Knowledge Graph Embedding via Dynamic Mapping Matrix. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics, Beijing, China.10.3115/v1/P15-1067 ↗
29. Xiao, H., Huang, M., Hao, Y., and Zhu, X. (2015). TransA: An Adaptive Approach for Knowledge Graph Embedding. arXiv.10.18653/v1/P16-1219 ↗
30. Dong, X., Gabrilovich, E., Heitz, G., Horn, W., Lao, N., Murphy, K., Strohmann, T., Sun, S., and Zhang, W. (2014, January 24–27). Knowledge vault: A web-scale approach to probabilistic knowledge fusion. Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA.10.1145/2623330.2623623 ↗
31. Liu, Q., Jiang, H., Evdokimov, A., Ling, Z.H., Zhu, X., Wei, S., and Hu, Y. (2016). Probabilistic Reasoning via Deep Learning: Neural Association Models. arXiv.
32. Schlichtkrull, M., Kipf, T.N., Bloem, P., Berg, R.V.D., Titov, I., and Welling, M. (2018, January 3–7). Modeling Relational Data with Graph Convolutional Networks. Proceedings of the 15th Extended Semantic Web Conference, Heraklion, Crete, Greece.10.1007/978-3-319-93417-4_38 ↗
33. Shi, B., and Weninger, T. (2017, January 4–9). ProjE: Embedding Projection for Knowledge Graph Completion. Proceedings of the 31st AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.10.1609/aaai.v31i1.10677 ↗
34. Dettmers, T., Minervini, P., Stenetorp, P., and Riedel, S. (2018, January 2–7). Convolutional 2D Knowledge Graph Embeddings. Proceedings of the 32nd AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11573 ↗
35. Zhang, W., Chen, J., Li, J., Xu, Z., Pan, J.Z., and Chen, H. (2022). Knowledge Graph Reasoning with Logics and Embeddings: Survey and Perspective. arXiv.
36. Gutiérrez-Basulto, V., and Schockaert, S. (November, January 30). From Knowledge Graph Embedding to Ontology Embedding? An Analysis of the Compatibility between Vector Space Representations and Rules. Proceedings of the Sixteenth International Conference on Principles of Knowledge Representation and Reasoning., Tempe, AZ, USA.
37. Guo, S., Wang, Q., Wang, L., Wang, B., and Guo, L. (2016, January 1–4). Jointly Embedding Knowledge Graphs and Logical Rules. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, TX, USA.10.18653/v1/D16-1019 ↗
38. Guo, S., Wang, Q., Wang, L., Wang, B., and Guo, L. (2018, January 2–7). Knowledge Graph Embedding With Iterative Guidance From Soft Rules. Proceedings of the 32nd AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11918 ↗
39. Zhang, W., Paudel, B., Wang, L., Chen, J., Zhu, H., Zhang, W., Bernstein, A., and Chen, H. (2019, January 13–17). Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning. Proceedings of the World Wide Web Conference, San Francisco, CA, USA.10.1145/3308558.3313612 ↗
40. Chen, M., Tian, Y., Chen, X., Xue, Z., and Zaniolo, C. (2018, January 3–5). On2Vec: Embedding-based Relation Prediction for Ontology Population. Proceedings of the 2018 SIAM International Conference on Data Mining, San Diego, CA, USA.10.1137/1.9781611975321.36 ↗
41. Garg, D., Ikbal, S., Srivastava, S.K., Vishwakarma, H., Karanam, H., and Subramaniam, L.V. (2019, January 8–14). Quantum Embedding of Knowledge for Reasoning. Proceedings of the Annual Conference on Neural Information Processing Systems, Vancouver, BC, Canada.
42. Staab, S., and Studer, R. (2009). Handbook on Ontologies, Springer. [2nd ed.].10.1007/978-3-540-92673-3 ↗
43. Pushing the envelope: Challenges in a frame-based representation of human anatomy10.1016/j.datak.2003.06.002 ↗
44. FoodOn: A harmonized food ontology to increase global food traceability, quality control and data integration10.1038/s41538-018-0032-6 ↗
45. Dragoni, M., Bailoni, T., Maimone, R., and Eccher, C. (2018, January 8–12). HeLiS: An Ontology for Supporting Healthy Lifestyles. Proceedings of the 17th International Semantic Web Conference, Monterey, Monterey, CA, USA.10.1007/978-3-030-00668-6_4 ↗
46. Schema induction from incomplete semantic data10.3233/IDA-173514 ↗
47. Liu, H., Wu, Y., and Yang, Y. (2017, January 6–11). Analogical Inference for Multi-relational Embeddings. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia.
48. Han, X., Cao, S., Lv, X., Lin, Y., Liu, Z., Sun, M., and Li, J. (November, January 31). OpenKE: An Open Toolkit for Knowledge Embedding. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.10.18653/v1/D18-2024 ↗
49. Gao, H., Zheng, X., Li, W., Qi, G., and Wang, M. (2019, January 9–14). Cosine-Based Embedding for Completing Schematic Knowledge. Proceedings of the 8th CCF International Conference on Natural Language Processing and Chinese Computing, Dunhuang, China.10.1007/978-3-030-32233-5_20 ↗
50. Schlobach, S., and Cornet, R. (2003, January 9–15). Non-Standard Reasoning Services for the Debugging of Description Logic Terminologies. Proceedings of the 18th International Joint Conference on Artificial Intelligence, Acapulco, Mexico.
51. Linking Data to Ontologies
52. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2–7). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA.
53. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Sastry, G., Askell, A., and Amodei, D. (2020, January 6–12). Language Models are Few-Shot Learners. Proceedings of the Annual Conference on Neural Information Processing Systems, Vancouver, BC, Canada.
54. Tay, Y., Dehghani, M., Bahri, D., and Metzler, D. (ACM Comput. Surv., 2022). Efficient Transformers: A Survey, ACM Comput. Surv., in press.10.1145/3530811 ↗
55. Xiao, G., Calvanese, D., Kontchakov, R., Lembo, D., Poggi, A., Rosati, R., and Zakharyaschev, M. (2018, January 2–7). Ontology-Based Data Access: A Survey. Proceedings of the 32nd AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.24963/ijcai.2018/777 ↗
56. Ontology matching: A literature review10.1016/j.eswa.2014.08.032 ↗
57. Knowledge graph refinement: A survey of approaches and evaluation methods10.3233/SW-160218 ↗