DRKG: Faithful and Interpretable Multi-Hop Knowledge Graph Question Answering via LLM-Guided Reasoning Plans
Source abstract
Abstract
Multi-Hop Knowledge Graph Question Answering (multi-hop KGQA) aims to obtain answers by analyzing the semantics of natural language questions and performing multi-step reasoning across multiple entities and relations in knowledge graphs. Traditional embedding-based methods map natural language questions and knowledge graphs into vector spaces for answer matching through vector operations. While these approaches have improved model performance, they face two critical challenges: the lack of clear interpretability caused by implicit reasoning mechanisms, and the semantic gap between natural language queries and structured knowledge representations. This study proposes the DRKG (Decomposed Reasoning over Knowledge Graph), a constrained multi-hop reasoning framework based on large language models (LLMs) that introduces explicit reasoning plans as logical boundary controllers. The innovation of the DRKG lies in two key aspects: First, the DRKG generates hop-constrained reasoning plans through semantic parsing based on LLMs, explicitly defining the traversal path length and entity-retrieval logic in knowledge graphs. Second, the DRKG conducts selective retrieval during knowledge graph traversal based on these reasoning plans, ensuring faithfulness to structured knowledge. We evaluate the DRKG on four datasets, and the experimental results demonstrate that the DRKG achieves 1%–5% accuracy improvements over the best baseline models. Additional ablation studies verify the effectiveness of explicit reasoning plans in enhancing interpretability while constraining path divergence. A reliability analysis further examines the impact of different parameters combinations on the DRKG’s performance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Graph Neural Networks · Topic Modeling · Bayesian Modeling and Causal Inference
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer and Electronic Information, Guangxi University, Nanning 530004, China
Guangxi Key Laboratory of Finance and Economics Big Data, Nanning 530007, China
References (44)
Show source-provided references
1. Knowledge graphs10.1145/3447772 ↗
2. Bollacker, K., Evans, C., Paritosh, P., Sturge, T., and Taylor, J. (2008, January 9–12). Freebase: A collaboratively created graph database for structuring human knowledge. Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, Vancouver, BC, Canada.10.1145/1376616.1376746 ↗
3. Wikidata: A free collaborative knowledgebase10.1145/2629489 ↗
4. Liu, J., and Wang, T. (April, January 31). Multi-hop knowledge base question answering by case retrieval. Proceedings of the 3rd International Conference on Artificial Intelligence, Automation, and High-Performance Computing (AIAHPC 2023), Wuhan, China.10.1117/12.2684740 ↗
5. Yang, J., Hu, X., Xiao, G., and Shen, Y. (2024). A survey of knowledge enhanced pre-trained language models. Proceedings of the 3rd ACM Transactions on Asian and Low-Resource Language Information Processing, Association for Computing Machinery.10.1145/3631392 ↗
6. Mavi, V., Jangra, A., and Jatowt, A. (2022). A survey on multi-hop question answering and generation. arXiv.
7. Saxena, A., Tripathi, A., and Talukdar, P. (2020, January 5–10). Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.10.18653/v1/2020.acl-main.412 ↗
8. Hic-KGQA: Improving multi-hop question answering over knowledge graph via hypergraph and inference chain10.1016/j.knosys.2023.110810 ↗
9. Shi, J., Cao, S., Hou, L., Li, J., and Zhang, H. (2021). TransferNet: An Effective and Transparent Framework for Multi-hop Question Answering over Relation Graph. arXiv.10.18653/v1/2021.emnlp-main.341 ↗
10. Jiang, J., Zhou, K., Zhao, W.X., and Wen, J.R. (2022). Unikgqa: Unified retrieval and reasoning for solving multi-hop question answering over knowledge graph. arXiv.
11. Sukhbaatar, S., Weston, J., and Fergus, R. (2015, January 7–12). End-to-end memory networks. Proceedings of the Annual Conference on Neural Information Processing Systems, Montreal, QC, Canada.
12. Miller, A., Fisch, A., Dodge, J., Karimi, A.H., Bordes, A., and Weston, J. (2016). Key-value memory networks for directly reading documents. arXiv.10.18653/v1/D16-1147 ↗
13. Sun, H., Bedrax-Weiss, T., and Cohen, W. (2019). PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text. arXiv.10.18653/v1/D19-1242 ↗
14. Sun, H., Dhingra, B., Zaheer, M., Mazaitis, K., Salakhutdinov, R., and Cohen, W.W. (2018). Open domain question answering using early fusion of knowledge bases and text. arXiv.10.18653/v1/D18-1455 ↗
15. Schlichtkrull, M., Kipf, T.N., Bloem, P., Van Den Berg, R., Titov, I., and Welling, M. (2018). Modeling relational data with graph convolutional networks. Lecture Notes in Computer Science, Proceedings of the Semantic Web—15th International Conference, Heraklion, Crete, Greece, 3–7 June 2018, Springer International Publishing.10.1007/978-3-319-93417-4_38 ↗
16. Abujabal, A., Yahya, M., Riedewald, M., and Weikum, G. (2017, January 3–7). Automated template generation for question answering over knowledge graphs. Proceedings of the 26th International Conference on World Wide Web, Perth, Australia.10.1145/3038912.3052583 ↗
17. AlAgha, I. (2015). Using linguistic analysis to translate arabic natural language queries to SPARQL. arXiv.10.5121/ijwest.2015.6303 ↗
18. Luz, F.F., and Finger, M. (2018). Semantic parsing natural language into SPARQL: Improving target language representation with neural attention. arXiv.
19. SPARQA: Skeleton-based semantic parsing for complex questions over knowledge bases
20. Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2025, June 10). Improving Language Understanding by Generative Pre-Training. Available online: https://scholar.google.com.hk/scholar?hl=zh-CN&as_sdt=0%2C5&q=Improving+language+understanding+by+generative+pre-training&btnG=.
21. Language models are unsupervised multitask learners
22. Language models are few-shot learners
23. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., and Azhar, F. (2023). Llama: Open and efficient foundation language models. arXiv.
24. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., and Bhosale, S. (2023). Llama 2: Open foundation and fine-tuned chat models. arXiv.
25. Team GLM, Zeng, A., Xu, B., Wang, B., Zhang, C., Yin, D., Zhang, D., Rojas, D., Feng, G., and Zhao, H. (2024). ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools. arXiv.
26. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Adv. Neural Inf. Process. Syst., 30.
27. Zhao, W.X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., and Dong, Z. (2023). A survey of large language models. arXiv.
28. Palm: Scaling language modeling with pathways
29. Chain-of-thought prompting elicits reasoning in large language models
30. Shengyu, Z., Linfeng, D., Xiaoya, L., Sen, Z., Xiaofei, S., Shuhe, W., Jiwei, L., Hu, R., Tianwei, Z., and Wu, F. (2023). Instruction tuning for large language models: A survey. arXiv.
31. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions10.1145/3703155 ↗
32. Lin, S., Hilton, J., and Evans, O. (2021). Truthfulqa: Measuring how models mimic human falsehoods. arXiv.10.18653/v1/2022.acl-long.229 ↗
33. Sun, J., Xu, C., Tang, L., Wang, S., Lin, C., Gong, Y., Ni, L.M., Shum, H.Y., and Guo, J. (2023). Think-on-graph: Deep and responsible reasoning of large language model with knowledge graph. arXiv.
34. Luo, L., Li, Y.F., Haffari, G., and Pan, S. (2023). Reasoning on graphs: Faithful and interpretable large language model reasoning. arXiv.
35. Luo, H., Tang, Z., Peng, S., Guo, Y., Zhang, W., Ma, C., Dong, G., Song, M., Lin, W., and Zhu, Y. (2023). Chatkbqa: A generate-then-retrieve framework for knowledge base question answering with fine-tuned large language models. arXiv.10.18653/v1/2024.findings-acl.122 ↗
36. Yani, M., and Krisnadhi, A.A. (2021). Challenges, techniques, and trends of simple knowledge graph question answering: A survey. Information, 12.10.3390/info12070271 ↗
37. Complex knowledge base question answering: A survey10.1109/TKDE.2022.3223858 ↗
38. Zhang, Y., Dai, H., Kozareva, Z., Smola, A., and Song, L. (2018, January 2–7). Variational reasoning for question answering with knowledge graph. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.12057 ↗
39. Zhou, M., Huang, M., and Zhu, X. (2018). An interpretable reasoning network for multi-relation question answering. arXiv.
40. Yih, W.T., Richardson, M., Meek, C., Chang, M.W., and Suh, J. (2016, January 7–12). The value of semantic parse labeling for knowledge base question answering. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany.10.18653/v1/P16-2033 ↗
41. Berant, J., Chou, A., Frostig, R., and Liang, P. (2013, January 18–21). Semantic parsing on freebase from question-answer pairs. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, WA, USA.10.18653/v1/D13-1160 ↗
42. Talmor, A., and Berant, J. (2018). The web as a knowledge-base for answering complex questions. arXiv.10.18653/v1/N18-1059 ↗
43. A knowledge graph based question answering method for medical domain10.7717/peerj-cs.667 ↗
44. Hu, N., Chen, J., Wu, Y., Qi, G., Bi, S., Wu, T., and Pan, J.Z. (2024). Benchmarking large language models in complex question answering attribution using knowledge graphs. arXiv.