ToR-Lite: A Lightweight Semantic Query Decomposition for Multi-Hop Retrieval-Augmented Generation in Cloud-Based AI Systems
Source abstract
Abstract
Cloud-based AI systems increasingly rely on Retrieval-Augmented Generation (RAG) to handle complex, knowledge-intensive queries. However, query decomposition for multi-hop retrieval—traditionally powered by large language models (LLMs)—incurs significant latency and cost, rendering it impractical for large-scale, cost-sensitive cloud deployments. We propose ToR-Lite, a lightweight, generative LLM-free semantic query decomposition framework designed to enhance multi-hop retrieval efficiency in cloud-based AI systems. ToR-Lite employs a novel Word-Window Splitting algorithm that detects semantic breakpoints via sliding window embeddings, effectively decomposing complex queries without expensive LLM inference. Experiments on the MultiHop-RAG benchmark (n = 2255) demonstrate that ToR-Lite achieves +6.03 pp Hits@10 and +0.89 pp Exact Match improvements over the Baseline, while operating 3.18 times faster than LLM-based Adaptive ToR. Retrieval performance correlates monotonically with decomposition granularity: three sub-query decompositions (#Dq = 3) yields a +7.00 pp Hits@10 improvement, confirming that semantic granularity is a key driver of retrieval performance. Comparison with rule-based Baselines confirms that these gains derive from the precision of semantic boundary detection rather than decomposition quantity alone. ToR-Lite delivers nearly twice the retrieval improvement per unit of computational cost, offering a practical and cost-effective solution for latency-sensitive cloud AI deployments.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Information Retrieval and Search Behavior · Big Data and Digital Economy · Machine Learning in Materials Science
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Convergence Engineering, Hoseo University, Seoul 06724, Republic of Korea
References (35)
Show source-provided references
1. Singh, A., Ehtesham, A., Kumar, S., and Khoei, T. (2025). Agentic retrieval-augmented generation: A survey on agentic RAG. arXiv.
2. Energy Efficient Hierarchical Federated Learning over Intelligent IIoT Devices
3. Deep Learning-Based Decision Support System for Identifying Emerging Items in National Strategic Technology Fields
4. Chen, K., Nadig, R., Frouzakis, M., Mansouri Ghiasi, N., Liang, Y., Mao, H., Park, J., Sadrosadati, M., and Mutlu, O. (2025, January 21–25). REIS: A High-Performance and Energy-Efficient Retrieval System with In-Storage Processing. Proceedings of the 52nd Annual International Symposium on Computer Architecture (ISCA’25), Tokyo, Japan.10.1145/3695053.3731116 ↗
5. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
6. Atlas: Few-Shot Learning with Retrieval Augmented Language Models
7. Yan, S., Gu, J., Zhu, Y., and Ling, Z. (2024). Corrective retrieval augmented generation. arXiv.10.2139/ssrn.5267341 ↗
8. Jeong, S., Baek, J., Cho, S., and Hwang, S.J. (2024). Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity. Proceedings of the NAACL-HLT 2024, Mexico City, Mexico, 16–21 June 2024, Association for Computational Linguistics.10.18653/v1/2024.naacl-long.389 ↗
9. Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., and Wang, H. (2024). Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv.
10. Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W.W., Salakhutdinov, R., and Manning, C.D. (2018). HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP 2018), Brussels, Belgium, 31 October–4 November 2018, Association for Computational Linguistics.10.18653/v1/D18-1259 ↗
11. Karpukhin, V., Oğuz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W. (2020). Dense Passage Retrieval for Open-Domain Question Answering. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020), Online, 16–20 November 2020, Association for Computational Linguistics.10.18653/v1/2020.emnlp-main.550 ↗
12. Reimers, N., and Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the EMNLP 2019, Hong Kong, China, 3–7 November 2019, Association for Computational Linguistics.10.18653/v1/D19-1410 ↗
13. Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., and Le, Q. (2023, January 1–5). Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. Proceedings of the ICLR 2023, Kigali, Rwanda.
14. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
15. Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A. (2023). Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions. Proceedings of the ACL 2023, Toronto, ON, Canada, 9–14 July 2023, Association for Computational Linguistics.10.18653/v1/2023.acl-long.557 ↗
16. Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N.A., and Lewis, M. (2023). Measuring and Narrowing the Compositionality Gap in Language Models. Proceedings of the Findings of EMNLP 2023, Singapore, 6–10 December 2023, Association for Computational Linguistics.10.18653/v1/2023.findings-emnlp.378 ↗
17. ToR-RAG: A Tree-of-Retrieval-based Retrieval-Augmented Generation for Complex Question Processing
18. Yoo, H.-K. (2026). Design and Verification of Adaptive ToR Structure via Query Complexity Awareness. [Ph.D. Thesis, Hoseo University]. Available online: http://www.dcollection.net/handler/hoseo/200000961944.
19. Liu, Y., Li, J., Wu, Y., and Chen, Z. (2025). POQD: Performance-oriented query decomposer for multi-vector retrieval. Proceedings of the 42nd International Conference on Machine Learning (ICML 2025), Vancouver, BC, Canada, 13–19 July 2025, PMLR.
20. Wang, L., Yang, N., and Wei, F. (2023). Query2Doc: Query Expansion with Large Language Models. Proceedings of the EMNLP 2023, Singapore, 6–10 December 2023, Association for Computational Linguistics.10.18653/v1/2023.emnlp-main.585 ↗
21. Gupta, S., and Ranjan, R. (2024). A Comprehensive Survey of Retrieval-Augmented Generation (RAG): Evolution, Current Landscape and Future Directions. arXiv.
22. Xu, Z., Mo, F., Huang, Z., Zhang, C., Yu, P., Wang, B., and Srikumar, V. (2025). A Survey of Model Architectures in Information Retrieval. arXiv.
23. TextTiling: Segmenting Text into Multi-paragraph Subtopic Passages
24. Max–Min Semantic Chunking of Documents for RAG Application10.1007/s10791-025-09638-7 ↗
25. Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs10.1109/TPAMI.2018.2889473 ↗
26. Kim, K., Kim, J., Chung, H., Cha, M.-H., Kim, H.-Y., and Kim, Y. (2025). Cost-efficient LLM serving in the cloud: VM selection with KV cache offloading. arXiv.10.1109/CLOUD67622.2025.00027 ↗
27. Tang, Y., and Yang, Y. (2024, January 7–9). MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries. Proceedings of the COLM 2024, Philadelphia, PA, USA.
28. Kostric, I., and Balog, K. (2024). A Surprisingly Simple yet Effective Multi-Query Rewriting Method for Conversational Passage Retrieval. Proceedings of the SIGIR 2024, Washington, DC, USA, 14–18 July 2024, ACM.10.1145/3626772.3657933 ↗
29. Khattab, O., and Zaharia, M. (2020). ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. Proceedings of the SIGIR 2020, Xi’an, China, 25–30 July 2020, ACM.10.1145/3397271.3401075 ↗
30. Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., and Larson, J. (2024). From Local to Global: A Graph RAG Approach to Query-Focused Summarization. arXiv.
31. Lu, Y., Wang, X., Chen, H., and Li, J. (2025). KEDKG: Knowledge Editing with Dynamic Knowledge Graphs for Multi-hop Question Answering. Proceedings of the AAAI 2025, Philadelphia, PA, USA, 25 February–4 March 2025, AAAI Press.
32. Casanueva, I., Temčinas, T., Gerber, D., Agez, M., and Henderson, M. (2022). NLU++: A Multi-Label, Slot-Rich, Generalisable Dataset for Natural Language Understanding in Task-Oriented Dialogue. Proceedings of the Findings of NAACL 2022, Seattle, WA, USA, 10–15 July 2022, Association for Computational Linguistics.10.18653/v1/2022.findings-naacl.154 ↗
33. Baker, C.F., Fillmore, C.J., and Lowe, J.B. (1998). The Berkeley FrameNet project. Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics (COLING-ACL 1998), Montreal, QC, Canada, 10–14 August 1998, Morgan Kaufmann Publishers.10.3115/980845.980860 ↗
34. Nivre, J., de Marneffe, M.-C., Ginter, F., Hajič, J., Manning, C.D., Pyysalo, S., Schuster, S., Tyers, F., and Zeman, D. (2020). Universal Dependencies v2: An evergrowing multilingual treebank collection. Proceedings of the Twelfth Language Resources and Evaluation Conference (LREC 2020), Marseille, France, 11–16 May 2020, European Language Resources Association.
35. Honnibal, M., Montani, I., Van Landeghem, S., and Boyd, A. (2020). spaCy: Industrial-Strength Natural Language Processing in Python, Zenodo.