journal article·2026-03-06Open access

A Logic-Guided and Explainable Approach to LLM-Based Unit Test Generation

Cong Zeng, Meng Li, Fei Liu, Xiaohua Yang, Jie Liu, Shiyu Yan

Applied Sciences· Volume 16· Issue 5

Abstract

Large Language Models (LLMs) have demonstrated considerable potential in automated unit test generation; however, most existing approaches rely on a black-box paradigm that directly maps code under test to test code, often resulting in low compilation success rates, limited branch coverage, high assertion failure rates, and poor interpretability. Inspired by the human process of developing test cases, this paper proposes Logic-CoT, a white-box generation paradigm that follows a code under test–logical reasoning–test code workflow. The proposed approach consists of three stages: in the logical inference stage, logical node state vectors and execution paths are constructed from the control flow graph of the code under test, and input values and oracles satisfying state constraints are derived; in the test case construction stage, a template-based method is used to initialize test code conforming to the Arrange–Act–Assert pattern, with test intentions explicitly documented as comments; in the repair stage, syntactic errors and assertion failures are handled in a layered manner, where the former are corrected without altering test logic and the latter trigger logic reflection based on discrepancies between expected and actual outcomes, leading to state updates and test case reconstruction. This design forms a closed-loop process of reasoning, generation, and repair. Experiments on the QuixBugs, Apache Commons, HumanEval, and SV-COMP benchmarks show that Logic-CoT consistently outperforms state-of-the-art approaches such as ChatUniTest in terms of compilation success rate, runtime pass rate, assertion pass rate, branch coverage, average repair iterations for faulty code, and interpretability. Ablation studies further demonstrate that each component of Logic-CoT contributes effectively to improving the overall quality and effectiveness of generated test cases. These results indicate that Logic-CoT improves the reliability and interpretability of LLM-generated unit tests in practical software testing scenarios.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Software Testing and Debugging Techniques · Software System Performance and Reliability · Software Engineering Research

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Computer Science, University of South China, Hengyang 421001, China

Hunan Engineering Research Center of Software Evaluation and Testing for Intellectual Equipment, Hengyang 421001, China

Key Laboratory on High Trusted Computing, CNNC (China National Nuclear Corporation), Hengyang 421001, China

Funding

National Natural Science Foundation of China · Hunan Provincial Natural Science Foundation · Hunan Provincial Teaching Reform Project · industry-funded research project

References (25)

Show source-provided references
1. Fraser, G., and Arcuri, A. (2011, January 5–9). EvoSuite: Automatic test suite generation for object-oriented software. Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering (FSE), Szeged, Hungary.10.1145/2025113.2025179 ↗
2. Pacheco, C., Lahiri, S.K., Ernst, M.D., and Ball, T. (2007, January 20–26). Feedback-directed random test generation. Proceedings of the 29th International Conference on Software Engineering (ICSE), Minneapolis, MN, USA.10.1109/ICSE.2007.37 ↗
3. An empirical evaluation of using large language models for test generation
4. Wegener, J., Baresel, A., and Sthamer, H. (2001, January 7–11). Evolutionary test environment for automatic structural testing. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), San Francisco, CA, USA.
5. Cadar, C., Dunbar, D., and Engler, D. (2008, January 8–10). KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. Proceedings of the 8th USENIX Symposium on Operating Systems Design and Implementation (OSDI), San Diego, CA, USA.
6. Watson, C., Bavota, G., Linares-Vásquez, M., Di Penta, M., and Palomba, F. (July, January 27). On learning meaningful assert statements for unit test cases. Proceedings of the 42nd International Conference on Software Engineering (ICSE), Seoul, Republic of Korea.
7. Nie, L., Wang, Y., Zhang, Y., and Chen, Z. (2023, January 17–21). TeCO: Text-code generation for unit test synthesis. Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), Seattle, WA, USA.
8. Almazan, J., Bavota, G., and Di Penta, M. (2024, January 14–20). TestGen-LLM: Empirical impact of large language models on developer productivity. Proceedings of the IEEE/ACM International Conference on Software Engineering—Software Engineering in Practice (ICSE-SEIP), Lisbon, Portugal.
9. Rae, J.W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., and Young, S. (2021). Scaling language models: Methods, analysis and insights. arXiv.
10. Xie, X., Li, Y., Chen, Z., and Lou, J.G. (2024, January 14–20). ChatUniTest: A ChatGPT-based automated unit test generation tool. Proceedings of the IEEE/ACM International Conference on Software Engineering (ICSE), Lisbon, Portugal.
11. Kang, H., Lee, J., and Kim, S. (2023, January 14–20). Large language models are few-shot testers. Proceedings of the IEEE/ACM International Conference on Software Engineering (ICSE), Melbourne, Australia.
12. Lemieux, C., Inala, J.P., Lahiri, S.K., and Sen, S. (2023, January 14–20). CODAMOSA: Escaping coverage plateaus in automated test generation. Proceedings of the IEEE/ACM International Conference on Software Engineering (ICSE), Melbourne, Australia.
13. Biagiola, M., Ghislotti, G., and Tonella, P. (2023, January 11–15). Enhancing the readability of automatically generated tests using large language models. Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE), Kirchberg, Luxembourg.
14. An empirical study of large language models for code generation
15. Fan, J., Li, Y., and Chen, Z. (2024, January 16–20). Are large language models reliable test oracles? An empirical study. Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), Vienna, Austria.
16. Chen, P., Feng, S., and Zhang, Y. (2024, January 14–20). On the reasoning patterns of large language models for program analysis. Proceedings of the IEEE/ACM International Conference on Software Engineering (ICSE), Lisbon, Portugal.
17. Liu, H., Wang, Q., and Zhou, M. (2023, January 11–15). Path coverage limitations of neural code generation models. Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE), Kirchberg, Luxembourg.
18. Myers, G.J., Sandler, C., and Badgett, T. (2011). The Art of Software Testing, Wiley. [3rd ed.].10.1002/9781119202486 ↗
19. Ammann, P., and Offutt, J. (2016). Introduction to Software Testing, Cambridge University Press. [2nd ed.].10.1017/9781316771273 ↗
20. QuixBugs: A multi-lingual program repair benchmark set
21. Apache Software Foundation (2025, December 26). Apache Commons Math. Available online: https://commons.apache.org/proper/commons-math/.
22. Apache Software Foundation (2025, December 26). Apache Commons Lang. Available online: https://commons.apache.org/proper/commons-lang/.
23. Liu, Q., Yang, S., Li, Y., Wang, M., Zhang, L., Luo, Y., Chang, B., and Sui, Z. (2023). HumanEval-X: A benchmark for multilingual code generation. arXiv.
24. Beyer, D. (April, January 24). Competition on software verification (SV-COMP). Proceedings of the International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS), Tallinn, Estonia.
25. Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., and Huang, F. (2023). Qwen technical report. arXiv.