CQLLM: A Framework for Generating CodeQL Security Vulnerability Detection Code Based on Large Language Model
Source abstract
Abstract
With the increasing complexity of software systems, the number of security vulnerabilities contained within software has risen accordingly. The existing shift-left security concept aims to detect and fix vulnerabilities during the software development cycle. While CodeQL stands as the premier static code analysis tool currently available on the market, its high barrier to entry poses challenges for meeting the implementation requirements of shift-left security initiatives. While large language model (LLM) offers potential assistance in QL code development, the inherent complexity of code generation tasks often leads to persistent issues such as syntactic inaccuracies and references to non-existent modules, which consequently constrains their practical applicability in this domain. To address these challenges, this paper proposes CQLLM (CodeQL-enhanced Large Language Model), a novel framework for automating the generation of CodeQL security vulnerability detection code by leveraging LLM. This framework is designed to enhance both the efficiency and the accuracy of automated QL code generation, thereby advancing static code analysis for a more efficient and intelligent paradigm for vulnerability detection. First, retrieval-augmented generation (RAG) is employed to search the vector database for dependency libraries and code snippets that are highly similar to the user’s input, thereby constraining the model’s generation process and preventing the import of invalid modules. Then, the user input and the knowledge chunks retrieved by RAG are fed into a fine-tuned LLM to perform reasoning and generate QL code. By integrating external knowledge bases with the large model, the framework enhances the correctness and completeness of the generated code. Experimental results show that CQLLM significantly improves the executability of the generated QL code, with the execution success rate improving from 0.31% to 72.48%, outperforming the original model by a large margin. Meanwhile, CQLLM also enhances the effectiveness of the generated results, achieving a CWE (Common Weakness Enumeration) coverage rate of 57.4% in vulnerability detection tasks, demonstrating its practical applicability in real-world vulnerability detection.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Malware Detection Techniques · Web Application Security Vulnerabilities · Software Engineering Research
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Cyberspace Institute of Advanced Technology, Guangzhou University, Guangzhou 510006, China
College of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China
Department of New Networks, Peng Cheng Laboratory, Xili Street, Shenzhen 518000, China
Funding
National Natural Science Foundation of China
References (30)
Show source-provided references
1. GitHub (2025, October 18). CodeQL for Research. Available online: https://securitylab.github.com/tools/codeql.
2. GitHub Security Lab (2025, October 18). CodeQL: Code Analysis for Security Research. Available online: https://codeql.github.com/.
3. Yamaguchi, F., Golde, N., Arp, D., and Rieck, K. (2014, January 18–21). Modeling and Discovering Vulnerabilities with Code Property Graphs. Proceedings of the 2014 IEEE Symposium on Security and Privacy, San Jose, CA, USA.10.1109/SP.2014.44 ↗
4. Static Analysis for Security10.1109/MSP.2004.111 ↗
5. Nanda, M.G., and Sinha, S. (2009, January 16–24). Accurate Interprocedural Null-Dereference Analysis for Java. Proceedings of the 2009 IEEE 31st International Conference on Software Engineering, Vancouver, BC, Canada.10.1109/ICSE.2009.5070515 ↗
6. A Practical Flow-Sensitive and Context-Sensitive C and C++ Memory Leak Detector10.1145/780822.781150 ↗
7. Aho, A.V., Lam, M.S., Sethi, R., and Ullman, J.D. (2006). Compilers: Principles, Techniques, and Tools, Addison-Wesley Longman Publishing Co., Inc.. [2nd ed.].
8. Control flow analysis10.1145/390013.808479 ↗
9. Constructing the Call Graph of a Program10.1109/TSE.1979.234183 ↗
10. Tai-e Project (2025, October 18). Tai-e: A Static Analysis Framework for Java. Available online: https://tai-e.pascal-lab.net/.
11. Vallée-Rai, R., Co, P., Gagnon, E., Hendren, L., Lam, P., and Sundaresan, V. (2010). Soot: A Java bytecode optimization framework. CASCON First Decade High Impact Papers, IBM Corp.10.1145/1925805.1925818 ↗
12. Santos, J.C.S., and Dolby, J. (2022, January 14–18). Program Analysis Using WALA (Tutorial). Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Singapore.10.1145/3540250.3569449 ↗
13. Kandpal, N., Deng, H., Roberts, A., Wallace, E., and Raffel, C. (2023, January 23–29). Large language models struggle to learn long-tail knowledge. Proceedings of the 40th International Conference on Machine Learning, Honolulu, HI, USA.
14. Sannigrahi, S., Fraga-Silva, T., Oualil, Y., and Van Gysel, C. (2024, January 14–18). Synthetic Query Generation Using Large Language Models for Virtual Assistants. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, Washington, DC, USA.10.1145/3626772.3661355 ↗
15. Choenni, S., Busker, T., and Bargh, M.S. (2023, January 14–15). Generating Synthetic Data from Large Language Models. Proceedings of the 2023 15th International Conference on Innovations in Information Technology (IIT), Al Ain, United Arab Emirates.10.1109/IIT59782.2023.10366424 ↗
16. Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2021). Lora: Low-rank adaptation of large language models. arXiv.
17. Hu, J., Jin, X., Zeng, Y., Liu, Y., Li, Y., Du, D., Xie, K., and Zhu, H. (2025). QLPro: Automated Code Vulnerability Discovery via LLM and Static Code Analysis Integration. arXiv.
18. Li, Z., Dutta, S., and Naik, M. (2024). IRIS: LLM-Assisted Static Analysis for Detecting Security Vulnerabilities. arXiv.
19. Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H.P.d.O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., and Brockman, G. (2021). Evaluating Large Language Models Trained on Code. arXiv.
20. Wang, Y., Wang, W., Joty, S., and Hoi, S.C. (2021, January 7–11). CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online; Punta Cana, Dominican Republic.10.18653/v1/2021.emnlp-main.685 ↗
21. Bouamor, H., Pino, J., and Bali, K. (2023, January 6–10). CodeT5+: Open Code Large Language Models for Code Understanding and Generation. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore.
22. Le, H., Wang, Y., Gotmare, A.D., Savarese, S., and Hoi, S.C.H. (December, January 28). Coderl: Mastering code generation through pretrained models and deep reinforcement learning. Proceedings of the 36th International Conference on Neural Information Processing Systems, New Orleans, LA, USA.
23. Zheng, Q., Xia, X., Zou, X., Dong, Y., Wang, S., Xue, Y., Shen, L., Wang, Z., Wang, A., and Li, Y. (2023, January 6–10). CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, New York, NY, USA.10.1145/3580305.3599790 ↗
24. Gao, D., Wang, H., Li, Y., Sun, X., Qian, Y., Ding, B., and Zhou, J. (2023). Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation. arXiv.10.14778/3641204.3641221 ↗
25. Db-gpt: Large language model meets database10.1007/s41019-023-00235-6 ↗
26. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.t., and Rocktäschel, T. (2020, January 6–12). Retrieval-augmented generation for knowledge-intensive nlp tasks. Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada.
27. Cho, J., Mahata, D., Irsoy, O., He, Y., and Bansal, M. (2024). M3docrag: Multi-modal retrieval is what you need for multi-page multi-document understanding. arXiv.
28. Zhang, B., and Xiang, Y. (2023, January 3–5). Knowledge Base Enhanced ChatGLM for RPA Robot Code Generation. Proceedings of the 2023 4th International Conference on Machine Learning and Computer Application, Hangzhou, China.10.1145/3650215.3650386 ↗
29. Xu, Z., Cruz, M.J., Guevara, M., Wang, T., Deshpande, M., Wang, X., and Li, Z. (2024, January 14–18). Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, Washington, DC, USA.10.1145/3626772.3661370 ↗
30. Dong, J., Fatemi, B., Perozzi, B., Yang, L.F., and Tsitsulin, A. (2024). Don’t Forget to Connect! Improving RAG with Graph-based Reranking. arXiv.