An Empirical Study of the Code Generation of Safety-Critical Software Using LLMs
Source abstract
Abstract
In the digital era of increasing software complexity, improving the development efficiency of safety-critical software is a challenging task faced by academia and industry in domains such as nuclear energy, aviation, the automotive industry, and rail transportation. Recently, people have been excited about using pre-trained large language models (LLMs) such as ChatGPT and GPT-4 to generate code. Professionals in the safety-critical software field are intrigued by the code generation capabilities of LLMs. However, there is currently a lack of systematic case studies in this area. Aiming at the need for automated code generation in safety-critical domains such as nuclear energy and the automotive industry, this paper conducts a case study on generating safety-critical software code using GPT-4 as the tool. Practical engineering cases from the industrial domain are employed. We explore different approaches, including code generation based on overall requirements, specific requirements, and augmented prompts. We propose a novel prompt engineering method called Prompt-FDC that integrates basic functional requirements, domain feature generalization, and domain constraints. This method improves code completeness from achieving 30% functions to 100% functions, increases the code comment rate to 26.3%, and yields better results in terms of code compliance, readability, and maintainability. The code generation approach based on LLMs also introduces a new software development process and V-model lifecycle for safety-critical software. Through systematic case studies, we demonstrate that, with appropriate prompt methods, LLMs can auto-generate safety-critical software code that meets practical engineering application requirements. It is foreseeable that LLMs can be applied to various engineering domains to improve software safety and development efficiency.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Software Reliability and Analysis Research · Software Testing and Debugging Techniques · Artificial Intelligence in Healthcare and Education
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer Science, Sichuan University, Chengdu 610065, China
Science and Technology on Reactor System Design Technology Laboratory, Nuclear Power Institute of China, Chengdu 610041, China
Funding
National Key Research and Development Program · Major Science and Technology Special Project of Sichuan Province
References (41)
Show source-provided references
1. Leveson, N.G. (2016). Engineering a Safer World: Systems Thinking Applied to Safety, The MIT Press.
2. Jolak, R., Ho-Quang, T., Chaudron, M.R.V., and Schiffelers, R.R.H. (2018, January 14–19). Model-Based Software Engineering: A Multiple-Case Study on Challenges and Development Efforts. Proceedings of the 21th ACM/IEEE International Conference on Model Driven Engineering Languages and Systems, New York, NY, USA. Association for Computing Machinery.10.1145/3239372.3239404 ↗
3. Colaco, J.-L., Pagano, B., and Pouzet, M. (2017, January 13–15). SCADE 6: A Formal Language for Embedded Critical Software Development (Invited Paper). Proceedings of the 2017 International Symposium on Theoretical Aspects of Software Engineering (TASE), Sophia Antipolis, France.10.1109/TASE.2017.8285623 ↗
4. SCADE: A Comprehensive Framework for Critical System and Software Engineering
5. Experiences with Incorporating Formal Techniques into Industrial Practice10.1007/s10664-013-9251-2 ↗
6. Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., and Metzler, D. (2022). Emergent Abilities of Large Language Models. arXiv.
7. Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y.T., Li, Y., and Lundberg, S. (2023). Sparks of Artificial General Intelligence: Early Experiments with GPT-4. arXiv.
8. Chen, M., Tworek, J., Jun, H., Yuan, Q., de Pinto, H.P.O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., and Brockman, G. (2021). Evaluating Large Language Models Trained on Code. arXiv.
9. Liu, J., Xia, C.S., Wang, Y., and Zhang, L. (2023). Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation. arXiv.
10. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention Is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
11. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019, January 14–17). BERT: Retraining of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North, Louis, MI, USA.
12. Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. (2023, December 25). Improving Language Understanding by Generative Pre-Training. Available online: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf.
13. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020, January 6–12). Language Models Are Few-Shot Learners. Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada.
14. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
15. Du, N., Huang, Y., Dai, A.M., Tong, S., Lepikhin, D., Xu, Y., Krikun, M., Zhou, Y., Yu, A.W., and Firat, O. (2022, January 17–23). GLaM: Efficient Scaling of Language Models with Mixture-of-Experts. Proceedings of the 39th International Conference on Machine Learning, Baltimore, MD, USA.
16. Training Language Models to Follow Instructions with Human Feedback
17. (2023, December 25). Introducing ChatGPT. Available online: https://openai.com/blog/chatgpt.
18. Open, A.I., Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., and Altman, S. (2023). GPT-4 Technical Report. arXiv.
19. Anil, R., Dai, A.M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., and Chen, Z. (2023). PaLM 2 Technical Report. arXiv.
20. (2023, December 24). Anthropic\Claude 2. Available online: https://www.anthropic.com/index/claude-2.
21. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., and Bhosale, S. (2023). Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv.
22. Competition-Level Code Generation with AlphaCode10.1126/science.abq1158 ↗
23. Nijkamp, E., Pang, B., Hayashi, H., Tu, L., Wang, H., Zhou, Y., Savarese, S., and Xiong, C. (2022). CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis. arXiv.
24. Wang, Y., Le, H., Gotmare, A.D., Bui, N.D.Q., Li, J., and Hoi, S.C.H. (2023). CodeT5+: Open Code Large Language Models for Code Understanding and Generation. arXiv.10.18653/v1/2023.emnlp-main.68 ↗
25. Fried, D., Aghajanyan, A., Lin, J., Wang, S., Wallace, E., Shi, F., Zhong, R., Yih, W., Zettlemoyer, L., and Lewis, M. (2022). InCoder: A Generative Model for Code Infilling and Synthesis. arXiv.
26. Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., and Le, Q. (2021). Program Synthesis with Large Language Models. arXiv.
27. Luo, Z., Xu, C., Zhao, P., Sun, Q., Geng, X., Hu, W., Tao, C., Ma, J., Lin, Q., and Jiang, D. (2023). WizardCoder: Empowering Code Large Language Models with Evol-Instruct. arXiv.
28. Lai, Y., Li, C., Wang, Y., Zhang, T., Zhong, R., Zettlemoyer, L., Yih, S.W., Fried, D., Wang, S., and Yu, T. (2023, January 23–29). DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation. Proceedings of the 40th International Conference on Machine Learning, Honolulu, HI, USA.
29. White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., and Schmidt, D.C. (2023). A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. arXiv.
30. ChatGPT: Five Priorities for Research10.1038/d41586-023-00288-7 ↗
31. Jung, J., Qin, L., Welleck, S., Brahman, F., Bhagavatula, C., Bras, R.L., and Choi, Y. (2022). Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations. arXiv.10.18653/v1/2022.emnlp-main.82 ↗
32. Arora, S., Narayan, A., Chen, M.F., Orr, L., Guha, N., Bhatia, K., Chami, I., and Re, C. (2022). Ask Me Anything: A Simple Strategy for Prompting Language Models. arXiv.
33. Zhou, Y., Muresanu, A.I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J. (2022). Large Language Models Are Human-Level Prompt Engineers. arXiv.
34. Liu, V., and Chilton, L.B. (May, January 29). Design Guidelines for Prompt Engineering Text-to-Image Generative Models. Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA.10.1145/3491102.3501825 ↗
35. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
36. Koziolek, H., Gruener, S., and Ashiwal, V. (2023). ChatGPT for PLC/DCS Control Logic Generation. arXiv.10.1109/ETFA54631.2023.10275411 ↗
37. (2010). Functional Safety of Electrical/Electronic/Programmable Electronic Safety-Related Systems (Standard No. IEC 61508:2010).
38. (2006). Nuclear Power Plants—Instrumentation and Control Systems Important to Safety—Software Aspects for Computer-Based Systems Performing Category A Functions (Standard No. IEC 60880:2006).
39. (2011). Road Vehicles Functional Safety (Standard No. ISO 26262:2011).
40. (2024, January 09). Robustness Testing: What Is It & How to Deliver Reliable Software Systems with Test Automation. Available online: https://www.parasoftchina.cn/what-is-robustness-testing/.
41. (2023, December 24). Cppcheck—A Tool for Static C/C++ Code Analysis. Available online: http://cppcheck.net/.