Neural Methods for Programming: A Comprehensive Survey and Future Directions
Source abstract
Abstract
The advancement of neural-based models has driven significant progress in modern code intelligence, accelerating the development of intelligent programming tools such as code assistants and automated software engineering systems. This study presents a comprehensive and systematic survey of neural methods for programming tasks within the broader context of software development. Guided by six research questions, this study synthesizes insights from more than 250 scientific papers, the majority of which were published between 2015 and 2025, with earlier foundational works (dating back to the late 1990s) included for historical context. The analysis spans 18 major programming tasks, including code generation, code translation, code clone detection, code classification, and vulnerability detection. The survey methodologically examines the development and evolution of neural approaches, the datasets employed, and the performance evaluation metrics adopted in this field. It traces the progress in neural techniques from early code modeling approaches to advanced Code-specific Large Language Models (Code LLMs), emphasizing their advantages over traditional rule-based and statistical methods. A taxonomy of evaluation metrics and a categorized summary of datasets and benchmarks reveal both progress and persistent limitations in data coverage and evaluation practices. The review further distinguishes neural models designed for natural language processing and programming languages, highlighting the structural and functional characteristics that influence model performance. Finally, the study discusses emerging trends, unresolved challenges, and potential research directions, underscoring the transformative role of neural-based architectures, particularly Code LLMs, in enhancing programming and software design activities and shaping the future of AI-driven software development.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Software Engineering Research · Software Engineering Techniques and Practices · Software Reliability and Analysis Research
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, Kangwon National University, Chuncheon 24341, Republic of Korea
Department of Data Science, Kangwon National University, Chuncheon 24341, Republic of Korea
Theory of Computation Laboratory, Yonsei University, Seoul 03722, Republic of Korea
Department of Artificial Intelligence, University of Seoul, Seoul 02504, Republic of Korea
Department of Computer Science and Engineering, Kangwon National University, Chuncheon 24341, Republic of Korea
Funding
Ministry of Science and ICT · Ministry of Education · Ministry of Education and Gangwon State
References (100+)
Show source-provided references
1. Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., and Bi, X. (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv.
2. Webber, B., Cohn, T., He, Y., and Liu, Y. (2020, January 16–20). Plan ahead: Self-Supervised Text Planning for Paragraph Completion Task. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.
3. OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., and Altman, S. (2024). GPT-4 Technical Report. arXiv.
4. A study on prompt design, advantages and limitations of ChatGPT for deep learning program repair10.1007/s10515-025-00492-x ↗
5. Zhao, H., Hui, J., Howland, J., Nguyen, N., Zuo, S., Hu, A., Choquette-Choo, C.A., Shen, J., Kelley, J., and Bansal, K. (2024). CodeGemma: Open code models based on Gemma. arXiv.
6. Exploring and Unleashing the Power of Large Language Models in Automated Code Translation10.1145/3660778 ↗
7. Rogers, A., Boyd-Graber, J., and Okazaki, N. (2023). AVATAR: A Parallel Corpus for Java-Python Program Translation. Findings of the Association for Computational Linguistics: EACL 2023, Association for Computational Linguistics.
8. Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J. (2020, January 10). A Transformer-based Approach for Source Code Summarization. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.
9. Chakraborty, S., and Ray, B. (2021, January 15–19). On multi-modal learning of editing source code. Proceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering, Melbourne, Australia.10.1109/ASE51524.2021.9678559 ↗
10. Katz, D.S., Ruchti, J., and Schulte, E. (2018, January 20–23). Using recurrent neural networks for decompilation. Proceedings of the 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER), Campobasso, Italy.10.1109/SANER.2018.8330222 ↗
11. Wu, Y., and Wang, W. (2021, January 19–21). Code Similarity Detection Based on Siamese Network. Proceedings of the 2021 IEEE International Conference on Information Communication and Software Engineering (ICICSE), Chengdu, China.10.1109/ICICSE52190.2021.9404110 ↗
12. Zhang, C., Wang, J., Zhou, Q., Xu, T., Tang, K., Gui, H., and Liu, F. (2022). A Survey of Automatic Source Code Summarization. Symmetry, 14.10.3390/sym14030471 ↗
13. Uddin, M.N., Zhang, Y., and Hei, X. (2025). Deep Learning Aided Software Vulnerability Detection: A Survey. arXiv.
14. A Survey on Deep Learning for Software Engineering10.1145/3505243 ↗
15. The integration of machine learning into automated test generation: A systematic mapping study10.1002/stvr.1845 ↗
16. A survey of machine learning for big code and naturalness10.1145/3212695 ↗
17. A systematic mapping study of source code representation for deep learning in software engineering10.1049/sfw2.12064 ↗
18. Artificial Intelligence Applied to Software Testing: A Tertiary Study10.1145/3616372 ↗
19. A Decade of Progress: A Systematic Literature Review on the Integration of AI in Software Engineering Phases and Activities (2013–2023)10.1109/ACCESS.2024.3488904 ↗
20. Systematic Mapping: Artificial Intelligence Techniques in Software Engineering10.1109/ACCESS.2022.3174115 ↗
21. Crawford, T., Duong, S., Fueston, R., Lawani, A., Owoade, S., Uzoka, A., Parizi, R.M., and Yazdinejad, A. (2023). AI in Software Engineering: A Survey on Project Management Applications. arXiv.
22. A Survey of Learning-based Automated Program Repair
23. Xiaomeng, W., Tao, Z., Wei, X., and Changyu, H. (2018, January 4–6). A survey on source code review using machine learning. Proceedings of the 2018 3rd International Conference on Information Systems Engineering (ICISE), Shanghai, China.10.1109/ICISE.2018.00018 ↗
24. Akimova, E.N., Bersenev, A.Y., Deikov, A.A., Kobylkin, K.S., Konygin, A.V., Mezentsev, I.P., and Misilov, V.E. (2021). A survey on software defect prediction using deep learning. Mathematics, 9.10.3390/math9111180 ↗
25. Survey of Code Search Based on Deep Learning
26. A systematic literature review on source code similarity measurement and clone detection: Techniques, applications, and challenges10.1016/j.jss.2023.111796 ↗
27. Al-Hossami, E., and Shaikh, S. (2022). A survey on artificial intelligence for source code: A dialogue systems perspective. arXiv.
28. A survey on data-driven software vulnerability assessment and prioritization10.1145/3529757 ↗
29. Das, S., and Shah, C. (2015). Contextual Code Completion Using Machine Learning, Stanford University. Technical Report.
30. GitHub Copilot AI pair programmer: Asset or Liability?10.1016/j.jss.2023.111734 ↗
31. Ouyang, S., Zhang, J., Sun, Z., and Merono Penuela, A. (May, January 27). Knowledge-Enhanced Program Repair for Data Science Code. Proceedings of the 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE), Ottawa, ON, Canada.10.1109/ICSE55347.2025.00246 ↗
32. Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C.B., Drain, D., Jiang, D., and Tang, D. (2021). CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation. arXiv.
33. Lessons from applying the systematic literature review process within the software engineering domain10.1016/j.jss.2006.07.009 ↗
34. Kitchenham, B., and Charters, S. (2007). Guidelines for Performing Systematic Literature Reviews in Software Engineering, EBSE. Version 2.3 EBSE Technical Report.
35. A Systematic Literature Review on the Use of Deep Learning in Software Engineering Research10.1145/3485275 ↗
36. Cao, S., Sun, X., Widyasari, R., Lo, D., Wu, X., Bo, L., Zhang, J., Li, B., Liu, W., and Wu, D. (2025). A Systematic Literature Review on Explainability for Machine/Deep Learning-based Software Engineering Research. arXiv.10.1145/3763230 ↗
37. Deep Learning for Source Code Modeling and Generation: Models, Applications, and Challenges10.1145/3383458 ↗
38. Large language models for software engineering: A systematic literature review10.1145/3695988 ↗
39. Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., and Zhang, J.M. (2023, January 14–20). Large language models for software engineering: Survey and open problems. Proceedings of the 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE), Melbourne, Australia.10.1109/ICSE-FoSE59343.2023.00008 ↗
40. Code vulnerability detection based on deep sequence and graph models: A survey10.1155/2022/1176898 ↗
41. Dou, S., Shan, J., Jia, H., Deng, W., Xi, Z., He, W., Wu, Y., Gui, T., Liu, Y., and Huang, X. (2023). Towards Understanding the Capability of Large Language Models on Code Clone Detection: A Survey. arXiv.
42. Zhong, W., Li, C., Ge, J., and Luo, B. (2022, January 11–12). Neural program repair: Systems, challenges and solutions. Proceedings of the 13th Asia-Pacific Symposium on Internetware, Hohhot, China.10.1145/3545258.3545268 ↗
43. Huang, K., Xu, Z., Yang, S., Sun, H., Li, X., Yan, Z., and Zhang, Y. (2023). A Survey on Automated Program Repair Techniques. arXiv.
44. Software testing with large language models: Survey, landscape, and vision10.1109/TSE.2024.3368208 ↗
45. A survey on deep learning approaches for text-to-SQL10.1007/s00778-022-00776-8 ↗
46. Code Search: A Survey of Techniques for Finding Code10.1145/3565971 ↗
47. Zheng, Z., Ning, K., Wang, Y., Zhang, J., Zheng, D., Ye, M., and Chen, J. (2024). A Survey of Large Language Models for Code: Evolution, Benchmarking, and Future Trends. arXiv.
48. Zan, D., Chen, B., Zhang, F., Lu, D., Wu, B., Guan, B., Yongji, W., and Lou, J.G. (2019, January 9–14). Large language models meet NL2Code: A survey. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, ON，Canada.
49. Wong, M.F., Guo, S., Hang, C.N., Ho, S.W., and Tan, C.W. (2023). Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review. Entropy, 25.10.3390/e25060888 ↗
50. Towards an understanding of large language models in software engineering tasks10.1007/s10664-024-10602-0 ↗
51. Xu, Y., and Zhu, Y. (2022). A Survey on Pretrained Language Models for Neural Code Intelligence. arXiv.
52. Batarseh, F.A., Mohod, R., Kumar, A., and Bui, J. (2020). The application of artificial intelligence in software engineering: A review challenging conventional wisdom. Data Democr., 179–232.10.1016/B978-0-12-818366-3.00010-1 ↗
53. A Survey of Automatic Generation of Source Code Comments: Algorithms and Techniques10.1109/ACCESS.2019.2931579 ↗
54. Source-Code Generation Using Deep Learning: A Survey
55. Chen, X., Xue, J., Xie, X., Liang, C., and Ju, X. (2025). A Systematic Literature Review on Neural Code Translation. arXiv.
56. Deep learning application on code clone detection: A review of current knowledge10.1016/j.jss.2021.111141 ↗
57. Deep learning-based software engineering: Progress, challenges, and opportunities10.1007/s11432-023-4127-5 ↗
58. Jiang, J., Wang, F., Shen, J., Kim, S., and Kim, S. (2025). A Survey on Large Language Models for Code Generation. ACM Trans. Softw. Eng. Methodol., accepted.10.1145/3731753 ↗
59. Mathai, A., Sedamaki, K., Das, D., Mathews, N.S., Tamilselvam, S., Chimalakonda, S., and Kumar, A. (2024). CodeSAM: Source Code Representation Learning by Infusing Self-Attention with Multi-Code-View Graphs. arXiv.
60. Sharma, T., Kechagia, M., Georgiou, S., Tiwari, R., and Sarro, F. (2024). A Survey on Machine Learning Techniques for Source Code Analysis, Elsevier Science Inc.10.1016/j.jss.2023.111934 ↗
61. Promises and perils of using Transformer-based models for SE research10.1016/j.neunet.2024.107067 ↗
62. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., and Azhar, F. (2023). LLaMA: Open and Efficient Foundation Language Models. arXiv.
63. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., and Bhosale, S. (2023). Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv.
64. Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., and Vaughan, A. (2024). The Llama 3 Herd of Models. arXiv.
65. Conneau, A., and Lample, G. (2019, January 8–14). Cross-lingual language model pretraining. Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, BC, Canada.
66. Liu, F., Li, G., Zhao, Y., and Jin, Z. (2020). Multi-task learning based pre-trained language model for code completion. ASE’20: Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, Association for Computing Machinery.10.1145/3324884.3416591 ↗
67. Deep learning for text style transfer: A survey10.1162/coli_a_00426 ↗
68. Pham, Q.N., Waibel, A., and Niehues, J. (2022, January 18–22). Adaptive multilingual speech recognition with pretrained models. Proceedings of the Conference of the International Speech Communication Association, Incheon, Republic of Korea.10.21437/Interspeech.2022-872 ↗
69. Guo, D., Ren, S., Lu, S., Feng, Z., Tang, D., Shujie, L., Zhou, L., Duan, N., Svyatkovskiy, A., and Fu, S. (2021, January 4). GraphCodeBERT: Pre-training Code Representations with Data Flow. Proceedings of the International Conference on Learning Representations, Vienna, Austria.
70. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
71. Moens, M.F., Huang, X., Specia, L., and Yih, S.W.t. (2021, January 7–11). CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Virtual.
72. The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
73. Kanade, A., Maniatis, P., Balakrishnan, G., and Shi, K. (2020, January 13–18). Learning and evaluating contextual embedding of source code. Proceedings of the International Conference on Machine Learning, Virtual.
74. Hägglund, M., Pena, F.J., Pashami, S., Al-Shishtawy, A., and Payberah, A.H. (2021, January 13–16). COCLUBERT: Clustering Machine Learning Source Code. Proceedings of the 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), Pasadena, CA, USA.10.1109/ICMLA52953.2021.00031 ↗
75. Cohn, T., He, Y., and Liu, Y. (2020). CodeBERT: A Pre-Trained Model for Programming and Natural Languages. Findings of the Association for Computational Linguistics: EMNLP 2020, Association for Computational Linguistics.
76. Guo, D., Lu, S., Duan, N., Wang, Y., Zhou, M., and Yin, J. (2022, January 22–27). UniXcoder: Unified cross-modal pre-training for code representation. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Dublin, Ireland.10.18653/v1/2022.acl-long.499 ↗
77. Guo, D., Xu, C., Duan, N., Yin, J., and McAuley, J. (2023, January 23–29). LongCoder: A long-range pre-trained language model for code completion. Proceedings of the International Conference on Machine Learning, Honolulu, HI, USA.
78. Muennighoff, N., Wang, T., Sutawika, L., Roberts, A., Biderman, S., Le Scao, T., Bari, M.S., Shen, S., Yong, Z.X., and Schoelkopf, H. (2023, January 9–14). Crosslingual Generalization through Multitask Finetuning. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, ON, Canada.10.18653/v1/2023.acl-long.891 ↗
79. Roziere, B., Lachaux, M.A., Chanussot, L., and Lample, G. (2020, January 6–12). Unsupervised translation of programming languages. Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, BC, Canada.
80. Liu, X., and Xv, L. (2019, January 6–8). Abstract summarization based on the combination of transformer and LSTM. Proceedings of the 2019 International Conference on Intelligent Computing, Automation and Systems (ICICAS), Chongqing, China.10.1109/ICICAS48597.2019.00199 ↗
81. Using machine translation for converting Python 2 to Python 3 code
82. Gu, X., Zhang, H., Zhang, D., and Kim, S. (2017, January 19–25). DeepAM: Migrate APIs with multi-modal sequence to sequence learning. Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, Australia.10.24963/ijcai.2017/514 ↗
83. Takizawa, H., Hirasawa, S., Hayashi, Y., Egawa, R., and Kobayashi, H. (2014, January 17–20). Xevolver: An XML-based code translation framework for supporting HPC application migration. Proceedings of the 2014 21st International Conference on High Performance Computing (HiPC), Goa, India.10.1109/HiPC.2014.7116902 ↗
84. Bhatia, S., Qiu, J., Hasabnis, N., Seshia, S.A., and Cheung, A. (2024, January 9–15). Verified Code Transpilation with LLMs. Proceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS 2024), Vancouver, BC, Canada.
85. Leveraging QLoRA on Code Large Language Models for Multilingual Code Translation10.5626/KTCP.2025.31.3.152 ↗
86. (2025, October 09). C2Rust—Migrate C code to Rust. GitHub Repository. Available online: https://github.com/immunant/c2rust.git.
87. (2025, October 09). J2cstranslator—Java to CSharp Translator. SourceForge Project. Available online: https://sourceforge.net/projects/j2cstranslator/.
88. Balogh, G., Mudalige, G., Reguly, I., Antao, S., and Bertolli, C. (2018, January 12). OP2-Clang: A Source-to-Source Translator Using Clang/LLVM LibTooling. Proceedings of the 2018 IEEE/ACM 5th Workshop on the LLVM Compiler Infrastructure in HPC (LLVM-HPC), Dallas, TX, USA.10.1109/LLVM-HPC.2018.8639205 ↗
89. Vinh, P.C., Alagar, V., Vassev, E., and Khare, A. (2014). Rule-Based Techniques Using Abstract Syntax Tree for Code Optimization and Secure Programming in Java. Context-Aware Systems and Applications, Springer.10.1007/978-3-319-05939-6 ↗
90. Nguyen, A.T., Nguyen, T.T., and Nguyen, T.N. (2013). Lexical statistical machine translation for language migration. ESEC/FSE 2013: Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, Association for Computing Machinery.10.1145/2491411.2494584 ↗
91. Karaivanov, S., Raychev, V., and Vechev, M. (2014). Phrase-Based Statistical Translation of Programming Languages. Onward! 2014, Proceedings of the 2014 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software, ACM.10.1145/2661136.2661148 ↗
92. Nguyen, T.T., Nguyen, A.T., Nguyen, H.A., and Nguyen, T.N. (2013). A statistical semantic language model for source code. ESEC/FSE 2013: Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, Association for Computing Machinery.10.1145/2491411.2491458 ↗
93. Roziere, B., Zhang, J.M., Charton, F., Harman, M., Synnaeve, G., and Lample, G. (2022). Leveraging Automated Unit Tests for Unsupervised Code Translation. arXiv.
94. Zhang, Q., Fang, C., Zheng, Y., Zhang, Y., Zhao, Y., Huang, R., Zhou, J., Yang, Y., Zheng, T., and Chen, Z. (2025). Improving Deep Assertion Generation via Fine-Tuning Retrieval-Augmented Pre-trained Language Models. ACM Trans. Softw. Eng. Methodol., accepted.10.1145/3721128 ↗
95. Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani-Tur, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., and Zhou, Y. (2021, January 6–11). Unified Pre-training for Program Understanding and Generation. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, online.
96. Ren, S., Guo, D., Lu, S., Zhou, L., Liu, S., Tang, D., Sundaresan, N., Zhou, M., Blanco, A., and Ma, S. (2020). CodeBLEU: A Method for Automatic Evaluation of Code Synthesis. arXiv.
97. Ku, L., Martins, A., and Srikumar, V. (2024, January 11–16). Functional Overlap Reranking for Neural Code Generation. Proceedings of the Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand.
98. Miceli-Barone, A.V., and Sennrich, R. (2017, January 27). A Parallel Corpus of Python Functions and Documentation Strings for Automated Code Documentation and Code Generation. Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers), Taipei, Taiwan.
99. Exploring the limits of transfer learning with a unified text-to-text transformer
100. Rabinovich, M., Stern, M., and Klein, D. (2017, January 30). Abstract Syntax Networks for Code Generation and Semantic Parsing. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, BC, Canada.10.18653/v1/P17-1105 ↗