Defect Detection in Source Code via Multimodal Feature Fusion
Source abstract
Abstract
To address the limitation of existing static defect detection methods in capturing code semantics and structural relationships—which leads to incomplete feature representation—we propose a multimodal feature fusion approach for source code defect detection. First, semantic features are extracted from code character sequences while structural features are derived from Abstract Syntax Trees (ASTs). Second, a structural attention mechanism dynamically models interdependencies between these two modalities and fuses them into comprehensive representation vectors. Finally, defect detection is performed based on the integrated representations. Experimental results on the Sard dataset demonstrate: Compared to baseline methods using single representations (semantic or structural), our approach improves F1-score by 1.96% to 11.76%. Against other feature fusion methods, it achieves 1.36% to 1.66% higher F1-score. The method demonstrates good stability when dealing with imbalanced defect category data. By effectively fusing multimodal code information, this approach significantly enhances the accuracy and adaptability of code defect detection in open-source environments.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Software Engineering Research · Advanced Malware Detection Techniques · Software Testing and Debugging Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science, Hunan University of Technology and Business, Changsha 410205, China
Funding
Postgraduate Scientific Research Innovation Project of Hunan Province · National Social Science Fund of China
References (60)
Show source-provided references
1. Open-Source Software Supply Chain for Reliability Assurance of Operating Systems
2. (2023, September 01). COPU (2023) 2023 China Open-Source Blue Book [EB/OL]. (2023-07-31). Available online: https://cosspu.org.cn/download/showdownload.php?id=26.
3. Li, D., Wang, H., and Wang, J. (2024). Artificial intelligence and technological innovation: Evidence from China’s strategic emerging industries. Sustainability, 16.10.3390/su16167226 ↗
4. Anxin Code Security Laboratory (2023). 2023 China Software Supply Chain Security Analysis Report, Qihoo 360 Technology Co., Ltd.
5. Multicenter hierarchical federated learning with fault-tolerance mechanisms for resilient edge computing networks10.1109/TNNLS.2024.3362974 ↗
6. Reconstructed graph neural network with knowledge distillation for lightweight anomaly detection10.1109/TNNLS.2024.3389714 ↗
7. Distributed resource scheduling for large-scale MEC systems: A multiagent ensemble deep reinforcement learning with imitation acceleration10.1109/JIOT.2021.3113872 ↗
8. Fischer, G., Lusiardi, J., and Von Gudenberg, J.W. (2007, January 25–31). Abstract syntax trees–and their role in model driven software development. Proceedings of the International Conference on Software Engineering Advances (ICSEA 2007), French Riviera, France.10.1109/ICSEA.2007.12 ↗
9. (2022, August 02). The MITRE Corporation (2022) CVE List Downloads [EB/OL]. Available online: https://www.cve.org/Downloads.
10. Survey of Source Code Bug Detection Based on Deep Learning
11. MCFNet: Multi-layer concatenation fusion network for medical images fusion10.1109/JSEN.2019.2913281 ↗
12. A cross-project defect prediction model based on deep learning with self-attention10.1109/ACCESS.2022.3214536 ↗
13. An efficient convolutional neural network model based on object-level attention mechanism for casting defect detection on radiography images10.1109/TIE.2019.2962437 ↗
14. Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., and Jiang, D. (2020). CodeBERT: A pre-trained model for programming and natural languages. arXiv.10.18653/v1/2020.findings-emnlp.139 ↗
15. Rong, X. (2014). Word2vec parameter learning explained. arXiv.
16. Akimova, E.N., Bersenev, A.Y., Deikov, A.A., Kobylkin, K.S., Konygin, A.V., Mezentsev, I.P., and Misilov, V.E. (2021). A Survey on Software Defect Prediction Using Deep Learning. Mathematics, 9.10.3390/math9111180 ↗
17. Zhu, H.N., Furth, R.M., Pradel, M., and Rubio-González, C. (2025). From Bugs to Benchmarks: A Comprehensive Survey of Software Defect Datasets. arXiv.
18. A survey of automatic software vulnerability detection, program repair, and defect prediction techniques10.1155/2020/8858010 ↗
19. A systematic survey of just-in-time software defect prediction10.1145/3567550 ↗
20. Generating test case for algebraic specification based on Tabu search and genetic algorithm10.1007/s10586-016-0681-7 ↗
21. E-Commerce Network Security Based on Big Data in Cloud Computing Environment
22. EdgeSanitizer: Locally differentially private deep inference at the edge for mobile data analytics10.1109/JIOT.2019.2897005 ↗
23. Yu, J., Fu, L., Liang, P., Tahir, A., and Shahin, M. (2023, January 26–27). Security defect detection via code review: A study of the openstack and qt communities. Proceedings of the 2023 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), New Orleans, LA, USA.10.1109/ESEM56168.2023.10304852 ↗
24. Chen, Y., Ding, Z., Alowain, L., Chen, X., and Wagner, D. (2023, January 16–18). Diversevul: A new vulnerable source code dataset for deep learning based vulnerability detection. Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses, Hong Kong, China.10.1145/3607199.3607242 ↗
25. Ziems, N., and Wu, S. (2021, January 10–13). Security vulnerability detection using deep learning natural language processing. Proceedings of the IEEE INFOCOM 2021-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Virtual.10.1109/INFOCOMWKSHPS51825.2021.9484500 ↗
26. Li, Z., Zou, D., Xu, S., Ou, X., Jin, H., Wang, S., Deng, Z., and Zhong, Y. (2018). VulDeePecker: A deep learning-based system for vulnerability detection. arXiv.10.14722/ndss.2018.23158 ↗
27. μVulDeePecker: A deep learning-based system for multiclass vulnerability detection
28. Xu, A., Dai, T., Chen, H., Ming, Z., and Li, W. (2018, January 10–12). Vulnerability detection for source code using contextual LSTM. Proceedings of the 2018 5th International Conference on Systems and Informatics (ICSAI), Nanjing, China.10.1109/ICSAI.2018.8599360 ↗
29. Yamaguchi, F., and Rieck, K. (2011, January 8–12). Vulnerability extrapolation: Assisted discovery of vulnerabilities using machine learning. Proceedings of the 5th USENIX Workshop on Offensive Technologies (WOOT 11), San Francisco, CA, USA.
30. Russell, R., Kim, L., Hamilton, L., Lazovich, T., Harer, J., and Ozdemir, O. (2018, January 17–20). Automated vulnerability detection in source code using deep representation learning. Proceedings of the 17th IEEE International Conference on Machine Learning and Applications (ICMLA), Orlando, FL, USA.10.1109/ICMLA.2018.00120 ↗
31. DeepBugs: A learning approach to name-based bug detection10.1145/3276517 ↗
32. SySeVR: A framework for using deep learning to detect software vulnerabilities10.1109/TDSC.2021.3051525 ↗
33. Hou, S., Chen, L., and Ye, Y. (2022, January 18–23). Summarizing source code from structure and context. Proceedings of the 2022 International Joint Conference on Neural Networks (IJCNN), Padua, Italy.10.1109/IJCNN55064.2022.9892013 ↗
34. Review of research on program algorithm recognition (in Chinese)
35. Li, J., He, P., Zhu, J., and Lyu, M.R. (2017, January 25–29). Software defect prediction via convolutional neural network. Proceedings of the 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS), Prague, Czech Republic.10.1109/QRS.2017.42 ↗
36. Anbiya, D.R., Purwarianti, A., and Asnar, Y. (2018, January 7–8). Vulnerability detection in PHP web application using lexical analysis approach with machine learning. Proceedings of the 2018 5th International Conference on Data and Software Engineering (ICoDSE), Mataram, Indonesia.10.1109/ICODSE.2018.8705809 ↗
37. Software defect prediction via heavy son node-based abstract syntax tree
38. Yamaguchi, F., Lottmann, M., and Rieck, K. (2012, January 3–7). Generalized vulnerability extrapolation using abstract syntax trees. Proceedings of the 28th Annual Computer Security Applications Conference, Orlando, FL, USA.10.1145/2420950.2421003 ↗
39. Cross-project transfer representation learning for vulnerable function discovery10.1109/TII.2018.2821768 ↗
40. Mou, L., Li, G., Zhang, L., Wang, T., and Jin, Z. (2016, January 12–17). Convolutional neural networks over tree structures for programming language processing. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10139 ↗
41. Improving bug detection via context-based code representation learning and attention-based neural networks10.1145/3360588 ↗
42. Wang, H., Meghawat, A., Morency, L.P., and Xing, E.P. (2017, January 10–14). Select-additive learning: Improving generalization in multimodal sentiment analysis. Proceedings of the 2017 IEEE International Conference on Multimedia and Expo (ICME), Hong Kong.10.1109/ICME.2017.8019301 ↗
43. Nojavanasghari, B., Gopinath, D., Koushik, J., Baltrušaitis, T., and Morency, L.P. (2016, January 12–16). Deep multimodal fusion for persuasiveness prediction. Proceedings of the 18th ACM International Conference on Multimodal Interaction, Tokyo, Japan.10.1145/2993148.2993176 ↗
44. Anastasopoulos, A., Kumar, S., and Liao, H. (2019). Neural language modeling with visual features. arXiv.
45. Weighted feature fusion of convolutional neural network and graph attention network for hyperspectral image classification10.1109/TIP.2022.3144017 ↗
46. Lu, J., Yang, J., Batra, D., and Parikh, D. (2016, January 5–10). Hierarchical question-image co-attention for visual question answering. Proceedings of the Advances in Neural Information Processing Systems 2016, Barcelona, Spain.
47. Wang, L., Hu, Z., Kong, Q., Qi, Q., and Liao, Q. (2023). Infrared and visible image fusion via attention-based adaptive feature fusion. Entropy, 25.10.3390/e25030407 ↗
48. Nam, H., Ha, J.W., and Kim, J. (2017, January 21–26). Dual attention networks for multimodal reasoning and matching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.232 ↗
49. Attentional feature fusion for end-to-end blind image quality assessment10.1109/TBC.2022.3204235 ↗
50. An approach to software defect prediction combining semantic features and code changes10.1142/S0218194022500504 ↗
51. Software defect prediction with semantic and structural information of codes based on graph neural networks10.1016/j.infsof.2022.107057 ↗
52. Semantic feature learning for software defect prediction from source code and external knowledge10.1016/j.jss.2023.111753 ↗
53. Boosting multi-objective just-in-time software defect prediction by fusing expert metrics and semantic metrics10.1016/j.jss.2023.111853 ↗
54. Cross-project defect prediction method using adversarial learning
55. Dam, H.K., Pham, T., Ng, S.W., Tran, T., Grundy, J., Ghose, A., Kim, T., and Kim, C.J. (2018). A deep tree-based model for software defect prediction. arXiv.10.1109/MSR.2019.00017 ↗
56. Mikolov, T., Karafiát, M., Burget, L., Cernocký, J., and Khudanpur, S. (2010, January 26–30). Recurrent neural network based language model. Proceedings of the Interspeech, Chiba, Japan.10.21437/Interspeech.2010-343 ↗
57. Recent advances in deep learning10.1007/s13042-020-01096-5 ↗
58. Bidirectional recurrent neural networks10.1109/78.650093 ↗
59. A software assurance reference dataset: Thousands of programs with known bugs10.6028/jres.123.005 ↗
60. Fu, M., and Tantithamthavorn, C. (2022, January 23–24). LineVul: A transformer-based line-level vulnerability prediction. Proceedings of the 19th International Conference on Mining Software Repositories (MSR), Pittsburgh, PA, USA.10.1145/3524842.3528452 ↗