A Cross-Attention Gating Mechanism-Based Multimodal Feature Fusion Method for Software Defect Prediction
Source abstract
Abstract
Early software defect prediction research primarily relied on software metric features. However, such features struggle to fully capture semantic information in source code. Consequently, methods based on semantic features have gradually become mainstream. Nevertheless, the empirical value of traditional metric features remains significant. To address this, this paper proposes a multi-modal feature fusion method based on a cross-attention gating mechanism for software defect prediction (GMCA-SDP). This method aims to effectively fuse multiple modal features of source code to improve defect prediction performance. Unlike previous approaches, GMCA-SDP can simultaneously integrate traditional metric features and various code semantic features. During the feature fusion stage, it considers both the contribution differences among different types of features and the information interaction between modalities. This paper selects three types of modal features as input for GMCA-SDP: traditional metric features, semantic features extracted from the abstract syntax tree, and structural features extracted from the control flow graph. Our experiments on nine open-source projects demonstrate that the GMCA-SDP method outperforms six mainstream defect prediction models, with average improvements of 18.7% in F1, 10.9% in AUC, and 14.1% in G-mean.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Software Engineering Research · Software Reliability and Analysis Research · Software Testing and Debugging Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Science and Technology, Beijing Jiaotong University, Beijing 100044, China
Funding
China State Railway Group Co., Ltd. · China Railway Information Technology Big Data Technology Co., Ltd.
References (46)
Show source-provided references
1. A Systematic Literature Review on Software Defect Prediction Using Artificial Intelligence: Datasets, Data Validation Methods, Approaches, and Tools10.1016/j.engappai.2022.104773 ↗
2. Liu, J., Zhu, X., Liu, C., Cui, X., and Liu, Q. (2022, January 9–11). CPGBERT: An Effective Model for Defect Detection by Learning Program Semantics via Code Property Graph. Proceedings of the 2022 IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Wuhan, China.10.1109/TrustCom56396.2022.00046 ↗
3. A Metrics Suite for Object Oriented Design10.1109/32.295895 ↗
4. An Evaluation of the MOOD Set of Object-Oriented Software Metrics10.1109/32.689404 ↗
5. A Complexity Measure10.1109/TSE.1976.233837 ↗
6. Li, J., He, P., Zhu, J., and Lyu, M.R. (2017, January 25–29). Software Defect Prediction via Convolutional Neural Network. Proceedings of the 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS), Prague, Czech Republic.10.1109/QRS.2017.42 ↗
7. Gated Homogeneous Fusion Networks with Jointed Feature Extraction for Defect Prediction10.1109/TR.2022.3165115 ↗
8. Dam, H.K., Tran, T., Pham, T., Ng, S.W., Grundy, J., and Ghose, A. (2017). Automatic Feature Learning for Vulnerability Prediction. arXiv.
9. Nagappan, N., Ball, T., and Zeller, A. (2006, January 20–28). Mining Metrics to Predict Component Failures. Proceedings of the ICSE ’06: Proceedings of the 28th International Conference on Software Engineering, Shanghai, China.10.1145/1134285.1134349 ↗
10. A Large-Scale Empirical Study of Just-in-Time Quality Assurance10.1109/TSE.2012.70 ↗
11. Wang, S., Liu, T., and Tan, L. (2016, January 14–22). Automatically Learning Semantic Features for Defect Prediction. Proceedings of the ICSE ’16: Proceedings of the 38th International Conference on Software Engineering, Austin, TX, USA.10.1145/2884781.2884804 ↗
12. Dam, H.K., Pham, T., Ng, S.W., Tran, T., Grundy, J., Ghose, A., Kim, T., and Kim, C.-J. (2019, January 25–31). Lessons Learned from Using a Deep Tree-Based Model for Software Defect Prediction in Practice. Proceedings of the 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR), Montreal, QC, Canada.10.1109/MSR.2019.00017 ↗
13. CFG2AT: Control Flow Graph and Graph Attention Network-Based Software Defect Prediction10.1109/TR.2024.3503688 ↗
14. Rajnish, K., Bhattacharjee, V., and Chandrabanshi, V. (2021, January 5–7). Applying Cognitive and Neural Network Approach over Control Flow Graph for Software Defect Prediction. Proceedings of the 2021 Thirteenth International Conference on Contemporary Computing, Noida, India.10.1145/3474124.3474127 ↗
15. software Defect Prediction Based on Gated Hierarchical LSTMs10.1109/TR.2020.3047396 ↗
16. software Defect Prediction Based on Deep Representation Learning of Source Code from Contextual Syntax and Semantic Graph10.1109/TR.2024.3354965 ↗
17. Li, X., and Zhu, Z. (2023, January 12–14). Software Defect Detection Based on Feature Fusion and Alias Analysis. Proceedings of the 2023 IEEE International Test Conference in Asia (ITC-Asia), Matsue, Japan.10.1109/ITC-Asia58802.2023.10301168 ↗
18. Lu, P., Hu, L., Mitelpunkt, A., Bhatnagar, S., Lu, L., and Liang, H. (2024). A Hierarchical Attention-Based Multimodal Fusion Framework for Predicting the Progression of Alzheimer’s Disease. Biomed. Signal Process. Control, 88.10.1016/j.bspc.2023.105669 ↗
19. Survey of Software Defect Prediction Features10.1007/s00521-024-10937-1 ↗
20. Abdu, A., Zhai, Z., Algabri, R., Abdo, H.A., Hamad, K., and Al-antari, M.A. (2022). Deep Learning-Based Software Defect Prediction via Semantic Key Features of Source Code—Systematic Survey. Mathematics, 10.10.3390/math10173120 ↗
21. A Review on Machine Learning Techniques for Software Defect Prediction
22. Rahman, F., and Devanbu, P. (2013, January 18–26). How, and Why, Process Metrics Are Better. Proceedings of the 2013 35th International Conference on Software Engineering (ICSE), San Francisco, CA, USA.10.1109/ICSE.2013.6606589 ↗
23. Halstead, M.H. (1977). Elements of Software Science, Elsevier Science Inc.
24. Multi-View Learning Based on Product and Process Metrics for Software Defect Prediction10.1007/s10489-025-06288-6 ↗
25. Viet Phan, A., Le Nguyen, M., and Thu Bui, L. (2017, January 6–8). Convolutional Neural Networks over Control Flow Graphs for Software Defect Prediction. Proceedings of the 2017 IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI), Boston, MA, USA.10.1109/ICTAI.2017.00019 ↗
26. Ni, C., Wang, W., Yang, K., Xia, X., Liu, K., and Lo, D. (2022, January 14–18). The Best of Both Worlds: Integrating Semantic Features with Expert Features for Defect Prediction and Localization. Proceedings of the ESEC/FSE ’22: 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Singapore.10.1145/3540250.3549165 ↗
27. semantic Feature Learning via Dual Sequences for Defect Prediction10.1109/ACCESS.2021.3051957 ↗
28. Fan, G., Diao, X., Yu, H., Yang, K., and Chen, L. (2019, January 2–5). Deep Semantic Feature Learning with Embedded Static Metrics for Software Defect Prediction. Proceedings of the 2019 26th Asia-Pacific Software Engineering Conference (APSEC), Putrajaya, Malaysia.10.1109/APSEC48747.2019.00041 ↗
29. Abdu, A., Zhai, Z., Abdo, H.A., Algabri, R., Al-masni, M.A., Muhammad, M.S., and Gu, Y.H. (2024). Semantic and Traditional Feature Fusion for Software Defect Prediction Using Hybrid Deep Learning Model. Sci. Rep., 14.10.1038/s41598-024-65639-4 ↗
30. Software Defect Prediction with Semantic and Structural Information of Codes Based on Graph Neural Networks10.1016/j.infsof.2022.107057 ↗
31. A Review of Affective Computing: From Unimodal Analysis to Multimodal Fusion10.1016/j.inffus.2017.02.003 ↗
32. A Survey on Deep Learning for Multimodal Data Fusion10.1162/neco_a_01273 ↗
33. Deep Multimodal Data Fusion10.1145/3649447 ↗
34. Hung, S.-W., Lo, S.-Y., and Hang, H.-M. (2019, January 22–25). Incorporating Luminance, Depth and Color Information by a Fusion-Based Network for Semantic Segmentation. Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan.10.1109/ICIP.2019.8803360 ↗
35. Mohla, S., Pande, S., Banerjee, B., and Chaudhuri, S. (2020, January 14–19). FusAtNet: Dual Attention Based SpectroSpatial Multimodal Fusion Network for Hyperspectral and LiDAR Classification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.21203/rs.3.rs-32802/v1 ↗
36. Zeng, C., Zhou, C.Y., Lv, S.K., He, P., and Huang, J. (2021, January 25–28). GCN2defect: Graph Convolutional Networks for SMOTETomek-Based Software Defect Prediction. Proceedings of the 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE), Wuhan, China.10.1109/ISSRE52982.2021.00020 ↗
37. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv.
38. Das, D., Mathews, N.S., Mathai, A., Tamilselvam, S., Sedamaki, K., Chimalakonda, S., and Kumar, A. (2023). COMEX: A Tool for Generating Customized Source Code Representations. arXiv.10.1109/ASE56229.2023.00010 ↗
39. Grover, A., and Leskovec, J. (2016, January 13–17). node2vec: Scalable Feature Learning for Networks. Proceedings of the KDD ’16; Association for Computing Machinery, New York, NY, USA.10.1145/2939672.2939754 ↗
40. defect Prediction with Semantics and Context Features of Codes Based on Graph Representation Learning10.1109/TR.2020.3040191 ↗
41. Sharma, T., Kechagia, M., Georgiou, S., Tiwari, R., Vats, I., Moazen, H., and Sarro, F. (2022). A Survey on Machine Learning Techniques for Source Code Analysis. arXiv.
42. Software Defect Prediction Using Bayesian Networks10.1007/s10664-012-9218-8 ↗
43. Software Defect Prediction via LSTM10.1049/iet-sen.2019.0149 ↗
44. Software Defect Prediction Employing BiLSTM and BERT-Based Semantic Feature10.1007/s00500-022-06830-5 ↗
45. On the Use of Deep Learning in Software Defect Prediction10.1016/j.jss.2022.111537 ↗
46. A Survey of Software Defect Prediction Based on Deep Learning10.1007/s11831-022-09787-8 ↗