A Method for Processing Static Analysis Alarms Based on Deep Learning
Source abstract
Abstract
Automatic static analysis tools (ASATs), also known as static analyzers, have demonstrated their significance and practicability in detecting software defects. ASATs assist developers to identify potential vulnerabilities, errors, and security hazards in source code without executing the software. As software systems grow in scale and complexity, ASATs are replacing manual security audits and becoming crucial for detecting issues in code. However, ASATs often generate numerous warnings with high false positive rates, while developers typically only take measures on a small portion of actionable alarms. To cope with this problem, we propose an innovative method that combines the pre-trained CodeBERT model and neural networks to reduce false positives detected by ASATs. Our approach was evaluated on the Defects4J dataset, which comprises 835 real-world software defects extracted from 17 open-source Java projects. The experimental results explicitly manifest the effectiveness in processing static analysis alarms. By employing a bidirectional recurrent neural network for context embeddings, our approach achieved an accuracy of 95.77% and an AUC score of 98.3%. This research enables developers to minimize false positive alarms and ensure a reasonable number of actionable warnings while guaranteeing software quality and security.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Fault Detection and Control Systems · Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Cyber Security and Computer, Hebei University, Baoding 071002, China
Funding
Natural Science Foundation of Hebei Province · Funds of the Central Government for Local Science and Technology Development
References (52)
Show source-provided references
1. Christakis, M., and Bird, C. (2016, January 3–7). What developers want and need from program analysis: An empirical study. Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, Singapore.10.1145/2970276.2970347 ↗
2. Survey of approaches for postprocessing of static analysis alarms10.1145/3494521 ↗
3. Language models are few-shot learners
4. Competition-level code generation with AlphaCode10.1126/science.abq1158 ↗
5. SySeVR: A Framework for Using Deep Learning to Detect Software Vulnerabilities10.1109/TDSC.2021.3051525 ↗
6. Hovemeyer, D., and Pugh, W. (2007, January 13–14). Finding more null pointer bugs, but not too many. Proceedings of the 7th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, San Diego, CA, USA.10.1145/1251535.1251537 ↗
7. Ayewah, N., Pugh, W., Morgenthaler, J.D., Penix, J., and Zhou, Y. (2007, January 13–14). Evaluating static analysis defect warnings on production software. Proceedings of the 7th ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, San Diego, CA, USA.10.1145/1251535.1251536 ↗
8. Thung, F., Lo, D., Jiang, L., Rahman, F., and Devanbu, P.T. (2012, January 3–7). To what extent could we detect field defects? An empirical study of false negatives in static bug finding tools. Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering, Essen, Germany.10.1145/2351676.2351685 ↗
9. To what extent could we detect field defects? An extended empirical study of false negatives in static bug-finding tools10.1007/s10515-014-0169-8 ↗
10. Ayewah, N., and Pugh, W. (2010, January 12–16). The google findbugs fixit. Proceedings of the 19th International Symposium on Software Testing and Analysis, Trento, Italy.10.1145/1831708.1831738 ↗
11. Aftandilian, E., Sauciuc, R., Priya, S., and Krishnan, S. (2012, January 23–24). Building useful program analysis tools using an extensible java compiler. Proceedings of the 2012 IEEE 12th International Working Conference on Source Code Analysis and Manipulation, Riva del Garda, Italy.10.1109/SCAM.2012.28 ↗
12. Lessons from building static analysis tools at google10.1145/3188720 ↗
13. Calcagno, C., Distefano, D., Dubreil, J., Gabi, D., Hooimeijer, P., Luca, M., O’Hearn, P., Papakonstantinou, I., Purbrick, J., and Rodriguez, D. (2015, January 27–29). Moving fast with software verification. Proceedings of the NASA Formal Methods: 7th International Symposium, NFM 2015, Pasadena, CA, USA. Proceedings 7.10.1007/978-3-319-17524-9_1 ↗
14. Scaling static analyses at Facebook10.1145/3338112 ↗
15. How developers engage with static analysis tools in different contexts10.1007/s10664-019-09750-5 ↗
16. Johnson, B., Song, Y., Murphy-Hill, E., and Bowdidge, R. (2013, January 18–26). Why don’t software developers use static analysis tools to find bugs?. Proceedings of the 2013 35th International Conference on Software Engineering (ICSE), San Francisco, CA, USA.10.1109/ICSE.2013.6606613 ↗
17. Habib, A., and Pradel, M. (2018, January 3–7). How many of all bugs do we find? A study of static bug detectors. Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, Montpellier, France.10.1145/3238147.3238213 ↗
18. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.
19. Wan, Y., Zhao, W., Zhang, H., Sui, Y., Xu, G., and Jin, H. (2022, January 25–27). What do they capture? A structural analysis of pre-trained language models for source code. Proceedings of the 44th International Conference on Software Engineering, Pittsburgh, PA, USA.10.1145/3510003.3510050 ↗
20. Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., and Jiang, D. (2020, January 16–20). CodeBERT: A Pre-Trained Model for Programming and Natural Languages. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online.10.18653/v1/2020.findings-emnlp.139 ↗
21. Taud, H., and Mas, J.F. (2018). Multilayer perceptron (MLP). Geomatic Approaches for Modeling Land Change Scenarios, Springer.10.1007/978-3-319-60801-3_27 ↗
22. Attention mechanisms in computer vision: A survey10.1007/s41095-022-0271-y ↗
23. Schmidt, R.M. (2019). Recurrent neural networks (rnns): A gentle introduction and overview. arXiv.
24. A survey of convolutional neural networks: Analysis, applications, and prospects10.1109/TNNLS.2021.3084827 ↗
25. Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014, January 8–13). Empirical evaluation of gated recurrent neural networks on sequence modeling. Proceedings of the NIPS 2014 Deep Learning and Representation Learning Workshop, Montreal, QC, Canada.
26. Bidirectional recurrent neural networks10.1109/78.650093 ↗
27. Kim, Y. (2014, January 25–29). Convolutional Neural Networks for Sentence Classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.10.3115/v1/D14-1181 ↗
28. Wu, Y., Zou, D., Dou, S., Yang, W., Xu, D., and Jin, H. (2022, January 25–27). Vulcnn: An image-inspired scalable vulnerability detection system. Proceedings of the 44th International Conference on Software Engineering, Pittsburgh, PA, USA.10.1145/3510003.3510229 ↗
29. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2–7). Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the NAACL-HLT 2019, Minneapolis, MI, USA.
30. Husain, H., Wu, H.H., Gazit, T., Allamanis, M., and Brockschmidt, M. (2019). Codesearchnet challenge: Evaluating the state of semantic code search. arXiv.
31. Just, R., Jalali, D., and Ernst, M.D. (2014, January 21–25). Defects4J: A database of existing faults to enable controlled testing studies for Java programs. Proceedings of the 2014 International Symposium on Software Testing and Analysis, San Jose, CA, USA.10.1145/2610384.2628055 ↗
32. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019, January 8–14). Pytorch: An imperative style, high-performance deep learning library. Proceedings of the NIPS’19: Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, BC, Canada.
33. Loshchilov, I., and Hutter, F. (2017, January 24–26). Decoupled weight decay regularization. Proceedings of the International Conference on Learning Representations (2017), Toulon, France.
34. Lee, S., Hong, S., Yi, J., Kim, T., Kim, C.J., and Yoo, S. (2019, January 22–27). Classifying false positive static checker alarms in continuous integration using convolutional neural networks. Proceedings of the 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST), Xi’an, China.10.1109/ICST.2019.00048 ↗
35. Learning to recognize actionable static code warnings (is intrinsically easy)10.1007/s10664-021-09948-6 ↗
36. An empirical assessment of machine learning approaches for triaging reports of static analysis tools10.1007/s10664-022-10253-z ↗
37. Muske, T., and Serebrenik, A. (2022, January 3–4). Classification and Ranking of Delta Static Analysis Alarms. Proceedings of the 2022 IEEE 22nd International Working Conference on Source Code Analysis and Manipulation (SCAM), Limassol, Cyprus.10.1109/SCAM55253.2022.00029 ↗
38. Venkatasubramanyam, R.D., and Gupta, S. (June, January 31). An automated approach to detect violations with high confidence in incremental code using a learning system. Proceedings of the Companion Proceedings of the 36th International Conference on Software Engineering, Hyderabad, India.
39. Heo, K., Oh, H., and Yi, K. (2017, January 20–28). Machine-learning-guided selectively unsound static analysis. Proceedings of the 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE), Buenos Aires, Argentina.10.1109/ICSE.2017.54 ↗
40. Alikhashashneh, E.A., Raje, R.R., and Hill, J.H. (November, January 28). Using machine learning techniques to classify and predict static code analysis tool warnings. Proceedings of the 2018 IEEE/ACS 15th International Conference on Computer Systems and Applications (AICCSA), Aqaba, Jordan.10.1109/AICCSA.2018.8612819 ↗
41. Koc, U., Saadatpanah, P., Foster, J.S., and Porter, A.A. (2017, January 18). Learning a classifier for false positive error reports emitted by static code analysis tools. Proceedings of the 1st ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, Barcelona, Spain.10.1145/3088525.3088675 ↗
42. Yüksel, U., and Sözer, H. (2013, January 22–28). Automated classification of static code analysis alerts: A case study. Proceedings of the 2013 IEEE International Conference on Software Maintenance, Eindhoven, The Netherlands.10.1109/ICSM.2013.89 ↗
43. Tripp, O., Guarnieri, S., Pistoia, M., and Aravkin, A. (2014, January 3–7). Aletheia: Improving the usability of static security analysis. Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, Scottsdale, AZ, USA.10.1145/2660267.2660339 ↗
44. Understanding static code warnings: An incremental AI approach10.1016/j.eswa.2020.114134 ↗
45. Hanam, Q., Tan, L., Holmes, R., and Lam, P. (June, January 31). Finding patterns in static analysis alerts: Improving actionable alert ranking. Proceedings of the 11th Working Conference on Mining Software Repositories, Hyderabad, India.
46. Automated error diagnosis using abductive inference10.1145/2345156.2254087 ↗
47. Deepbugs: A learning approach to name-based bug detection10.1145/3276517 ↗
48. Mining fix patterns for findbugs violations10.1109/TSE.2018.2884955 ↗
49. Wang, J., Wang, S., and Wang, Q. (2018, January 11–12). Is there a “golden” feature set for static warning identification? An experimental evaluation. Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, Oulu, Finland.10.1145/3239235.3239523 ↗
50. Yang, X., and Menzies, T. (2021, January 23–28). Documenting evidence of a reproduction of ‘is there a “golden” feature set for static warning identification?—An experimental evaluation’. Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Athens, Greece.10.1145/3468264.3477220 ↗
51. Kang, H.J., Aw, K.L., and Lo, D. (2022, January 25–27). Detecting false alarms from automatic static analysis tools: How far are we?. Proceedings of the 44th International Conference on Software Engineering, Pittsburgh, PA, USA.10.1145/3510003.3510214 ↗
52. How to Find Actionable Static Analysis Warnings: A Case Study With FindBugs10.1109/TSE.2023.3234206 ↗