journal article·2020-03-02Open access

Automated Vulnerability Detection in Source Code Using Minimum Intermediate Representation Learning

Xin Li, Lu Wang, Yang Xin, Yixian Yang, Yuling Chen

Applied Sciences· Volume 10· Issue 5

Abstract

Vulnerability is one of the root causes of network intrusion. An effective way to mitigate security threats is to discover and patch vulnerabilities before an attack. Traditional vulnerability detection methods rely on manual participation and incur a high false positive rate. The intelligent vulnerability detection methods suffer from the problems of long-term dependence, out of vocabulary, coarse detection granularity and lack of vulnerable samples. This paper proposes an automated and intelligent vulnerability detection method in source code based on the minimum intermediate representation learning. First, the sample in the form of source code is transformed into a minimum intermediate representation to exclude the irrelevant items and reduce the length of the dependency. Next, the intermediate representation is transformed into a real value vector through pre-training on an extended corpus, and the structure and semantic information are retained. Then, the vector is fed to three concatenated convolutional neural networks to obtain high-level features of vulnerability. Last, a classifier is trained using the learned features. To validate this vulnerability detection method, an experiment was performed. The empirical results confirmed that compared with the traditional methods and the state-of-the-art intelligent methods, our method has a better performance with fine granularity.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Advanced Malware Detection Techniques · Web Application Security Vulnerabilities · Software Engineering Research

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing 100876, China

Guizhou Provincial Key Laboratory of Public Big Data, Guizhou University, Guiyang 550025, China

National Engineering Laboratory for Disaster Backup Recovery, Beijing 100876, China

Funding

National Key R&D Program of China · Major Scientific and Technological Special Project of Guizhou Province · Foundation of Guizhou Provincial Key Laboratory of Public Big Data

References (39)

Show source-provided references
1. (2019, December 29). WannaCry Ransomware Attack. Available online: https://en.wikipedia.org/wiki/WannaCry_ransomware_attack.
2. (2019, December 29). National Vulnerability Database, Available online: https://nvd.nist.gov.
3. Brooks, T.N. (2018, January 10–12). Survey of automated vulnerability detection and exploit generation techniques in cyber reasoning systems. Proceedings of the Science and Information Conference, London, UK.10.1007/978-3-030-01177-2_79 ↗
4. (2019, December 29). Flawfinder Software Official Website. Available online: https://www.dwheeler.com/flawfinder/.
5. (2019, December 29). CheckMarx Software Official Website. Available online: https://www.checkmarx.com.
6. Cadar, C., Dunbar, D., and Engler, D.R. (2008, January 8–12). KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs. Proceedings of the OSDI, San Diego, CA, USA.
7. Henzinger, T.A., Jhala, R., Majumdar, R., and Sutre, G. (2003, January 9–10). Software verification with BLAST. Proceedings of the International SPIN Workshop on Model Checking of Software, Portland, OR, USA.10.1007/3-540-44829-2_17 ↗
8. Coverage-based greybox fuzzing as markov chain
9. Stephens, N., Grosen, J., Salls, C., Dutcher, A., Wang, R., Corbetta, J., Shoshitaishvili, Y., Kruegel, C., and Vigna, G. (2016, January 21–24). Driller: Augmenting Fuzzing Through Selective Symbolic Execution. Proceedings of the Network and Distributed System Security Symposium, San Diego, CA, USA.10.14722/ndss.2016.23368 ↗
10. (2019, December 29). Cyber Grand Challenge. Available online: https://www.darpa.mil/program/cyber-grand-challenge.
11. Software vulnerability analysis and discovery using machine-learning and data-mining techniques: A survey
12. Awad, Y., Yashwant, M., Charles, A., and Indrajit, R. (2016, January 9–11). To fear or not to fear that is the question: Code characteristics of a vulnerable function with an existing exploit. Proceedings of the 6th ACM Conference on Data and Application Security and Privacy (CODASPY’16), New Orleans, LA, USA.
13. Evaluating complexity, code churn, and developer activity metrics as indicators of software vulnerabilities10.1109/TSE.2010.81 ↗
14. Perl, H., Dechand, S., Smith, M., Arp, D., Yamaguchi, F., Rieck, K., and Acar, Y. (2015, January 12–16). Vccfinder: Finding potential vulnerabilities in open-source projects to assist code audits. Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, Denver, CO, USA.10.1145/2810103.2813604 ↗
15. A survey of machine learning for big code and naturalness10.1145/3212695 ↗
16. Wang, S., Chollak, D., Movshovitz-Attias, D., and Tan, L. (2016, January 3–7). Bugram: bug detection with n-gram language models. Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, Singapore, Singapore.10.1145/2970276.2970341 ↗
17. Grieco, G., Grinblat, G.L., Uzal, L., Rawat, S., Feist, J., and Mounier, L. (2016, January 9–11). Toward large-scale vulnerability discovery using machine learning. Proceedings of the Sixth ACM Conference on Data and Application Security and Privacy, New Orleans, LA, USA.10.1145/2857705.2857720 ↗
18. Yamaguchi, F., Maier, A., Gascon, H., and Rieck, K. (2015, January 17–21). Automatic inference of search patterns for taint-style vulnerabilities. Proceedings of the 2015 IEEE Symposium on Security and Privacy, San Jose, CA, USA.10.1109/SP.2015.54 ↗
19. Kim, S., Woo, S., Lee, H., and Oh, H. (2017, January 22–24). Vuddy: A scalable approach for vulnerable code clone discovery. Proceedings of the 2017 IEEE Symposium on Security and Privacy (SP), San Jose, CA, USA.10.1109/SP.2017.62 ↗
20. Wang, S., Liu, T., and Tan, L. (2016, January 14–22). Automatically learning semantic features for defect prediction. Proceedings of the 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE), Austin, TX, USA.10.1145/2884781.2884804 ↗
21. Lin, G., Zhang, J., Luo, W., Pan, L., and Xiang, Y. (November, January 30). POSTER: Vulnerability discovery with function representation learning from unlabeled projects. Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, Dallas, TX, USA.10.1145/3133956.3138840 ↗
22. Cross-project transfer representation learning for vulnerable function discovery10.1109/TII.2018.2821768 ↗
23. Pradel, M., and Sen, K. (2018, January 7–9). DeepBugs: A learning approach to name-based bug detection. Proceedings of the ACM on Programming Languages(OOPSLA), Boston, MA, USA.10.1145/3276517 ↗
24. Detecting bugs by discovering expectations and their violations10.1109/TSE.2018.2816639 ↗
25. Li, Z., Zou, D., Xu, S., Ou, X., Jin, H., Wang, S., Deng, Z., and Zhong, Y. (2018, January 18–21). Vuldeepecker: A deep learning-based system for vulnerability detection. Proceedings of the 25th Annual Network and Distributed System Security Symposium (NDSS), San Diego, CA, USA.10.14722/ndss.2018.23158 ↗
26. Li, Z., Zou, D., Xu, S., Jin, H., Zhu, Y., Chen, Z., and Wang, J. (2018). SySeVR: A framework for using deep learning to detect software vulnerabilities. arXiv.
27. Zhou, Y., Liu, S., Siow, J., Du, X., and Liu, Y. (2019, January 8–14). Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.
28. Cummins, C., Petoumenos, P., Wang, Z., and Leather, H. (2017, January 4–8). Synthesizing benchmarks for predictive modeling. Proceedings of the 2017 International Symposium on Code Generation and Optimization, Austin, TX, USA.10.1109/CGO.2017.7863731 ↗
29. Allamanis, M., Peng, H., and Sutton, C. (2016, January 19–24). A convolutional attention network for extreme summarization of source code. Proceedings of the International Conference on Machine Learning, New York, NY, USA.
30. Gu, X., Zhang, H., and Kim, S. (June, January 27). Deep code search. Proceedings of the 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE), Gothenburg, Sweden.10.1145/3180155.3180167 ↗
31. Mou, L., Li, G., Zhang, L., Wang, T., and Jin, Z. (2016, January 12–17). Convolutional neural networks over tree structures for programming language processing. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.10.1609/aaai.v30i1.10139 ↗
32. Brockschmidt, M., Allamanis, M., Gaunt, A.L., and Polozov, O. (2018). Generative code modeling with graphs. arXiv.
33. Ben-Nun, T., Jakobovits, A.S., and Hoefler, T. (2018, January 3–8). Neural code comprehension: A learnable representation of code semantics. Proceedings of the Advances in Neural Information Processing Systems, Montréal, QC, Canada.
34. Yamaguchi, F., Golde, N., Arp, D., and Rieck, K. (2014, January 18–21). Modeling and discovering vulnerabilities with code property graphs. Proceedings of the 2014 IEEE Symposium on Security and Privacy, San Jose, CA, USA.10.1109/SP.2014.44 ↗
35. (2019, December 29). 2019 CWE Top 25 Most Dangerous Software Errors. Available online: https://cwe.mitre.org/top25/archive/2019/2019_cwe_top25.html.
36. Pantel, P. (2005, January 25–30). Inducing ontological co-occurrence vectors. Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, Ann Arbor, MI, USA.10.3115/1219840.1219856 ↗
37. (2019, December 29). Gensim: Word2Vec Model. Available online: https://radimrehurek.com/gensim/auto_examples/tutorials/run_word2vec.html.
38. (2019, December 29). SARD Manual, Available online: https://samate.nist.gov/index.php/SARD.html.
39. (2019, December 29). Common Vulnerabilities and Exposures. Available online: https://cve.mitre.org.