journal article·2020-11-28Open access

Machine Learning-Based Code Auto-Completion Implementation for Firmware Developers

Junghyun Kim, Kyuman Lee, Sanghyun Choi

Applied Sciences· Volume 10· Issue 23

Abstract

With the advent of artificial intelligence, the research paradigm in natural language processing has been transitioned from statistical methods to machine learning-based approaches. One application is to develop a deep learning-based language model that helps software engineers write code faster. Although there have already been many attempts to develop code auto-completion functionality from different research groups, a need to establish an in-house code has been identified for the following reasons: (1) a security-sensitive company (e.g., Samsung Electronics) may not want to utilize commercial tools given that there is a risk of leaked source codes and (2) commercial tools may not be applicable to the specific domain (e.g., SSD firmware development) especially if one needs to predict unique code patterns and style. This research proposes a hybrid approach that harnesses the synergy between machine learning techniques and advanced design methods aiming to develop a code auto-completion framework that helps firmware developers write code in a more efficient manner. The sensitivity analysis results show that the deterministic design results in reducing prediction accuracy as it generates output in some unexpected ways, while the probabilistic design provides a list of reasonable next code elements in which one could select it manually to increase prediction accuracy.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Software Engineering Research · Advanced Malware Detection Techniques · Software Reliability and Analysis Research

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Computational Science and Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA

Department of Robot and Smart System Engineering, Kyungpook National University, Daegu 41566, Korea

Memory S/W Development Team, Samsung Electronics, Hwasung 18448, Korea

Funding

Kyungpook National University

References (22)

Show source-provided references
1. Kim, J., Briceno, S., Justin, C., and Mavris, D. (2020, January 10). A Data-Driven Approach Using Machine Learning to Enable Real-Time Flight Path Planning. Proceedings of the AIAA Aviation 2020 Forum, USA. Available online: https://arc.aiaa.org/doi/abs/10.2514/6.2020-2873.10.2514/6.2020-2873 ↗
2. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv.
3. Language models are unsupervised multitask learners
4. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020). Language models are few-shot learners. arXiv.
5. (2020, November 12). The illustrated GPT-2 (Visualizing Transformer Language Models). Available online: http://jalammar.github.io/illustrated-gpt2/.
6. (2020, November 12). AllenNLP Language Modeling. Available online: https://demo.allennlp.org/next-token-lm?text=AllenNLP%20is.
7. (2020, October 19). Working with Content Assist. Available online: https://www.eclipse.org/pdt/help/html/working_with_code_assist.htm.
8. Das, S., and Shah, C. (2015). Contextual Code Completion Using Machine Learning, Stanford University. Technical Report.
9. Intelligent code completion with Bayesian networks10.1145/2744200 ↗
10. Raychev, V., Vechev, M., and Yahav, E. (2014, January 9–11). Code completion with statistical language models. Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation, Edinburgh, UK.10.1145/2594291.2594321 ↗
11. (2020, November 27). Neural Code Completion. Available online: https://openreview.net/pdf?id=rJbPBt9lg.
12. Karampatsis, R.M., Babii, H., Robbes, R., Sutton, C., and Janes, A. (2020). Big Code!= Big Vocabulary: Open-Vocabulary Models for Source Code. arXiv.10.1145/3377811.3380342 ↗
13. White, M., Vendome, C., Linares-Vásquez, M., and Poshyvanyk, D. (2015, January 16–17). Toward deep learning software repositories. Proceedings of the 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories, Florence, Italy.10.1109/MSR.2015.38 ↗
14. Li, J., Wang, Y., Lyu, M.R., and King, I. (2017). Code completion with neural attention and pointer networks. arXiv.10.24963/ijcai.2018/578 ↗
15. Svyatkovskiy, A., Deng, S.K., Fu, S., and Sundaresan, N. (2020). IntelliCode Compose: Code Generation Using Transformer. arXiv.10.1145/3368089.3417058 ↗
16. Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., Gao, J., Zhou, M., and Hon, H.W. (2019). Unified language model pre-training for natural language understanding and generation. arXiv.
17. (2020, October 19). Autocompletion with Deep Learning. Available online: https://www.tabnine.com/blog/deep/.
18. (2020, October 19). Better Language Models and Their Implications. Available online: https://openai.com/blog/better-language-models/.
19. Multi-objective Optimization of Departure Procedures at Gimpo International Airport10.1007/s42405-018-0027-1 ↗
20. Simplessd: Modeling solid state drives for holistic system simulation10.1109/LCA.2017.2750658 ↗
21. (2020, October 19). SimpleSSD Version 2.0 GitHub. Available online: https://github.com/SimpleSSD/SimpleSSD.
22. (2020, October 19). Beginner’s Guide to Retrain GPT-2 (117M) to Generate Custom Text Content. Available online: https://medium.com/@ngwaifoong92/beginners-guide-to-retrain-gpt-2-117m-to-generate-custom-text-content-8bb5363d8b7f.