Enhancements to Neural Language Model for Generating System Configuration Code: A Study with Maven Dependency
Source abstract
Abstract
Thanks to the widespread application of software frameworks, OTS components, DSLs, and new-generation software building and construction systems, leveraging system configuration code to assist in development is increasingly common, especially in agile software development. In software system implementation, the system configuration code is used to separate configuration items from the underlying logic, of which Maven dependency code is a typical example. To improve software productivity, developers often reuse existing dependency libraries. However, the large quantity, rapid iteration, and various application scenarios exacerbate the challenge for researchers to reuse the library efficiently and appropriately. Proper reuse of Maven dependencies requires correct importation, which is the research priority of this article; putting it into practical usage at the functional level is the next step. In order to overcome this barrier, researchers have proposed a number of recommendation and intelligent generation models based on deep learning algorithms and code learning strategies. We first introduce an enhancement path for the generation model in order to propose novel models that are more targeted than previous studies. We propose EMGSCC (Enhanced Model for Generating System Configuration Code), which generates accompanying dependency libraries based on the libraries already employed by the current system. EMGSCC uses content-based attention to cope with dependency language features and integrate additional domain information. Finally, we evaluate EMGSCC on the DDDI dataset with extra domain information, and findings show that improvement varies from 1% to 8% on all metrics compared with the baseline. We show empirical evidence of our enhancement path for generating system configuration code based on neural language models, and continuous improvement in this direction would yield promising results.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Software Engineering Research · Software Engineering Techniques and Practices · Software System Performance and Reliability
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Software Engineering, Tongji University, Shanghai 201804, China
Funding
Science and Technology Commission of Shanghai Municipality
References (42)
Show source-provided references
1. Bass, L., Weber, I., and Zhu, L. (2015). DevOps: A Software Architect’s Perspective, Addison-Wesley Professional.
2. DeepLib: Machine translation techniques to recommend upgrades for third-party libraries10.1016/j.eswa.2022.117267 ↗
3. A survey of machine learning for big code and naturalness10.1145/3212695 ↗
4. Thung, F., Lo, D., and Lawall, J. (2013, January 14–17). Automated library recommendation. Proceedings of the 2013 20th Working Conference on Reverse Engineering (WCRE), Koblenz, Germany.10.1109/WCRE.2013.6671293 ↗
5. Search-based software library recommendation using multi-objective optimization10.1016/j.infsof.2016.11.007 ↗
6. Sun, Z., Liu, Y., Cheng, Z., Yang, C., and Che, P. (2020, January 18–21). Req2Lib: A Semantic Neural Model for Software Library Recommendation. Proceedings of the 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER), London, ON, Canada.10.1109/SANER48275.2020.9054865 ↗
7. Nguyen, P.T., Di Rocco, J., Rubei, R., Di Sipio, C., and Di Ruscio, D. (2022, June 02). Recommending Third-Party Library Updates with LSTM Neural Networks 2021. Available online: http://ceur-ws.org/Vol-2947/paper7.pdf.
8. Yang, C., Liu, Y., and Yin, C. (2021). Recent Advances in Intelligent Source Code Generation: A Survey on Natural Language Based Studies. Entropy, 23.10.3390/e23091174 ↗
9. Houidi, Z.B., and Rossi, D. (2022). Neural language models for network configuration: Opportunities and reality check. arXiv.
10. Benelallam, A., Harrand, N., Soto-Valero, C., Baudry, B., and Barais, O. (2019, January 25–31). The maven dependency graph: A temporal graph-based representation of maven central. Proceedings of the 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR), Montreal, QC, Canada.10.1109/MSR.2019.00060 ↗
11. A comprehensive study of bloated dependencies in the maven ecosystem10.1007/s10664-020-09914-8 ↗
12. (2022, July 05). Apache Maven Pom Introduction. Available online: https://maven.apache.org/guides/introduction/introduction-to-the-pom.html.
13. Blech, E., Grishchenko, A., Kniazkov, I., Liang, G., Serebrennikov, O., Tatarnikov, A., Volkhontseva, P., and Yakimets, K. (2021, January 25–28). Patternika: A Pattern-Mining-Based Tool For Automatic Library Migration. Proceedings of the 2021 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), Wuhan, China.10.1109/ISSREW53611.2021.00098 ↗
14. Thung, F., Wang, S., Lo, D., and Lawall, J. (2013, January 11–15). Automatic recommendation of API methods from feature requests. Proceedings of the 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE), Silicon Valley, CA, USA.10.1109/ASE.2013.6693088 ↗
15. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
16. Kaplan, J., McCandlish, S., Henighan, T., Brown, T.B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. (2020). Scaling laws for neural language models. arXiv.
17. Yin, P., and Neubig, G. (2017). A syntactic neural model for general-purpose code generation. arXiv.10.18653/v1/P17-1041 ↗
18. Rabinovich, M., Stern, M., and Klein, D. (2017). Abstract syntax networks for code generation and semantic parsing. arXiv.10.18653/v1/P17-1105 ↗
19. Zhong, H., Xie, T., Zhang, L., Pei, J., and Mei, H. (2009). MAPO: Mining and recommending API usage patterns. European Conference on Object-Oriented Programming, Springer.10.1007/978-3-642-03013-0_15 ↗
20. Bengio, Y., and LeCun, Y. (2016, January 2–4). Order Matters: Sequence to sequence for sets. Proceedings of the 4th International Conference on Learning Representations (ICLR 2016), San Juan, Puerto Rico. Conference Track Proceedings.
21. A review on multi-label learning algorithms10.1109/TKDE.2013.39 ↗
22. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
23. Sorower, M.S. (2010). A Literature Survey on Algorithms for Multi-Label Learning, Oregon State University.
24. Godbole, S., and Sarawagi, S. (2004). Discriminative methods for multi-labeled classification. Pacific-Asia Conference on Knowledge Discovery and Data Mining, Springer.10.1007/978-3-540-24775-3_5 ↗
25. Katsuragawa, D., Ihara, A., Kula, R.G., and Matsumoto, K. (June, January 27). Maintaining Third-Party Libraries through Domain-Specific Category Recommendations. Proceedings of the 2018 IEEE/ACM 1st International Workshop on Software Health (SoHeal), Gothenburg, Sweden.10.1145/3194124.3194129 ↗
26. He, H., He, R., Gu, H., and Zhou, M. (2021, January 20). A large-scale empirical study on Java library migrations: Prevalence, trends, and rationales. Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Athens, Greece.10.1145/3468264.3468571 ↗
27. Rubei, R., Di Ruscio, D., Di Sipio, C., Di Rocco, J., and Nguyen, P.T. (2022). Providing Upgrade Plans for Third-party Libraries: A Recommender System using Migration Graphs. arXiv.10.1007/s10489-021-02911-4 ↗
28. He, H., Xu, Y., Ma, Y., Xu, Y., Liang, G., and Zhou, M. (2021, January 9–12). A multi-metric ranking approach for library migration recommendations. Proceedings of the 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), Honolulu, HI, USA.10.1109/SANER50967.2021.00016 ↗
29. Bengio, Y., Ducharme, R., and Vincent, P. (2000). A neural probabilistic language model. Advances in Neural Information Processing Systems 13, MIT Press.
30. Mikolov, T., Karafiát, M., Burget, L., Cernockỳ, J., and Khudanpur, S. (2010, January 26–30). Recurrent neural network based language model. Proceedings of the Interspeech 2010, Makuhari, Japan.10.21437/Interspeech.2010-343 ↗
31. Sundermeyer, M., Schlüter, R., and Ney, H. (2012, January 9–13). LSTM neural networks for language modeling. Proceedings of the Thirteenth Annual Conference of the International Speech Communication Association, Portland, OR, USA.10.21437/Interspeech.2012-65 ↗
32. Balog, M., Gaunt, A.L., Brockschmidt, M., Nowozin, S., and Tarlow, D. (2016). Deepcoder: Learning to write programs. arXiv.
33. Ling, W., Grefenstette, E., Hermann, K.M., Kočiskỳ, T., Senior, A., Wang, F., and Blunsom, P. (2016). Latent predictor networks for code generation. arXiv.10.18653/v1/P16-1057 ↗
34. Chen, Q., and Zhou, M. (2018, January 3–7). A neural framework for retrieval and summarization of source code. Proceedings of the 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE), Montpellier, France.10.1145/3238147.3240471 ↗
35. Lee, D., Yoon, J., Song, J., Lee, S., and Yoon, S. (2019). One-shot learning for text-to-sql generation. arXiv.
36. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv.
37. Krishnamurthy, J., Dasigi, P., and Gardner, M. (2017, January 9–11). Neural semantic parsing with type constraints for semi-structured tables. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.10.18653/v1/D17-1160 ↗
38. Tai, K.S., Socher, R., and Manning, C.D. (2015). Improved semantic representations from tree-structured long short-term memory networks. arXiv.10.3115/v1/P15-1150 ↗
39. Stylianou, N., and Vlahavas, I. (2020). ET: Entity-transformers. coreference augmented neural language model for richer mention representations via entity-transformer blocks. arXiv.
40. Nguyen, A.T., Nguyen, T.D., Phan, H.D., and Nguyen, T.N. (2018, January 20–23). A deep neural network language model with contexts for source code. Proceedings of the 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER), Campobasso, Italy.10.1109/SANER.2018.8330220 ↗
41. Sun, Z. (2021). Research on Code Search Using Semantic Vector Matching. [Master’s Thesis, Tongji University].
42. Liu, F., Li, G., Fu, Z., Lu, S., Hao, Y., and Jin, Z. (2022). Learning to Recommend Method Names with Global Context. arXiv.10.1145/3510003.3510154 ↗