Generating Music with Data: Application of Deep Learning Models for Symbolic Music Composition
Source abstract
Abstract
Language models based on deep learning showed promising results for artistic generation purposes, including musical generation. However, the evaluation of symbolic musical generation models is mostly based on low-level mathematical metrics (e.g., the result of the loss function) due to the inherent difficulty in measuring the musical quality of a given performance due to the subjective nature of music. This work sought to measure and evaluate musical excerpts generated by deep learning models from a human perspective, limited to the scope of classical piano music generation. In this assessment, a population of 117 people performed blind tests with musical excerpts of human composition and musical excerpts generated using artificial intelligence models, including the models PerformanceRNN, Music Transformer, MuseNet and a custom model based on GRUs. The experiments demonstrated that musical excerpts generated using models based on the Transformer neural network architecture obtained the greatest receptivity within the tested population, surpassing the results of human compositions. In addition, the experiments also demonstrated that people with greater musical sensitivity and musical experience were more able to identify the compositional origin of the excerpts heard.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Music and Audio Processing · Music Technology and Sound Studies · Neuroscience and Music Perception
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Economics, Business and Accounting, University of São Paulo, São Paulo 05508-900, Brazil
Faculty of Business Administration, Accountability and Economics, Federal University of Goiás, Goiânia 74690-900, Brazil
Funding
Beslogic
References (28)
Show source-provided references
1. Music, language, and composition10.1093/mq/77.3.401 ↗
2. Walder, C. (2016, January 16–18). Modeling symbolic music: Beyond the piano roll. Proceedings of the Asian Conference on Machine Learning, Hamilton, New Zealand.
3. Patel, A.D. (2010). Music, Language, and the Brain, Oxford University Press.
4. Graves, A. (2013). Generating sequences with recurrent neural networks. arXiv.10.1007/978-3-642-24797-2_3 ↗
5. Mathews, A., Xie, L., and He, X. (2016, January 12–17). Senticap: Generating image descriptions with sentiments. Proceedings of the AAAI Con-ference on Artificial Intelligence, Phoenix, AZ, USA. No. 1.10.1609/aaai.v30i1.10475 ↗
6. Musicians make a standard: The MIDI phenomenon10.2307/3679619 ↗
7. Hawthorne, C., Stasyuk, A., Roberts, A., Simon, I., Huang, C.Z.A., Dieleman, S., Elsen, E., Engel, J., and Eck, D. (2018). Enabling factorized piano music modeling and generation with the MAESTRO dataset. arXiv.
8. Boulanger-Lewandowski, N., Bengio, Y., and Vincent, P. (2012). Modeling temporal dependencies in high-dimensional sequences: Application to polyphonic music generation and transcription. arXiv.10.1109/ICASSP.2013.6638244 ↗
9. Oore, S., Simon, I., Dieleman, S., Eck, D., and Simonyan, K. (2018). This time with feeling: Learning expressive musical performance. Neural Computing and Applications. arXiv.10.1007/s00521-018-3758-9 ↗
10. Huang, C.Z.A., Vaswani, A., Uszkoreit, J., Shazeer, N., Simon, I., Hawthorne, C., Dai, A.M., Hoffman, M.D., Dinculescu, M., and Eck, D. (2018). Music transformer. arXiv.
11. Ens, J., and Pasquier, P. (2020). Mmm: Exploring conditional multi-track music generation with the transformer. arXiv.
12. Huang, C.Z.A., Simon, I., and Dinculescu, M. (2023, February 04). Music Transformer: Generating Music with Long-Term Structure. Available online: https://magenta.tensorflow.org/music-transformer.
13. Perception of just-noticeable time displacement of a tone presented in a metrical sequence at different tempos10.1121/1.407650 ↗
14. A review of recurrent neural networks: LSTM cells and network architectures10.1162/neco_a_01199 ↗
15. Cho, K., Van Merriënboer, B., Bahdanau, D., and Bengio, Y. (2014). On the properties of neural machine translation: Encoder-decoder approaches. arXiv.10.3115/v1/W14-4012 ↗
16. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Advances in Neural Information Processing Systems, MIT Press.
17. Payne, C. (2023, February 04). Musenet. Available online: https://openai.com/blog/musenet.
18. Ji, S., Luo, J., and Yang, X. (2020). A comprehensive survey on deep music generation: Multi-level representations, algorithms, evaluations, and future directions. arXiv.
19. Chollet, F. (2023, February 04). Keras. Available online: https://github.com/keras-team/keras.
20. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G., Davis, A., Dean, J., and Devin, M. (2016). Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv.
21. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). Advances in Neural Information Processing Systems, MIT Press.
22. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., and Funtowicz, M. (2019). Huggingface’s transformers: State-of-the-art natural language processing. arXiv.10.18653/v1/2020.emnlp-demos.6 ↗
23. Lhoest, Q., del Moral, A., Jernite, Y., Thakur, A., von Platen, P., Patil, S., Chaumond, J., Drame, M., Plu, J., and Tunstall, L. (2021). Datasets: A Community Library for Natural Language Processing. arXiv.10.18653/v1/2021.emnlp-demo.21 ↗
24. Simon, I., and Oore, S. (2023, February 04). Performance RNN: Generating Music with Expressive Timing and Dynamics. Available online: https://magenta.tensorflow.org/performance-rnn.
25. Lee, Y. (2023, February 04). PyTorch implementation of Performance RNN, Inspired by Ian Simon and Sageev Oore. “Performance RNN: Generating Music with Expressive Timing and Dynamics. Magenta Blog. Available online: https://github.com/djosix/Performance-RNN-PyTorch.
26. Glorot, X., and Bengio, Y. (2010, January 13–15). Understanding the difficulty of training deep feedforward neural networks. Proceedings of the Thir-teenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings, Sardinia, Italy.
27. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
28. Language models are unsupervised multitask learners