Digital Technology in Cultural Heritage: Construction and Evaluation Methods of AI-Based Ethnic Music Dataset
Source abstract
Abstract
This study focuses on the construction and evaluation of a high-quality Chinese Manchu music dataset designed to facilitate Artificial Intelligence (AI) research and applications within cultural heritage and ethnomusicology. Through a systematic collection and organization of diverse Manchu music resources, including folk songs, dance music, and ceremonial pieces, this dataset effectively represents the cultural breadth of Manchu music. The dataset includes digitized and preprocessed audio data, with comprehensive metadata annotations, such as essential information, musical features, and cultural context, creating a robust foundation for AI-based analysis. Experimental evaluations highlight the dataset’s utility across various AI-driven applications: in music classification, using a CNN model, an accuracy of 90% was achieved in the “folk ensemble” category, with an overall accuracy of 85.7% and a precision of 82.3%. For music generation, a Generative Adversarial Network (GAN) model yielded a quality score of 7.8/10 and a Fréchet Audio Distance (FAD) of 0.32. In emotion recognition, the Random Forest model achieved 87% accuracy in identifying the emotion “joy”. These results underscore the dataset’s potential in supporting digital preservation and expanding AI applications in ethnic music classification, generation, and emotional analysis, contributing to both cultural heritage preservation and AI advancement in ethnomusicology.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Music and Audio Processing · Music Technology and Sound Studies · Diverse Musicological Studies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Academy of Music, Beihua University, Jilin City 132013, China
Department of Music Education, Kyungnam University, Changwon-si 51767, Republic of Korea
College of Computer Science and Technology, Beihua University, Jilin City 132013, China
Department of IT Convergence Engineering, Kyungnam University, Changwon-si 51767, Republic of Korea
Department of Computer Engineering, Kyungnam University, Changwon-si 51767, Republic of Korea
Funding
Jilin Provincial Department of Education · Beihua University · Jilin Province Social Science Foundation · Beihua University Education and Teaching Reform Research · Jilin Province Education Science · Ministry of Education’s Industry-University Collaborative Education
References (64)
Show source-provided references
1. Lin, Q., and Lian, Z. (2018). On Protection of Intangible Cultural Heritage in China from the Intellectual Property Rights Perspective. Sustainability, 10.10.3390/su10124369 ↗
2. Application of Digital Technology in Intangible Cultural Heritage Protection
3. Digital Preservation of Intangible Cultural Heritage
4. Traditional music protection from the perspective of intangible cultural heritage10.18282/l-e.v9i4.1689 ↗
5. National Music Promotion and Inheritance Strategies Based on the Perspective of Intangible Cultural Heritage
6. Research on Regional Characteristics and Cultural Value in the Inheritance of Intangible Cultural Heritage Music10.54097/jeer.v6i1.14233 ↗
7. Zhou, Y. (2017, January 9–11). Relevant Conceptions on the Inheritance and Protection of Manchu Music in Liaoning Province. Proceedings of the 2017 International Conference on Art Studies: Science, Experience, Education (ICASSEE 2017), Moscow, Russia.10.2991/icassee-17.2018.77 ↗
8. Musical Scholarship of the “Golden Age” of the Qing Dynasty Based on 17th and 18th Century Books and Treatises10.56620/RM.2024.3.101-108 ↗
9. Tian, Y., Meng, M., Mei, L., and Dong, S. (2021, January 23–25). Research on Development Mechanism and Strategy of Folk Art Industry in Northeast of China. Proceedings of the International Conference on Environmental and Engineering Management (EEM 2021), Changsha, China.
10. Wen, H.-Q. (2017, January 26–28). National Belonging Needs and Natural Historical Cultural Analysis of Chinese Manchu. Proceedings of the 3rd Annual International Conference on Management, Economics and Social Development (ICMESD 17), Guangzhou, China.10.2991/icmesd-17.2017.28 ↗
11. “My Culture, My People, My Hometown”: Chinese Ethnic Minorities Seeking Cultural Sustainability by Video Blogging10.1145/3579509 ↗
12. Dai, J., Wang, K., and Sun, Y. (2020, January 8–9). Analysis on the Inheritance and Development of Manchu Intangible Cultural Heritage in Changbai Mountain by the Creation of Animation Short Films in the New Media Era. Proceedings of the 7th International Conference on Education, Language, Art and Inter-cultural Communication (ICELAIC 2020), Moscow, Russia.10.2991/assehr.k.201215.361 ↗
13. (2023, February 20). Exploring the Ethnic Groups of China. Available online: https://www.cusef.org.hk/en/cusef-blog/exploring-the-ethnic-groups-of-china.
14. Miranda, E.R. (2021). Handbook of Artificial Intelligence for Music, Springer.10.1007/978-3-030-72116-9 ↗
15. Artificial Intelligence in Music Education10.1080/10447318.2023.2209984 ↗
16. A systematic review of artificial intelligence-based music generation: Scope, applications, and future trends10.1016/j.eswa.2022.118190 ↗
17. Kaliakatsos-Papakostas, M., Floros, A., and Vrahatis, M.N. (2020). Artificial intelligence methods for music generation: A review and future perspectives. Nature-Inspired Computation and Swarm Intelligence, Academic Press.10.1016/B978-0-12-819714-1.00024-5 ↗
18. Chen, M. (2019). Analysis on Industrialization Development Path of Intangible Cultural Heritages of Jilin Province. the Proceedings of the 3rd International Conference on Culture, Education and Economic Development of Modern Society (ICCESE 2019), Moscow, Russia, 1–3 March 2019, Atlantis Press.10.2991/iccese-19.2019.112 ↗
19. Musical Genre Classification of Audio Signals10.1109/TSA.2002.800560 ↗
20. Experimenting with the National Language: Use of Manchu in Bannermen Poetry and Songs in the Nineteenth Century
21. Chiu, E.S.-Y. (2018). Bannermen Tales (Zidishu): Manchu Storytelling and Cultural Hybridity in the Qing Dynasty, Brill.10.2307/j.ctv47w7tr ↗
22. Beyond Diverse Datasets: Responsible MIR, Interdisciplinarity, and the Fractured Worlds of Music
23. Vatolkin, I., Ginsel, P., and Rudolph, G. Advancements in the Music Information Retrieval Framework AMUSE over the Last Decade. SIGIR ’21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Montréal, Canada, 11–15 July 2021.10.1145/3404835.3463252 ↗
24. Fessahaye, F., Perez, L., Zhan, T., Zhang, R., Fossier, C., Markarian, R., Chiu, C., Zhan, J., Gewali, L., and Oh, P. (2019, January 8–11). T-RECSYS: A Novel Music Recommendation System Using Deep Learning. Proceedings of the 2019 IEEE International Conference on Consumer Electronics (ICCE), Berlin, Germany.10.1109/ICCE.2019.8662028 ↗
25. Wang, Z., Chen, K., Jiang, J., Zhang, Y., Xu, M., Dai, S., Gu, X., and Xia, G. (2020). Pop909: A pop-song dataset for music arrangement generation. arXiv.
26. Aljanaki, A., Yang, Y.H., and Soleymani, M. (2017). Developing a benchmark for emotional analysis of music. PLoS ONE, 12.10.1371/journal.pone.0173392 ↗
27. GAN computers generate arts? A survey on visual arts, music, and literary text generation using generative adversarial network10.1016/j.displa.2022.102237 ↗
28. Choi, K., Fazekas, G., Sandler, M., and Cho, K. (2017, January 5–9). Convolutional recurrent neural networks for music classification. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7952585 ↗
29. Huang, A., and Wu, R. (2016). Deep learning for music. arXiv.
30. Briot, J.P., Hadjeres, G., and Pachet, F.D. (2017). Deep learning techniques for music generation—A survey. arXiv.
31. Hernandez-Olivan, C., and Beltran, J.R. (2022). Music composition with deep learning: A review. Advances in Speech and Music Technology: Computational Aspects and Applications, Springer.10.1007/978-3-031-18444-4_2 ↗
32. Research on Cultural Confidence Leading the Inheritance and Development of Liaoning Manchu Folk Dance10.47297/wspiedWSP2516-250019.20210512 ↗
33. Bertin-Mahieux, T., Ellis, D.P.W., Whitman, B., and Lamere, P. (2011, January 24–28). The Million Song Dataset. Proceedings of the 12th International Conference on Music Information Retrieval (ISMIR), Miami, FL, USA. Available online: http://millionsongdataset.com.
34. Law, E., and Von Ahn, L. (2009, January 4–9). Input-Agreement: A New Mechanism for Collecting Data Using Human Computation Games. Proceedings of the 27th International Conference on Human Factors in Computing Systems (CHI), Boston, MA, USA. Available online: http://mirg.city.ac.uk/codeapps/the-magnatagatune-dataset.10.1145/1518701.1518881 ↗
35. Engel, J., Agrawal, K., Chen, S., Gulrajani, I., Donahue, C., and Roberts, A. (2017, January 4–9). Nsynth: A Large-Scale Dataset of Annotated Musical Notes. Proceedings of the 2017 Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, USA. Available online: https://magenta.tensorflow.org/datasets/nsynth.
36. Defferrard, M., Benzi, K., Vandergheynst, P., and Bresson, X. (2017, January 23–27). FMA: A Dataset for Music Analysis. Proceedings of the 18th Inter-national Society for Music Information Retrieval Conference (ISMIR), Suzhou, China. Available online: https://freemusicarchive.org/static.
37. Music Information Retrieval: Recent Developments and Applications10.1561/1500000042 ↗
38. Generative Adversarial Networks10.1145/3422622 ↗
39. Automatic Music Generator Using Recurrent Neural Network10.2991/ijcis.d.200519.001 ↗
40. Machine learning research that matters for music creation: A case study10.1080/09298215.2018.1515233 ↗
41. Bahuleyan, H. (2018). Music genre classification using machine learning techniques. arXiv.
42. Ndou, N., Ajoodha, R., and Jadhav, A. (2021, January 21–24). Music Genre Classification: A Review of Deep-Learning and Traditional Machine-Learning Approaches. Proceedings of the 2021 IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS), Toronto, ON, Canada.10.1109/IEMTRONICS52119.2021.9422487 ↗
43. Singing a Chinese Nation: Heritage Preservation, the Yuanshengtai Movement, and New Trends in Chinese Folk Music in the Twenty-First Century10.5406/ethnomusicology.65.1.0001 ↗
44. Fan, J., Yang, Y.-H., Dong, K., and Pasquier, P.A. (2020, January 4–8). Comparative Study of Western and Chinese Classical Music Based on Soundscape Models. Proceedings of the ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9052994 ↗
45. Jiang, F., Zhang, L., Wang, K., Deng, X., and Yang, W. (2022). BoYaTCN: Research on Music Generation of Traditional Chinese Pentatonic Scale Based on Bidirectional Octave Your Attention Temporal Convolutional Network. Appl. Sci., 12.10.3390/app12189309 ↗
46. Regional classification of Chinese folk songs based on CRF model10.1007/s11042-018-6637-6 ↗
47. Luo, J., Yang, X., Ji, S., and Li, J. (2019, January 26–29). MG-VAE: Deep Chinese folk songs generation with specific regional styles. Proceedings of the 7th Conference on Sound and Music Technology (CSMT): Revised Selected Papers, Harbin, China.10.1007/978-981-15-2756-2_8 ↗
48. Chen, Q., Zhao, W., Wang, Q., and Zhao, Y. (2022). The Sustainable Development of Intangible Cultural Heritage with AI: Cantonese Opera Singing Genre Classification Based on CoGCNet Model in China. Sustainability, 14.10.3390/su14052923 ↗
49. Li, Q., and Hu, B. (2023, January 20–24). Joint Time and Frequency Transformer for Chinese Opera Classification. Proceedings of the INTERSPEECH 2023, Dublin, Ireland.10.21437/Interspeech.2023-1582 ↗
50. Roche, F., Hueber, T., Limier, S., and Girin, L. (2018). Autoencoders for music sound modeling: A comparison of linear, shallow, deep, recurrent and variational models. arXiv.
51. Monophonic Music Generation with a Given Emotion Using Conditional Variational Autoencoder10.1109/ACCESS.2021.3113829 ↗
52. Pei, X. (2019, January 16–17). Analysis of Manchu Shaman Music Cultural Form from the Perspective of Art. Proceedings of the 2019 5th International Conference on Economics, Management and Humanities Science (ECOMHS 2019), Bangkok, Thailand.
53. Howard, K. (2017). Sacred and profane: Music in Korean shaman rituals. Indigenous Religious Musics, Routledge.10.4324/9781315092706-4 ↗
54. Seo, M.K. (2020). Hanyang Kut: Korean Shaman Ritual Music from Seoul, Routledge.10.4324/9780429286902 ↗
55. Howard, K. (2017). Shamanism, Music, and the Soul Train. Music as Medicine, Routledge.10.4324/9781315090894-15 ↗
56. Li, X. (2022). A General History of Chinese Art, Walter de Gruyter GmbH & Co KG.
57. (2019, March 06). The Musical Bridge–China: Perennial Music and Arts, “Intro to Chinese Music”. Available online: https://www.perennialmusicandarts.com/post/intro-to-chinese-music.
58. Dai, J., and Zhang, L. (2014, January 5–7). Manchu Intangible Cultural Heritage Protection Research Based on Digital Multimedia Technology. Proceedings of the International Conference on Education, Language, Art and Intercultural Communication (ICELAIC-14), Zhengzhou, China.10.2991/icelaic-14.2014.171 ↗
59. Research on the Technology of Virtual Reality Empowering Manchu Dance Cultural Communication
60. A Survey of Audio-Based Music Classification and Annotation10.1109/TMM.2010.2098858 ↗
61. Deep Convolutional Neural Networks for Predominant Instrument Recognition in Polyphonic Music10.1109/TASLP.2016.2632307 ↗
62. Dieleman, S., and Schrauwen, B. (2014, January 4–9). End-to-end learning for music audio. Proceedings of the ICASSP 2014–2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy.10.1109/ICASSP.2014.6854950 ↗
63. Sturm, B.L.T., Iglesias, M., Ben-Tal, O., Miron, M., and Gómez, E. (2019). Artificial Intelligence and Music: Open Questions of Copyright Law and Engineering Praxis. Arts, 8.10.3390/arts8030115 ↗
64. Lyons, F., Sun, H., Collopy, D.P., Curran, K., and Ohagan, P. (2024, March 06). Music 2025—The Music Data Dilemma: Issues Facing the Music Industry in Improving Data Management. Intellectual Property Office Research Paper. Published 18 June 2019. Available online: http://hdl.handle.net/2299/21408.10.2139/ssrn.3437670 ↗