ATOSE: Audio Tagging with One-Sided Joint Embedding
Source abstract
Abstract
Audio auto-tagging is the process of assigning labels to audio clips for better categorization and management of audio file databases. With the advent of advanced artificial intelligence technologies, there has been increasing interest in directly using raw audio data as input for deep learning models in order to perform tagging and eliminate the need for preprocessing. Unfortunately, most current studies of audio auto-tagging cannot effectively reflect the semantic relationships between tags—for instance, the connection between “classical music” and “cello”. In this paper, we propose a novel method that can enhance audio auto-tagging performance via joint embedding. Our model has been carefully designed and architected to recognize the semantic information within the tag domains. In our experiments using the MagnaTagATune (MTAT) dataset, which has high inter-tag correlations, and the Speech Commands dataset, which has no inter-tag correlations, we showed that our approach improves the performance of existing models when there are strong inter-tag correlations.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Music and Audio Processing · Speech and Audio Processing · Music Technology and Sound Studies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Com2uS Corporation, Seoul 08506, Republic of Korea
Department of Computer Engineering, Myongji University, Yongin 17058, Republic of Korea
Funding
National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) · 2022 Research Fund of Myongji University
References (29)
Show source-provided references
1. Introduction of decision trees10.1007/BF00116251 ↗
2. An introduction to kernel and nearest-neighbor nonparametric regression10.1080/00031305.1992.10475879 ↗
3. Support-vector networks10.1007/BF00994018 ↗
4. Detection and classification of acoustic scenes and events10.1109/TMM.2015.2428998 ↗
5. Mesaros, A., Heittola, T., Dikmen, O., and Virtanen, T. (2015, January 19–24). Sound event detection in real life recordings using coupled matrix factorization of spectral representations and class activity annotations. Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, QL, Australia.10.1109/ICASSP.2015.7177950 ↗
6. Li, T., Ogihara, M., and Li, Q. (August, January 28). A comparative study on content-based music genre classification. Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval (SIGIR ’03), Toronto, ON, Canada.
7. Rabiner, L., and Schafer, R. (2010). Theory and Applications of Digital Speech Processing, Pearson.
8. Dieleman, S., and Schrauwen, B. (2014, January 4–9). End-to-end learning for music audio. Proceedings of the International Conference on Acoustics, Speech and Signal Processing, (ICASSP), Florence, Italy.10.1109/ICASSP.2014.6854950 ↗
9. Lee, J., Park, J., Kim, K., and Nam, J. (2017, January 5–8). Sample-level deep convolutional neural networks for music auto-tagging using raw waveforms. Proceedings of the Sound and Music Computing Conference, (SMC), Espoo, Finland.
10. Hu, J., Shen, L., Albanie, S., Sun, G., and Wu, E. (2019, January 16–20). Squeeze-and-excitation networks. Proceedings of the Computer Vision and Pattern Recognition, (CVPR), Long Beach, CA, USA.10.1109/CVPR.2018.00745 ↗
11. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the Computer Vision and Pattern Recognition, (CVPR), Las Vegas, NV, USA.
12. Lee, J., Kim, T., and Nam, J. (2017). Raw waveform-based audio classification using sample-level CNN architectures. arXiv.
13. Kim, T., Lee, J., and Nam, J. (2018, January 15–20). Sample-level CNN architectures for music auto-tagging using raw waveforms. Proceedings of the International Conference on Acoustics, Speech and Signal Processing, (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462046 ↗
14. Won, M., Chun, S., Nieto, O., and Serra, X. (2020, January 4–8). Data-driven harmonic filters for audio representation learning. Proceedings of the International Conference on Acoustics, Speech and Signal Processing, (ICASSP), Online.10.1109/ICASSP40776.2020.9053669 ↗
15. Ravanelli, M., and Bengio, Y. (2018). Speaker recognition from raw waveform with sincnet. arXiv.10.1109/SLT.2018.8639585 ↗
16. Elizalde, B., Zarar, S., and Raj, B. (2019, January 12–17). Cross modal audio search and retrieval with joint embedding based in text and audio. Proceedings of the International Conference in Acoustics, Speech, and Signal Processing, (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8682632 ↗
17. Favory, X., Drossos, K., Virtanen, T., and Serra, X. (2020). Coala: Co-aligned autoencoders for learning semantically enriched audio representations. arXiv.
18. Baldi, P. (2011, January 2). Autoencoders, unsupervised learning, and deep architectures. Proceedings of the International Conference on Unsupervised and Trasnfer Learning Workshop, (UTLW), Bellevue, Washington, DC, USA.
19. Akbari, H., Yuan, L., Quian, R., Chuang, W.H., Chang, S.F., Cui, Y., and Gong, B. (2021). “VATT: Transformers for multimodal self-supervised learning from raw video, audio and text. arXiv.
20. Ba, J.L., Kiros, J.R., and Hinton, G.E. (2016). Layer normalization. arXiv.
21. Won, M., Chun, S., and Serra, X. (2019). Toward interpretable music tagging with self-attention. arXiv.
22. Law, E., West, K., Mandel, M., Bay, M., and Downie, J.S. (2009, January 26–30). Evaluation of algorithms using games: The case of music tagging. Proceedings of the International Society for Music Information Retrieval, (ISMIR), Kobe, Japan.
23. Warden, P. (2018). Speech Commands: A dataset for limited-vocabulary speech recognition. arXiv.
24. Xu, Y., Kong, Q., Wang, W., and Plumbley, M.D. (2017, January 16–17). Survey-cvssp system for dcase2017 challenge task 4. Proceedings of the Detection and Classification of Acoustic Scenes and Events, (DCASE), Munich, Germany.
25. Gemmeke, J.F., Ellis, P.D., Freedman, D., Jansen, A., Lawrence, W., Moore, R.C., Plakal, M., and Ritter, M. (2017, January 5–9). Audioset: An ontology and human-labeled dataset for audio events. Proceedings of the International Conference on Acoustics, Speech and Signal Processing. (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7952261 ↗
26. Huang, Q., Jansen, A., Lee, J., Ganti, R., Li, J., and Ellis, D. (2022). Mulan: A joint embedding of music audio and natural language. arXiv.
27. Manco, I., Benetos, E., Quinton, E., and Fazekas, G. (2022, January 22–27). Learning music audio representations via weak language supervision. Proceedings of the 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9746996 ↗
28. Zhong, Z., Hirano, M., Shimada, K., Tateishi, K., Takahashi, S., and Mitsufuji, Y. (2023, January 4–10). An Attention-Based Approach to Hierarchical Multi-Label Music Instrument Classification. Proceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes, Greece.10.1109/ICASSP49357.2023.10095162 ↗
29. Movie genre classification: A multi-label approach based on convolutions through time10.1016/j.asoc.2017.08.029 ↗