Automatic Spatial Audio Scene Classification in Binaural Recordings of Music
Source abstract
Abstract
The aim of the study was to develop a method for automatic classification of the three spatial audio scenes, differing in horizontal distribution of foreground and background audio content around a listener in binaurally rendered recordings of music. For the purpose of the study, audio recordings were synthesized using thirteen sets of binaural-room-impulse-responses (BRIRs), representing room acoustics of both semi-anechoic and reverberant venues. Head movements were not considered in the study. The proposed method was assumption-free with regards to the number and characteristics of the audio sources. A least absolute shrinkage and selection operator was employed as a classifier. According to the results, it is possible to automatically identify the spatial scenes using a combination of binaural and spectro-temporal features. The method exhibits a satisfactory classification accuracy when it is trained and then tested on different stimuli but synthesized using the same BRIRs (accuracy ranging from 74% to 98%), even in highly reverberant conditions. However, the generalizability of the method needs to be further improved. This study demonstrates that in addition to the binaural cues, the Mel-frequency cepstral coefficients constitute an important carrier of spatial information, imperative for the classification of spatial audio scenes.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Music and Audio Processing · Speech and Audio Processing · Hearing Loss and Rehabilitation
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Computer Science, Białystok University of Technology, 15-351 Białystok, Poland
Applied Psychoacoustics Laboratory (APL), University of Huddersfield, Huddersfield HD1 3DH, UK
Funding
Ministerstwo Nauki i Szkolnictwa Wyższego
References (60)
Show source-provided references
1. Kelion, L. (2016, April 18). YouTube Live-Streams in Virtual Reality and Adds 3D Sound, BBC News. Available online: http://www.bbc.com/news/technology-36073009.
2. Parnell, T. (2017, July 14). Binaural Audio at the BBC Proms. Available online: https://www.bbc.co.uk/rd/blog/2016-09-binaural-proms.
3. (2016, July 25). Omnitone: Spatial Audio on the Web, Google, USA. Available online: https://opensource.googleblog.com/2016/07/omnitone-spatial-audio-on-web.html.
4. Blauert, J. (2013). The Technology of Binaural Listening, Springer.10.1007/978-3-642-37762-4 ↗
5. Spatial quality evaluation for reproduced sound: Terminology, meaning, and a scene-based paradigm
6. May, T., Ma, N., and Brown, G.J. (2015, January 19–24). Robust localisation of multiple speakers exploiting head movements and multi-conditional training of binaural cues. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, Australia.10.1109/ICASSP.2015.7178457 ↗
7. Ma, N., and Brown, G.J. (2016, January 8–12). Speech localisation in a multitalker mixture by humans and machines. Proceedings of the INTERSPEECH 2016, San Francisco, CA, USA.10.21437/Interspeech.2016-1149 ↗
8. Robust Binaural Localization of a Target Sound Source by Combining Spectral Source Models and Deep Neural Networks10.1109/TASLP.2018.2855960 ↗
9. Binaural Localization of Multiple Sound Sources by Non-Negative Tensor Factorization10.1109/TASLP.2018.2806745 ↗
10. Lovedee-Turner, M., and Murphy, D. (2018). Application of Machine Learning for the Spatial Analysis of Binaural Room Impulse Responses. Appl. Sci., 8.10.3390/app8010105 ↗
11. A place theory of sound localization10.1037/h0061495 ↗
12. Binaural processing model based on contralateral inhibition. I. Model structure10.1121/1.1383297 ↗
13. Han, Y., Park, J., and Lee, K. (2017, January 16). Convolutional neural networks with binaural representations and background subtraction for acoustic scene classification. Proceedings of the Conference on Detection and Classification of Acoustic Scenes and Events 2017, Munich, Germany.
14. Blauert, J. (1974). Spatial Hearing. The Psychology of Human Sound Localization, The MIT Press.
15. Käsbach, J., Marschall, M., Epp, B., and Dau, T. (2013, January 18–21). The relation between perceived apparent source width and interaural cross-correlation in sound reproduction spaces with low reverberation. Proceedings of the DAGA 2013, Merano, Italy.
16. Spectral Weighting Underlies Perceived Sound Elevation
17. Raake, A. (2019, March 08). A Computational Framework for Modelling Active Exploratory Listening that Assigns Meaning to Auditory Scenes—Reading the World with Two Ears. Available online: http://twoears.eu.
18. Ibrahim, K.M., and Allam, M. (2018, January 15–20). Primary-ambient source separation for upmixing to surround sound systems. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, Canada.10.1109/ICASSP.2018.8461459 ↗
19. Dynamic Precedence Effect Modeling for Source Separation in Reverberant Environments10.1109/TASL.2010.2051354 ↗
20. Zieliński, S.K., and Lee, H. (2018, January 9–12). Feature Extraction of Binaural Recordings for Acoustic Scene Classification. Proceedings of the 2018 Federated Conference on Computer Science and Information Systems (FedCSIS), Poznań, Poland.10.15439/2018F182 ↗
21. Sturm, B.L. (2014). A Survey of Evaluation in Music Genre Recognition. Adaptive Multimedia Retrieval: Semantics, Context, and Adaptation, Springer. Lecture Notes in Computer Science.10.1007/978-3-319-12093-5_2 ↗
22. Development and Initial Validation of a Multichannel Audio Quality Expert System
23. Zieliński, S.K. (2018). Feature Extraction of Surround Sound Recordings for Acoustic Scene Classification. Artificial Intelligence and Soft Computing, Proceedings of the ICAISC 2018, Springer. Lecture Notes in Computer Science.10.1007/978-3-319-91262-2_43 ↗
24. Zieliński, S.K. (2019). Spatial Audio Scene Characterization (SASC). Automatic Classification of Five-Channel Surround Sound Recordings According to the Foreground and Background Content. Multimedia and Network Information Systems, Proceedings of the MISSI 2018, Springer. Advances in Intelligent Systems and Computing.10.1007/978-3-319-98678-4_46 ↗
25. Beresford, K., Zieliński, S., and Rumsey, F. (2006, January 20–23). Listener Opinions of Novel Spatial Audio Scenes. Proceedings of the 120th AES Convention, Paris, France.
26. Lee, H., and Millns, C. (2017, January 21). Microphone Array Impulse Response (MAIR) Library for Spatial Audio Research. Proceedings of the 143rd AES Convention, New York, NY, USA.
27. Effects of Down-Mix Algorithms on Quality of Surround Sound
28. Computational models of auditory scene analysis: A review
29. Joint Mixing Vector and Binaural Model Based Stereo Source Separation10.1109/TASLP.2014.2320637 ↗
30. Acoustic scene classification: Classifying environments from the sounds they produce10.1109/MSP.2014.2326181 ↗
31. Anechoic Recording System for Symphony Orchestra10.3813/AAA.918104 ↗
32. Recordings of Italian opera orchestra and soloists in a silent room
33. (2019, March 08). Mixing Secrets for The Small Studio. Available online: http://www.cambridge-mt.com/ms-mtk.htm.
34. Bittner, R., Salamon, J., Tierney, M., Mauch, M., Cannam, C., and Bello, J.P. (2014, January 27). MedleyDB: A Multitrack Dataset for Annotation-Intensive MIR Research. Proceedings of the 15th International Society for Music Information Retrieval Conference, Taipei, Taiwan.
35. Studio Sessions (2019, March 08). Telefunken Elektroakustik. Available online: https://telefunken-elektroakustik.com/multitracks.
36. Satongar, D., Lam, Y.W., and Pike, C.H. (2014, January 13–17). Measurement and analysis of a spatially sampled binaural room impulse response dataset. Proceedings of the 21st International Congress on Sound and Vibration, Beijing, China.
37. Stade, P., Bernschütz, B., and Rühl, M. (2012, January 20). A Spatial Audio Impulse Response Compilation Captured at the WDR Broadcast Studios. Proceedings of the 27th Tonmeistertagung—VDT International Convention, Cologne, Germany.
38. Wierstorf, H. (2016, October 14). Binaural Room Impulse Responses of a 5.0 Surround Setup for Different Listening Positions. Zenodo. Available online: https://zenodo.org.
39. Sound Source and Loudspeaker Base Angle Dependency of Phantom Image Elevation Effect10.17743/jaes.2017.0028 ↗
40. Werner, S., Voigt, M., and Klein, F. (2018, July 26). Dataset of Measured Binaural Room Impulse Responses for Use in an Position-Dynamic Auditory Augmented Reality Application. Zenodo. Available online: https://zenodo.org.
41. Klein, F., Werner, S., Chilian, A., and Gadyuchko, M. (2017, January 20–23). Dataset of In-The-Ear and Behind-The-Ear Binaural Room Impulse Responses used for Spatial Listening with Hearing Implants. Proceedings of the 142nd AES Convention, Berlin, Germany.
42. Erbes, V., Geier, M., Weinzierl, S., and Spors, S. (2015, January 7–10). Database of single-channel and binaural room impulse responses of a 64-channel loudspeaker array. Proceedings of the 138th AES Convention, Warsaw, Poland.
43. Virtual Sound Source Positioning Using Vector Base Amplitude Panning
44. Politis, A. (2015, January 01). Vector-Base Amplitude Panning Library. Available online: https://github.com.
45. Wierstorf, H., and Spors, S. (2012, January 26–29). Sound Field Synthesis Toolbox. Proceedings of the 132nd AES Convention, Budapest, Hungary.
46. Rabiner, L., Juang, B.-H., and Yegnanarayana, B. (2008). Fundamentals of Speech Recognition, Pearson India.
47. A quantitative model of the “effective” signal processing in the auditory system. I. Model structure10.1121/1.414959 ↗
48. Computational auditory scene analysis10.1006/csla.1994.1016 ↗
49. Development and Validation of an Unintrusive Model for Predicting the Sensation of Envelopment Arising from Surround Sound Recordings
50. Spatial audio quality perception (part 2): A linear regression model10.17743/jaes.2014.0047 ↗
51. Lerch, A. (2012). An Introduction to Audio Content Analysis Applications in Signal. Processing and Music Informatics, IEEE Press.10.1002/9781118393550 ↗
52. Extracting audio descriptors from musical signals10.1121/1.3642604 ↗
53. Jensen, K., and Andersen, T.H. (2004). Real-time beat estimation using feature extraction. Computer Music Modeling and Retrieval, Springer. Lecture Notes in Computer Science.10.1007/978-3-540-39900-1_2 ↗
54. Scheirer, E., and Slaney, M. (1997, January 21–24). Construction and evaluation of a robust multifeature speech/music discriminator. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Munich, Germany.
55. McCree, A., Sell, G., and Garcia-Romero, D. (2017, January 20–24). Extended Variability Modeling and Unsupervised Adaptation for PLDA Speaker Recognition. Proceedings of the INTERSPEECH 2017, Stockholm, Sweden.10.21437/Interspeech.2017-1586 ↗
56. A Deep Learning Method for Chinese Singer Identification10.26599/TST.2018.9010121 ↗
57. Collinearity: A review of methods to deal with it and a simulation study evaluating their performance10.1111/j.1600-0587.2012.07348.x ↗
58. James, G., Witten, D., Hastie, T., and Tibshirani, R. (2017). An Introduction to Statistical Learning with Applications in R, Springer.
59. Kuhn, M. (2018, May 26). The Caret Package. Available online: https://topepo.github.io/caret.
60. Resolution of Front–Back Ambiguity in Spatial Hearing by Listener and Source Movement10.1121/1.426899 ↗