Prediction and Controlling of Auditory Perception in Augmented Environments. A Loudness-Based Dynamic Mixing Technique
Source abstract
Abstract
At the core of augmented reality audio (ARA) technology lies the ARA mix, a process responsible for the assignment of a virtual environment to a real one. Legacy ARA mix models have focused on the natural reproduction of the real environment, whereas the virtual environment is simply mixed through fixed gain methods. This study presents a novel approach of a dynamic ARA mix that facilitates a smooth adaptation of the virtual environment to the real one, as well as dynamic control of the virtual audio engine, by taking into account the inherent characteristics of both ARA technology and binaural auditory perception. A prototype feature extraction technique of auditory perception characteristics through a real-time binaural loudness prediction method was used to upgrade the legacy ARA mix model into a dynamic model, which was evaluated through benchmarks and subjective tests and showed encouraging results in terms of functionality and acceptance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Hearing Loss and Rehabilitation · Tactile and Sensory Interactions · Image and Video Quality Assessment
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Audiovisual Signal Processing Laboratory, Ionian University, Plateia Tsirigoti 7, GR-491 00 Corfu, Greece
Funding
Operational Program Human Resources Development, Education and Life- long learning, Priority Axes 6, 8, 9, Act "Supporting Researchers with emphasis on New Researchers" which is co-financed by Greece and the European Union (European Social Fund - ESF)
References (42)
Show source-provided references
1. Augmented Reality Audio for Mobile and Wearable Appliances
2. Tikander, M., Karjalainen, M., and Riikonen, V. (2008, January 1–4). An augmented reality audio headset. Proceedings of the 11th International Conference on Digital Audio Effects DAFx-08, Espoo, Finland.
3. Digital augmented reality audio headset10.1155/2012/457374 ↗
4. LISTEN: A user-adaptive audio-augmented museum guide10.1007/s11257-008-9049-x ↗
5. Chatzidimitris, T., Gavalas, D., and Michael, D. (2016, January 18–20). SoundPacman: Audio augmented reality in location-based games. Proceedings of the 2016 18th Mediterranean Electrotechnical Conference (MELECON), Limassol, Cyprus.10.1109/MELCON.2016.7495414 ↗
6. Moustakas, N., Floros, A., and Kanellopoulos, N. (2009). Eidola: An interactive augmented reality audio-game prototype. Audio Engineering Society Convention 127, Audio Engineering Society.
7. Moustakas, N., Floros, A., and Grigoriou, N. (2011). Interactive audio realities: An augmented/mixed reality audio game prototype. Audio Engineering Society Convention 130, Audio Engineering Society.
8. Rovithis, E., Moustakas, N., Floros, A., and Vogklis, K. (2019). Audio Legends: Investigating Sonic Interaction in an Augmented Reality Audio Game. Multimodal Technol. Interact., 3.10.3390/mti3040073 ↗
9. Blum, J.R., Bouchard, M., and Cooperstock, J.R. (2011). What’s around me? Spatialized audio augmented reality for blind users with a smartphone. International Conference on Mobile and Ubiquitous Systems: Computing, Networking, and Services, Springer.10.1007/978-3-642-30973-1_5 ↗
10. Navigation and space perception assistance for the visually impaired: The NAVIG project10.1016/j.irbm.2012.01.009 ↗
11. Bederson, B.B. (1995, January 7–11). Audio augmented reality: A prototype automated tour guide. Proceedings of the Conference Companion on Human Factors in Computing Systems, Denver, CO, USA.10.1145/223355.223526 ↗
12. Mantell, J., Rod, J., Kage, Y., Delmotte, F., and Leu, J. (April, January 29). Navinko: Audio augmented reality-enabled social navigation for city cyclists. Proceedings of the Programme, Workshop Pervasive 2010 Proc, Mannheim, Germany.
13. Boletsis, C., and Chasanidou, D. (2018, January 26–29). Smart Tourism in Cities: Exploring Urban Destinations with Audio Augmented Reality. Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference, Corfu, Greece.10.1145/3197768.3201549 ↗
14. Boletsis, C., and Chasanidou, D. (October, January 29). Audio augmented reality in public transport for exploring tourist sites. Proceedings of the 10th Nordic Conference on Human-Computer Interaction, Oslo, Norway.
15. Moustakas, N., Floros, A., and Kapralos, B. (October, January 30). An Augmented Reality Audio Live Network for Live Electroacoustic Music Concerts. Proceedings of the Audio Engineering Society Conference: 2016 AES International Conference on Audio for Virtual and Augmented Reality, Los Angeles, CA, USA.
16. Karjalainen, M., Riikonen, V., and Tikander, M. (2008). An augmented reality audio mixer and equalizer. Audio Engineering Society Convention 124, Audio Engineering Society.
17. Liski, J., Väänänen, R., Vesa, S., and Välimäki, V. (2016, January 24–26). Adaptive equalization of acoustic transparency in an augmented-reality headset. Proceedings of the Audio Engineering Society Conference: 2016 AES International Conference on Headphone Technology, Aalborg, Denmark.
18. Albrecht, R., Lokki, T., and Savioja, L. (2011, January 30). A mobile augmented reality audio system with binaural microphones. Proceedings of the Interacting with Sound Workshop: Exploring Context-Aware, Local and Social Audio Applications, Stockholm, Sweden.10.1145/2019335.2019337 ↗
19. A model of loudness applicable to time-varying sounds
20. Dependence of binaural loudness summation on interaural level differences, spectral distribution, and temporal distribution10.1121/1.1894635 ↗
21. Ward, D., and Reiss, J.D. (2016, January 13). Loudness algorithms for automatic mixing. Proceedings of the 2nd AES Workshop on Intelligent Music Production, London, UK.
22. Reiss, J.D. (2016). An intelligent systems approach to mixing multitrack audio. Mixing Music, Routledge.
23. Natural listening over headphones in augmented reality using adaptive filtering techniques10.1109/TASLP.2015.2460459 ↗
24. Loudness, its definition, measurement and calculation10.1002/j.1538-7305.1933.tb00403.x ↗
25. Loudness and intensity relations10.1121/1.1915576 ↗
26. The measurement of loudness10.1121/1.1908048 ↗
27. ISO (2003). ISO 226:2003: Acoustics–Normal Equal-Loudness-Level Contours, International Organization for Standardization.
28. The loudness of diffuse sound fields
29. Directional loudness in an anechoic sound field, head-related transfer functions, and binaural summation10.1121/1.2184268 ↗
30. Blauert, J. (1997). Spatial Hearing: The Psychophysics of Human Sound Localization, MIT Press.10.7551/mitpress/6391.001.0001 ↗
31. Modeling binaural loudness10.1121/1.2431331 ↗
32. Ward, D., Enderby, S., Athwal, C., and Reiss, J.D. (December, January 30). Real-Time Excitation Based Binaural Loudness Meters. Proceedings of the 18th International Conference on DAFx, Trondheim, Norway.
33. Tuomi, O., and Zacharov, N. (June, January 30). A real-time binaural loudness meter. Proceedings of the 139th meeting of the Acoustical Society of America, Atlanta, GA, USA.
34. Schlittenlacher, J., Turner, R.E., and Moore, B.C. (2019). Fast computation of loudness using a deep neural network. arXiv.
35. Moustakas, N., Floros, A., Rovithis, E., and Vogklis, K. (2019). Augmented Audio-Only Games: A New Generation of Immersive Acoustic Environments through Advanced Mixing. Audio Engineering Society Convention 146, Audio Engineering Society.
36. Cheng, C.I., and Wakefield, G.H. (1999). Introduction to head-related transfer functions (HRTFs): Representations of HRTFs in time, frequency, and space. Audio Engineering Society Convention 107, Audio Engineering Society.
37. Boren, B., Geronazzo, M., Brinkmann, F., and Choueiri, E. (2015). Coloration Metrics for Headphone Equalization, Georgia Institute of Technology.
38. Genesis, S. (2021, June 21). Loudness Toolbox. For Matlab. Available online: https://www.mathworks.com/help//audio/ref/acousticloudness.html.
39. Documentation, S. (2021, June 21). Simulation and Model-Based Design. MathWorks. Available online: https://www.mathworks.com/products/simulink.html.
40. Goodwin, M. (1996, January 3–6). Nonuniform filterbank design for audio signal modeling. Proceedings of the Conference Record of the Thirtieth Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, USA.
41. A personal history of the Parks-McClellan algorithm10.1109/MSP.2005.1406492 ↗
42. Assembly, I.R. (2021, June 21). ITU-R BS. 1534-3: Method for the Subjective Assessment of Intermediate Quality Level of Audio Systems. Available online: https://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1534-3-201510-I!!PDF-E.pdf.