Magnitude Modeling of Personalized HRTF Based on Ear Images and Anthropometric Measurements
Source abstract
Abstract
In this paper, we propose a global personalized head-related transfer function (HRTF) method based on anthropometric measurements and ear images. The model consists of two sub-networks. The first is the VGG-Ear Model, which extracts features from the ear images. The second sub-network uses anthropometric measurements, ear features, and frequency information to predict the spherical harmonic (SH) coefficients. Finally, the personalized HRTF is obtained through inverse spherical harmonic transform (SHT) reconstruction. With only one training, the HRTF in all directions can be obtained, which greatly reduces the parameters and training cost of the model. To objectively evaluate the proposed method, we calculate the spectral distance (SD) between the predicted HRTF and the actual HRTF. The results show that the SD provided by this method is 5.31 dB, which is better than the average HRTF of 7.61 dB. In particular, the SD value is only increased by 0.09 dB compared to directly using the pinna measurements.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech and Audio Processing · Hearing Loss and Rehabilitation · Gait Recognition and Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Shanghai Institute for Advanced Communication and Data Science, Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Shanghai University, Shanghai 200444, China
Funding
Science and Technology Commission of Shanghai Municipality, the key technology research of spherical harmonic domain panoramic audio
References (40)
Show source-provided references
1. Wu, X., Talagala, D.S., Zhang, W., and Abhayapala, T.D. (2019). Individualized interaural feature learning and personalized binaural localization model. Appl. Sci., 9.10.3390/app9132682 ↗
2. Blauert, J., and Hearing, S. (1997). The psychophysics of human sound localization. Spatial Hearing, MIT Press.10.7551/mitpress/6391.001.0001 ↗
3. Xie, B. (2013). Head-Related Transfer Function and Virtual Auditory Display, J. Ross Publishing. [2nd ed.].
4. Howard, D.M., and Angus, J. (2009). Acoustics and Psychoacoustics, J. Ross Publishing. [4th ed.].
5. Binaural technique: Do we need individual recordings?
6. HRTF adjustments with audio quality assessments10.2478/aoa-2013-0007 ↗
7. HRTF measurements of a KEMAR10.1121/1.412407 ↗
8. Multiple exponential sweep method for fast measurement of head-related transfer functions
9. Li, S., and Peissig, J. (2020). Measurement of head-related transfer functions: A review. Appl. Sci., 10.10.3390/app10145014 ↗
10. Virtual audio system customization using visual matching of ear parameters. Object recognition supported by user interaction for service robots
11. Personalization of head-related transfer functions (HRTF) based on automatic photo-anthropometry and inference from a database10.1016/j.apacoust.2015.04.009 ↗
12. Kahana, Y., Nelson, P.A., Petyt, M., and Choi, S. (1999, January 10–12). Numerical modelling of the transfer functions of a dummy-head and of the external ear. Proceedings of the Audio Engineering Society Conference: 16th International Conference: Spatial Sound Reproduction, Rovaniemi, Finland.
13. Boundary element method calculation of individual head-related transfer function. I. Rigid model calculation10.1121/1.1412440 ↗
14. Fast calculation system specialized for head-related transfer function based on boundary element method10.1121/1.2191608 ↗
15. Lei, W., and Xiangyang, Z. (2016, January 13–16). New method for synthesizing personalized head-related transfer function. Proceedings of the 2016 IEEE International Workshop on Acoustic Signal Enhancement (IWAENC), Xi’an, China.10.1109/IWAENC.2016.7602913 ↗
16. Deep neural network based HRTF personalization using anthropometric measurements10.1109/TASLP.2016.2517565 ↗
17. A cybertwin based multimodal network for ecg patterns monitoring using deep learning10.1109/TII.2022.3159583 ↗
18. Chun, C.J., Moon, J.M., Lee, G.W., Kim, N.K., and Kim, H.K. (2017, January 18–21). Deep neural network based HRTF personalization using anthropometric measurements. Proceedings of the Audio Engineering Society Convention 143, New York, NY, USA.
19. Lee, G.W., and Kim, H.K. (2018). Personalized HRTF modeling based on deep neural network using anthropometric measurements and images of the ear. Appl. Sci., 8.10.3390/app8112180 ↗
20. Binaural reproduction based on bilateral Ambisonics and ear-aligned HRTFs10.1109/TASLP.2021.3055038 ↗
21. Wang, Y., Zhang, Y., Duan, Z., and Bocko, M. (2021, January 25–28). Global HRTF Personalization Using Anthropometric Measures. Proceedings of the Audio Engineering Society Convention 150, Online.
22. Role of spectral detail in sound-source localization10.1038/25526 ↗
23. The role of spatial detail in sound-source localization: Impact on HRTF modeling and personalization10.1121/1.4799575 ↗
24. Algazi, V.R., Duda, R.O., Thompson, D.M., and Avendano, C. (2001, January 21–24). The cipic hrtf database. Proceedings of the 2001 IEEE Workshop on the Applications of Signal Processing to Audio and Acoustics (Cat. No. 01TH8575), New Platz, NY, USA.
25. Ear recognition: More than a survey10.1016/j.neucom.2016.08.139 ↗
26. Convolutional encoder–decoder networks for pixel-wise ear detection and segmentation10.1049/iet-bmt.2017.0240 ↗
27. Evaluation and analysis of ear recognition models: Performance, complexity and resource requirements10.1007/s00521-018-3530-1 ↗
28. Wen, Y., Zhang, K., Li, Z., and Qiao, Y. (2016, January 8–16). A discriminative feature learning approach for deep face recognition. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46478-7_31 ↗
29. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
30. Emeršič, Ž., Playà, N.O., Štruc, V., and Peer, P. (2018, January 18–20). Towards accessories-aware ear recognition. Proceedings of the 2018 IEEE International Work Conference on Bioinspired Intelligence (IWOBI), San Carlos, Alajuela, Costa Rica.10.1109/IWOBI.2018.8464138 ↗
31. Alshazly, H., Linse, C., Barth, E., and Martinetz, T. (2019). Ensembles of deep learning models and transfer learning for ear recognition. Sensors, 19.10.3390/s19194139 ↗
32. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
33. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
34. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
35. Chollet, F. (2017, January 21–26). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
36. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv.
37. Nair, V., and Hinton, G.E. (2010, January 21–24). Rectified linear units improve restricted boltzmann machines. Proceedings of the ICML’10: 27th International Conference on Machine Learning, Haifa, Israel.
38. Zhi, B., Zotkin, D.N., and Duraiswami, R. (2022, January 22–27). Towards Fast And Convenient End-To-End HRTF Personalization. Proceedings of the ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9746315 ↗
39. Wang, M., and Deng, W. (2018). Deep Face Recognition: A Survey. arXiv.
40. A multimodal wearable system for continuous and real-time breathing pattern monitoring during daily activity10.1109/JBHI.2019.2963048 ↗