A Deep Learning Method for DOA Estimation with Covariance Matrices in Reverberant Environments
Source abstract
Abstract
Acoustic source localization in the spherical harmonic domain with reverberation has hitherto not been extensively investigated. Moreover, deep learning frameworks have been utilized to estimate the direction-of-arrival (DOA) with spherical microphone arrays under environments with reverberation and noise for low computational complexity and high accuracy. This paper proposes three different covariance matrices as the input features and two different learning strategies for the DOA task. There is a progressive relationship among the three covariance matrices. The second matrix can be obtained by processing the first matrix and it effectively filters out the effects of the microphone array and mode strength to some extent. The third matrix can be obtained by processing the second matrix and it further efficiently removes information irrelevant to location information. In terms of the strategies, the first strategy is a regular learning strategy, while the second strategy is to split the task into three parts to be performed in parallel. Experiments were conducted both on the simulated and real datasets to show that the proposed method has higher accuracy than the conventional methods and lower computational complexity. Thus, the proposed method can effectively resist reverberation and noise.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech and Audio Processing · Underwater Acoustics Research · Music and Audio Processing
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Shanghai University, Shanghai 200444, China
Funding
National Natural Science Foundation of China
References (47)
Show source-provided references
1. Digital signal processing for sonar10.1109/PROC.1981.12186 ↗
2. Localization systems for wireless sensor networks10.1109/MWC.2007.4407221 ↗
3. Two decades of array signal processing research: The parametric approach10.1109/79.526899 ↗
4. A Consolidated Perspective on Multimicrophone Speech Enhancement and Source Separation10.1109/TASLP.2016.2647702 ↗
5. Wang, H., and Chu, P. (1997, January 21–24). Voice source localization for automatic camera pointing system in videoconferencing. Proceedings of the 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing, Munich, Germany.10.1109/ASPAA.1997.625639 ↗
6. Yu, Y., Wang, W., Luo, J., and Feng, P. (2015, January 21–24). Localization based stereo speech separation using deep networks. Proceedings of the 2015 IEEE International Conference on Digital Signal Processing (DSP), Singapore.10.1109/ICDSP.2015.7251849 ↗
7. Multiple emitter location and signal parameter estimation10.1109/TAP.1986.1143830 ↗
8. ESPRIT-estimation of signal parameters via rotational invariance techniques10.1109/29.32276 ↗
9. High-resolution frequency-wavenumber spectrum analysis10.1109/PROC.1969.7278 ↗
10. Yang, J.-M., Lee, C.-H., Kim, S., and Kang, H.-G. (2009, January 24–28). A robust time difference of arrival estimator in reverberant environments. Proceedings of the 2009 17th European Signal Processing Conference, Glasgow, Scotland.
11. The generalized correlation method for estimation of time delay10.1109/TASSP.1976.1162830 ↗
12. Spatial Filtering Based on Differential Spectrum for Improving ML DOA Estimation Performance10.1109/LSP.2016.2605006 ↗
13. Huang, Z., Xu, J., and Pan, J. (2018, January 13–16). A Regression Approach to Speech Source Localization Exploiting Deep Neural Network. Proceedings of the 2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM), Xi’an, China.10.1109/BigMM.2018.8499477 ↗
14. Zhu, W., and Zhang, M. (2019, January 16–19). A Deep Learning Architecture for Broadband DOA Estimation. Proceedings of the 2019 IEEE 19th International Conference on Communication Technology (ICCT), Xi’an, China.10.1109/ICCT46805.2019.8947053 ↗
15. Two-Dimensional DOA Estimation via Deep Ensemble Learning10.1109/ACCESS.2020.3005221 ↗
16. Robust Sound Source Tracking Using SRP-PHAT and 3D Convolutional Neural Networks10.1109/TASLP.2020.3040031 ↗
17. Shimada, K., Koyama, Y., Takahashi, N., Takahashi, S., and Mitsufuji, Y. (2021, January 6–11). Accdoa: Activity-Coupled Cartesian Direction of Arrival Representation for Sound Event Localization and Detection. Proceedings of the ICASSP 2021—2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9413609 ↗
18. Xiao, X., Zhao, S., Zhong, X., Jones, D.L., Chng, E.S., and Li, H. (2015, January 19–24). A learning based approach to direction of arrival estimation in noisy and reverberantenvironments. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, QLD, Australia.10.1109/ICASSP.2015.7178484 ↗
19. Stöter, F., Chakrabarty, S., Edler, B., and Habets, E.A.P. (2018, January 15–20). Classification vs. Regression in Supervised Learning for Single Channel Speaker Count Estimation . Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462159 ↗
20. Perotin, L., Défossez, A., Vincent, E., Serizel, R., and Guérin, A. (2019, January 20–23). Regression Versus Classification for Neural Network Based Audio Source Localization. Proceedings of the 2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA.10.1109/WASPAA.2019.8937277 ↗
21. A Deep Learning Framework for Robust DOA Estimation Using Spherical Harmonic Decomposition10.1109/TASLP.2020.2984852 ↗
22. Chakrabarty, S., and Habets, E.A.P. (2017, January 15–18). Broadband DOA estimation using convolutional neural networks trained with noise signals. Proceedings of the 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA.10.1109/WASPAA.2017.8170010 ↗
23. Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks10.1109/JSTSP.2018.2885636 ↗
24. Direction-of-Arrival Estimation Based on Deep Neural Networks with Robustness to Array Imperfections10.1109/TAP.2018.2874430 ↗
25. Perotin, L., Serizel, R., Vincent, E., and Guérin, A. (2018, January 17–20). CRNN-based Joint Azimuth and Elevation Localization with the Ambisonics Intensity Vector. Proceedings of the 2018 16th International Workshop on Acoustic Signal Enhancement (IWAENC), Tokyo, Japan.10.1109/IWAENC.2018.8521403 ↗
26. Analysis and design of spherical microphone arrays10.1109/TSA.2004.839244 ↗
27. On multiplicative transfer function approximationin the short-time fourier transform domain10.1109/LSP.2006.888292 ↗
28. Localization of multiple speakers under high reverberation using a spherical microphone array and the direct-pathdominance test10.1109/TASLP.2014.2337846 ↗
29. Plane-wave decomposition of the pressure on a sphere by spherical convolution10.1121/1.1792643 ↗
30. Paulraj, A., Roy, R., and Kailath, T. (1985, January 6–8). Estimation of signal parameters via rotational invariance techniques-esprit. Proceedings of the Nineteeth Asilomar Conference on Circuits, Systems and Computers, Pacific Grove, CA, USA.10.1109/ACSSC.1985.671426 ↗
31. One-Dimensional MUSIC-Type Algorithm for Spherical Microphone Arrays10.1109/ACCESS.2020.2972069 ↗
32. Imagenet classification with deep convolutional neural networks
33. Wang, C., Xu, Q., Li, X., Zheng, G., Liu, B., and Cheng, Y. (2019, January 17–20). An Objective Technique for Typhoon Monitoring with Satellite Infrared Imagery. Proceedings of the 2019 Photonics & Electromagnetics Research Symposium-Fall (PIERS-Fall), Xiamen, China.10.1109/PIERS-Fall48861.2019.9021497 ↗
34. Singh, P.P., Kaushik, R., Singh, H., Kumar, N., and Rana, P.S. (2019, January 9–13). Convolutional Neural Networks Based Plant Leaf Diseases Detection Scheme. Proceedings of the 2019 IEEE Globecom Workshops (GC Wkshps), Waikoloa, HI, USA.10.1109/GCWkshps45667.2019.9024434 ↗
35. Kingma, D.P., and Ba, J. (2015, January 7–9). Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference for Learning Representations, ICLR 2015, San Diego, CA, USA.
36. Rigidsphere room impulse response simulation: Algorithm and applications10.1121/1.4740497 ↗
37. (2022, April 03). The Eigenmike Microphone Array. Available online: http://www.mhacoustics.com/.
38. (2004). Speech activity detection and enhancement of a moving speaker based on the wideband generalized likelihood ratio and microphone arrays. J. Acoust. Soc. Am., 116, 2406–2415.10.1121/1.1781622 ↗
39. Varzandeh, R., Adiloğlu, K., Doclo, S., and Hohmann, V. (2020, January 4–8). Exploiting Periodicity Features for Joint Detection and DOA Estimation of Speech Sources Using Convolutional Neural Networks. Proceedings of the ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9054754 ↗
40. Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. (2015, January 19–24). Librispeech: An ASR corpus based on public domain audio books. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, QLD, Australia.10.1109/ICASSP.2015.7178964 ↗
41. The LOCATA Challenge: Acoustic Source Localization and Tracking10.1109/TASLP.2020.2990485 ↗
42. Kumar, L., Bi, G., and Hegde, R.M. (2016, January 20–25). The spherical harmonics root-music. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7472237 ↗
43. Rafaely, B., and Kolossa, D. (2017, January 5–9). Speaker localization in reverberant rooms based on direct path dominance test statistics. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7953332 ↗
44. Tang, Z., Kanu, J.D., Hogan, K., and Manocha, D. (2019). Regression and Classification for Direction-of-Arrival Estimation with Convolutional Recurrent Neural Networks. arXiv.10.21437/Interspeech.2019-1111 ↗
45. Eigenvalues and Singular Value Decompositions of Reduced Biquaternion Matrices10.1109/TCSI.2008.920068 ↗
46. Direction-of-arrival estimation for nonuniform sensor arrays: From manifold separation to fourier domain music methods10.1109/TSP.2008.2008560 ↗
47. Cheng, Y., Yu, F.X., Feris, R.S., Kumar, S., Choudhary, A., and Chang, S.-F. (2015, January 7–13). An exploration of parameter redundancy in deep networks with circulant projections. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.327 ↗