journal article·2024-05-24Open access

Robust DOA Estimation Using Multi-Scale Fusion Network with Attention Mask

Yuting Yan, Qinghua Huang

Applied Sciences· Volume 14· Issue 11

Abstract

To overcome the limitations of traditional methods in reverberant and noisy environments, a robust multi-scale fusion neural network with attention mask is designed to improve direction-of-arrival (DOA) estimation accuracy for acoustic sources. It combines the benefits of deep learning and complex-valued operations to effectively deal with the interference of reverberation and noise in speech signals. The unique properties of complex-valued signals are exploited to fully capture inherent features and rich information is preserved in the complex field. An attention mask module is designed to generate distinct masks for selectively focusing and masking based on the input. After that, the multi-scale fusion block efficiently captures multi-scale spatial features by stacking complex-valued convolutional layers with small size kernels, and reduces the module complexity through special branching operations. Experimental results demonstrate that the model achieves significant improvements over other methods for speaker localization in reverberant and noisy environments. It provides a new solution for DOA estimation for acoustic sources in different scenarios, which has significant theoretical and practical implications.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Speech and Audio Processing · Underwater Acoustics Research · Indoor and Outdoor Localization Technologies

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China

References (60)

Show source-provided references
1. A Multiscale Autoencoder (MSAE) Framework for End-to-End Neural Network Speech Enhancement10.1109/TASLP.2024.3389632 ↗
2. Leveraging Non-Causal Knowledge Via Cross-Network Knowledge Distillation for Real-Time Speech Enhancement10.1109/LSP.2024.3388956 ↗
3. Lee, Y., Choi, S., Kim, B.-Y., Wang, Z.-Q., and Watanabe, S. (2024, January 14–19). Boosting Unknown-Number Speaker Separation with Transformer Decoder-Based Attractor. Proceedings of the ICASSP 2024—2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea.10.1109/ICASSP48485.2024.10446032 ↗
4. Reverberant Source Separation Using NTF With Delayed Subsources and Spatial Priors10.1109/TASLP.2024.3374065 ↗
5. Unified Cross-Modal Attention: Robust Audio-Visual Speech Recognition and Beyond10.1109/TASLP.2024.3375641 ↗
6. Computation and Parameter Efficient Multi-Modal Fusion Transformer for Cued Speech Recognition10.1109/TASLP.2024.3363446 ↗
7. Multiple emitter location and signal parameter estimation10.1109/TAP.1986.1143830 ↗
8. Palanisamy, P., and Kishore, C. (2011, January 3–5). 2-D DOA estimation of quasi-stationary signals based on Khatri-Rao subspace approach. Proceedings of the 2011 International Conference on Recent Trends in Information Technology (ICRTIT), Chennai, India.10.1109/ICRTIT.2011.5972295 ↗
9. Wang, X., and Amin, M. (2017, January 10–13). Design of optimum sparse array for robust MVDR beamforming against DOA mismatch. Proceedings of the 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP), Curacao, The Netherlands.10.1109/CAMSAP.2017.8313065 ↗
10. Impaired Sensor Diagnosis, Beamforming, and DOA Estimation with Difference Co-Array Processing10.1109/JSEN.2015.2399510 ↗
11. Neural-Network-Based Direction-of-Arrival Estimation for Reverberant Speech—The Importance of Energetic, Temporal, and Spatial Information10.1109/TASLP.2024.3357037 ↗
12. DOA Estimation Method Based on Unsupervised Learning Network With Threshold Capon Spectrum Weighted Penalty10.1109/LSP.2023.3349078 ↗
13. End-to-End Regression Neural Network for Coherent DOA Estimation with Dual-Branch Outputs10.1109/JSEN.2023.3342796 ↗
14. Two-Stage Deep Convolutional Neural Networks for DOA Estimation in Impulsive Noise10.1109/TAP.2023.3332502 ↗
15. Robust DoA Estimation in a Uniform Circular Array Antenna With Errors and Unknown Parameters Using Deep Learning10.1109/TGCN.2023.3294448 ↗
16. Adaptive Direction-of-Arrival Estimation Using Deep Neural Network in Marine Acoustic Environment10.1109/JSEN.2023.3274309 ↗
17. (2024, May 22). The Eigenmike Microphone Array. [Online]. Available online: http://www.mhacoustics.com/.
18. Fourier Acoustics: Sound Radiation and Nearfield Acoustical Holography10.1121/1.1289662 ↗
19. Zhao, S., Nguyen, T.H., and Ma, B. (2021, January 6–11). Monaural Speech Enhancement with Complex Convolutional Block Attention Module and Joint Time Frequency Losses. Proceedings of the ICASSP 2021—2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9414569 ↗
20. Shahhoud, F., Deeb, A.A., and Terekhov, V.I. (March, January 29). PESQ enhancement for decoded speech audio signals using complex convolutional recurrent neural network. Proceedings of the 2024 6th International Youth Conference on Radio Electronics, Electrical and Power Engineering (REEPE), Moscow, Russia.10.1109/REEPE60449.2024.10479874 ↗
21. Guo, P., Yu, M., Shen, L., Lin, Z., An, K., and Wang, J. (IEEE Wirel. Commun. Lett., 2024). Single-Channel Blind Source Separation in Wireless Communications: A Complex-Domain Deep Learning Approach, IEEE Wirel. Commun. Lett., early access.10.1109/LWC.2024.3384813 ↗
22. Saadati, M., Toroghi, R.M., and Zareian, H. (2024, January 21–22). Multi-Level Speaker-Independent Emotion Recognition Using Complex-MFCC and Swin Transformer. Proceedings of the 2024 20th CSI International Symposium on Artificial Intelligence and Signal Processing (AISP), Babol, Iran.10.1109/AISP61396.2024.10475274 ↗
23. Deb, S., and Dandapat, S. (2017, January 15–17). Emotion Classification using Dual-Tree Complex Wavelet Transform. Proceedings of the 2017 14th IEEE India Council International Conference (INDICON), Roorkee, India.10.1109/INDICON.2017.8487854 ↗
24. Kong, Y., Wu, J., Wang, Q., Gao, P., Zhuang, W., Wang, Y., and Xie, L. (2021, January 19–22). Multi-Channel Automatic Speech Recognition Using Deep Complex Unet. Proceedings of the 2021 IEEE Spoken Language Technology Workshop (SLT), Shenzhen, China.10.1109/SLT48900.2021.9383492 ↗
25. Xiang, Y., Tian, J., Hu, X., Xu, X., and Yin, Z. (2024, January 14–19). A Deep Representation Learning-Based Speech Enhancement Method Using Complex Convolution Recurrent Variational Autoencoder. Proceedings of the ICASSP 2024—2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea.10.1109/ICASSP48485.2024.10448125 ↗
26. Blind Localization of Early Room Reflections Using Phase Aligned Spatial Correlation10.1109/TSP.2021.3057495 ↗
27. Acoustic analysis by spherical microphone array processing of room impulse responses10.1121/1.4726012 ↗
28. Spherical array processing for acoustic analysis using room impulse responses and time-domain smoothing10.1121/1.4804314 ↗
29. Sun, H., Teutsch, H., Mabande, E., and Kellermann, W. (2011, January 22–27). Robust localization of multiple sources in reverberant environments using EB-ESPRIT with spherical microphone arrays. Proceedings of the 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Prague, Czech Republic.10.1109/ICASSP.2011.5946342 ↗
30. MUSIC, G-MUSIC, and Maximum-Likelihood Performance Breakdown10.1109/TSP.2008.921729 ↗
31. Direction of arrival estimation of multiple acoustic sources using a maximum likelihood method in the spherical harmonic domain10.1016/j.apacoust.2018.02.005 ↗
32. Localization of Multiple Speakers under High Reverberation using a Spherical Microphone Array and the Direct-Path Dominance Test10.1109/TASLP.2014.2337846 ↗
33. Multiple Source Direction of Arrival Estimations Using Relative Sound Pressure Based MUSIC10.1109/TASLP.2020.3039569 ↗
34. Pavlidi, D., Delikaris-Manias, S., Pulkki, V., and Mouchtaris, A. (September, January 31). 3D localization of multiple sound sources with intensity vector estimates in single source zones. Proceedings of the 2015 23rd European Signal Processing Conference (EUSIPCO), Nice, France.10.1109/EUSIPCO.2015.7362645 ↗
35. Augmented Intensity Vectors for Direction of Arrival Estimation in the Spherical Harmonic Domain10.1109/TASLP.2017.2736067 ↗
36. A Deep Learning Framework for Robust DOA Estimation Using Spherical Harmonic Decomposition10.1109/TASLP.2020.2984852 ↗
37. DOA estimation using two independent convolutional neural networks with residual blocks10.1016/j.dsp.2022.103765 ↗
38. Dwivedi, P., Routray, G., and Hegde, R.M. (2024). Octant Spherical Harmonics Features for Source Localization using Artificial Intelligence based on Unified Learning Framework. IEEE Trans. Artif. Intell., early access.10.1109/TAI.2024.3352530 ↗
39. A training algorithm with selectable search direction for complex-valued feedforward neural networks10.1016/j.neunet.2021.01.014 ↗
40. Costanzo, S., and Flores, A. (2023, January 15–17). CVNN-Based Microwave Imaging Approach. Proceedings of the 2023 IEEE Conference on Antenna Measurements and Applications (CAMA), Genoa, Italy.10.1109/CAMA57522.2023.10352766 ↗
41. Costanzo, S., and Flores, A. (2024, January 17–22). CVNN Approach for Microwave Imaging Applications in Brain Cancer: Preliminary Results. Proceedings of the 2024 18th European Conference on Antennas and Propagation (EuCAP), Glasgow, UK.10.23919/EuCAP60739.2024.10501161 ↗
42. A Zynq-Based Platform With Conditional-Reconfigurable Complex-Valued Neural Network for Specific Emitter Identification10.1109/TIM.2024.3391818 ↗
43. Hirose, A. (2009, January 14–19). Complex-valued neural networks: The merits and their origins. Proceedings of the 2009 International Joint Conference on Neural Networks, Atlanta, GA, USA.10.1109/IJCNN.2009.5178754 ↗
44. Solving the XOR problem and the detection of symmetry using a single complex-valued neuron10.1016/S0893-6080(03)00168-0 ↗
45. ESPRIT-estimation of signal parameters via rotational invariance techniques10.1109/29.32276 ↗
46. Zhang, X., Zhou, X., Lin, M., and Sun, J. (2018, January 18–23). ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00716 ↗
47. Research on Recognition of Fly Species Based on Improved RetinaNet and CBAM10.1109/ACCESS.2020.2997466 ↗
48. ImageNet classification with deep convolutional neural networks
49. Tachibana, K., and Otsuka, K. (2018, January 11–14). Wind Prediction Performance of Complex Neural Network with ReLU Activation Function. Proceedings of the 2018 57th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), Nara, Japan.10.23919/SICE.2018.8492660 ↗
50. Rigid sphere room impulse response simulation: Algorithm and applications10.1121/1.4740497 ↗
51. Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. (2015, January 19–24). Librispeech: An ASR corpus based on public domain audio books. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, QLD, Australia.10.1109/ICASSP.2015.7178964 ↗
52. Voice Activity Detection Using an Adaptive Context Attention Model10.1109/LSP.2018.2811740 ↗
53. Löllmann, H.W., Evers, C., Schmidt, A., Mellmann, H., Barfuss, H., Naylor, P.A., and Kellermann, W. (2018, January 8–11). The LOCATA Challenge Data Corpus for Acoustic Source Localization and Tracking. Proceedings of the 2018 IEEE 10th Sensor Array and Multichannel Signal Processing Workshop (SAM), Sheffield, UK.10.1109/SAM.2018.8448644 ↗
54. Multi-Stream Acoustic Modelling Using Raw Real and Imaginary Parts of the Fourier Transform10.1109/TASLP.2023.3237167 ↗
55. Hu, S., Zeng, C., Liu, M., Tao, H., Zhao, S., and Liu, Y. (2023, January 23–25). Robust DOA Estimation Using Deep Complex-Valued Convolutional Networks with Sparse Prior. Proceedings of the 2023 6th International Conference on Information Communication and Signal Processing (ICICSP), Xi’an, China.10.1109/ICICSP59554.2023.10390873 ↗
56. Complex-Valued Neural Network with Multistep Training for Single-Snapshot DOA Estimation10.1109/LGRS.2024.3404481 ↗
57. Zheng, R., Sun, S., Liu, H., Chen, H., Soltanalian, M., and Li, J. (2024). Antenna Failure Resilience: Deep Learning-Enabled Robust DOA Estimation with Single Snapshot Sparse Arrays. Invited paper for IEEE Asilomar conference 2024. arXiv.
58. SongGong, K., Zhang, P., Zhang, X., Sun, M., and Wang, W. (2024, January 14–19). Multi-Speaker Localization in the Circular Harmonic Domain on Small Aperture Microphone Arrays Using Deep Convolutional Networks. Proceedings of the ICASSP 2024—2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea.10.1109/ICASSP48485.2024.10448198 ↗
59. Generating sensor signals in isotropic noise fields10.1121/1.2799929 ↗
60. Sound localization in web-based 3D environments10.1038/s41598-022-15931-y ↗