journal article·2025-12-03Open access

Acoustic Vector Sensor–Based Speaker Diarization Using Sound Intensity Analysis for Two-Speaker Dialogues

Grzegorz Szwoch, Józef Kotus, Szymon Zaporowski

Applied Sciences· Volume 15· Issue 23

Abstract

Speaker diarization is a key component of automatic speech recognition (ASR) systems, particularly in interview scenarios where speech segments must be assigned to individual speakers. This study presents a diarization algorithm based on sound intensity analysis using an Acoustic Vector Sensor (AVS). The algorithm determines the azimuth of each speaker, defines directional beams, and detects speaker activity by analyzing intensity distributions within each beam, enabling identification of both single and overlapping speech segments. A dedicated dataset of interview recordings involving five speakers was created for evaluation. Performance was assessed using the Diarization Error Rate (DER) metric and compared with the State-of-the-Art Pyannote.audio system. The proposed AVS-based method achieved a lower DER value (0.112) than Pyannote (0.213) without overlapping speech, and a DER equal to 0.187 with overlapping speech included, demonstrating improved diarization accuracy and better handling of overlapping speech. The algorithm does not require training, operates independently of speaker-specific features, and can be adapted to various acoustic conditions. The results confirm that AVS-based diarization provides a robust and interpretable alternative to neural approaches, particularly suitable for structured two-speaker dialogues such as physician–patient or interviewer–interviewee scenarios.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Speech Recognition and Synthesis · Voice and Speech Disorders · Emotion and Mood Recognition

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Multimedia Systems, Faculty of Electronics, Telecommunication and Informatics, Gdańsk University of Technology, 80-222 Gdańsk, Poland

Funding

Polish National Centre for Research and Development

References (44)

Show source-provided references
1. An overview of automatic speaker diarization systems10.1109/TASL.2006.878256 ↗
2. Speaker diarization: A review of recent research10.1109/TASL.2011.2125954 ↗
3. Horiguchi, S., Fujita, Y., Watanabe, S., Xue, Y., and Nagamatsu, K. (2020, January 25–29). End-to-end speaker diarization for an unknown number of speakers with encoder-decoder based attractors. Proceedings of the Interspeech 2020, Shanghai, China.10.21437/Interspeech.2020-1022 ↗
4. A review of speaker diarization: Recent advances with deep learning10.1016/j.csl.2021.101317 ↗
5. O’Shaughnessy, D. (2025). Diarization: A review of objectives and methods. Appl. Sci., 15.10.3390/app15042002 ↗
6. Xue, Y., Horiguchi, S., Fujita, Y., Takashima, Y., Watanabe, S., Perera, L.P.G., and Nagamatsu, K. (September, January 30). Online Streaming End-to-End Neural Diarization Handling Overlapping Speech and Flexible Numbers of Speakers. Proceedings of the Interspeech 2021, Brno, Czech Republic.10.21437/Interspeech.2021-708 ↗
7. Du, Z., Zhang, S., Zheng, S., and Yan, Z. (2022, January 7–11). Speaker Overlap-aware Neural Diarization for Multi-party Meeting Analysis. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. EMNLP 2022, Abu Dhabi, United Arab Emirates.10.18653/v1/2022.emnlp-main.505 ↗
8. Broughton, S.J., and Samarakoon, L. (2023, January 20–24). Improving End-to-End Neural Diarization Using Conversational Summary Representations. Proceedings of the Interspeech 2023, Dublin, Ireland.10.21437/Interspeech.2023-2401 ↗
9. Diliberto, J., Pereira, C., Nikiforovskaja, A., and Sahidullah, M. (2021). Speaker Diarization with Overlapped Speech. [Master’s Thesis, Universite de Loraine: Institut des sciences du Digital].
10. Raj, D., Denisov, P., Chen, Z., Erdogan, H., Huang, Z., He, M., Watanabe, S., Du, J., Yoshioka, T., and Luo, Y. (2021, January 19–22). Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis. Proceedings of the IEEE Spoken Language Technology Workshop (SLT), Shenzhen, China.10.1109/SLT48900.2021.9383556 ↗
11. Accurate speaker counting, diarization and separation for advanced recognition of multichannel multispeaker conversations10.1016/j.csl.2025.101780 ↗
12. Optimization of modular multi-speaker distant conversational speech recognition10.1016/j.csl.2025.101816 ↗
13. Snyder, D., Garcia-Romero, D., Sell, G., Povey, D., and Khudanpur, S. (2018, January 15–20). X-Vectors: Robust DNN Embeddings for Speaker Recognition. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8461375 ↗
14. Speaker Diarization as a Fully Online Bandit Learning Problem in MiniVox
15. Raj, D., García-Perera, L.P., Huang, Z., Watanabe, S., Povey, D., Stolcke, A., and Khudanpur, S. (2021, January 19–22). DOVER-Lap: A Method for Combining Overlap-Aware Diarization Outputs. Proceedings of the 2021 IEEE Spoken Language Technology Workshop (SLT), Shenzhen, China.10.1109/SLT48900.2021.9383490 ↗
16. Zheng, N., Li, N., Yu, J., Weng, C., Su, D., Liu, X., and Meng, H.M. (2022, January 23–27). Multi-channel speaker diarization using spatial features for meetings. Proceedings of the ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9747343 ↗
17. Coria, J.M., Bredin, H., Ghannay, S., and Rosset, S. (2021, January 13–17). Overlap-Aware Low-Latency Online Speaker Diarization Based on End-to-End Local Segmentation. Proceedings of the 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Cartagena, Colombia.10.1109/ASRU51503.2021.9688044 ↗
18. Medennikov, I., Korenevsky, M., Prisyach, T., Khokhlov, Y., Korenevskaya, M., Sorokin, I., Timofeeva, T., Mitrofanov, A., Andrusenko, A., and Podluzhny, I. (2020, January 25–29). Target-speaker voice activity detection: A novel approach for multi-speaker diarization in a dinner party scenario. Proceedings of the Interspeech 2020, Shanghai, China.10.21437/Interspeech.2020-1602 ↗
19. Horiguchi, S., Takashima, Y., García, P., Watanabe, S., and Kawaguchi, Y. (2022, January 23–27). Multi-channel end-to-end neural diarization with distributed microphones. Proceedings of the ICASSP 2022—2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9746749 ↗
20. Wang, J., Liu, Y., Wang, B., Zhi, Y., Li, S., Xia, S., Zhang, J., Tong, F., Li, L., and Hong, Q. (2022, January 18–22). Spatial-aware speaker diarization for multi-channel multi-party meeting. Proceedings of the Interspeech 2022, Incheon, Republic of Korea.10.21437/Interspeech.2022-11412 ↗
21. Multi-channel conversational speaker separation via neural diarization10.1109/TASLP.2024.3393726 ↗
22. Speaker diarization and identification from single channel classroom audio recordings using virtual microphones10.1109/ACCESS.2022.3177584 ↗
23. Ma, F., Tu, Y., He, M., Wang, R., Niu, S., Sun, L., Ye, Z., Du, J., Pan, J., and Lee, C.-H. (2024, January 14–19). A spatial long-term iterative mask estimation approach for multi-channel speaker diarization and speech recognition. Proceedings of the ICASSP 2024—2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea.10.1109/ICASSP48485.2024.10446168 ↗
24. Wang, R., Niu, S., Yang, G., Du, J., Qian, S., Gao, T., and Pan, J. (2025, January 6–11). Incorporating spatial cues in modular speaker diarization for multi-channel multi-party meetings. Proceedings of the ICASSP 2025—2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Hyderabad, India.10.1109/ICASSP49660.2025.10890545 ↗
25. Xylogiannis, P., Vryzas, N., Vrysis, L., and Dimoulas, C. (2024). Multisensory fusion for unsupervised spatiotemporal speaker diarization. Sensors, 24.10.3390/s24134229 ↗
26. Cord-Landwehr, T., Gburrek, T., Deegen, M., and Haeb-Umbach, R. (2025, January 17–21). Spatio-spectral diarization of meetings by combining TDOA-based segmentation and speaker embedding-based clustering. Proceedings of the Interspeech 2025, Rotterdam, The Netherlands.10.21437/Interspeech.2025-1663 ↗
27. Ryant, N., Church, K., Cieri, C., Cristia, A., Du, J., Ganapathy, S., and Liberman, M. (2019, January 15–19). The Second DIHARD Diarization Challenge: Dataset, task, and baselines. Proceedings of the Interspeech 2019, Graz, Austria.10.21437/Interspeech.2019-1268 ↗
28. Ryant, N., Singh, P., Krishnamohan, V., Varma, R., Church, K., Cieri, C., Du, J., Ganapathy, S., and Liberman, M. (September, January 30). The Third DIHARD Diarization Challenge. Proceedings of the Interspeech 2021, Brno, Czech Republic.10.21437/Interspeech.2021-1208 ↗
29. Dawalatabad, D., Ravanelli, M., Grondin, G., Thienpondt, J., Desplanques, B., and Na, H. (September, January 30). ECAPA-TDNN embeddings for speaker diarization. Proceedings of the Interspeech 2021, Brno, Czech Republic.10.21437/Interspeech.2021-941 ↗
30. Yue, Y., Du, J., He, M.-K., Yeung, Y., and Wang, R. (2022, January 18–22). Online Speaker Diarization with Core Samples Selection. Proceedings of the Interspeech 2022, Incheon, Republic of Korea.10.21437/Interspeech.2022-10363 ↗
31. Acoustic vector sensor: Reviews and future perspectives10.1049/iet-spr.2016.0111 ↗
32. Shujau, M., Ritz, C.H., and Burnett, I.S. (2011, January 17–19). Separation of speech sources using an Acoustic Vector Sensor. Proceedings of the 2011 IEEE 13th International Workshop on Multimedia Signal Processing, Hangzhou, China.10.1109/MMSP.2011.6093797 ↗
33. Kotus, J., and Szwoch, G. (2025). Separation of simultaneous speakers with Acoustic Vector Sensor. Sensors, 25.10.3390/s25051509 ↗
34. Gburrek, T., Schmalenstroeer, J., and Haeb-Umbach, R. (November, January 29). Spatial diarization for meeting transcription with ad-hoc Acoustic Sensor Networks. Proceedings of the 2023 57th Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, USA.10.1109/IEEECONF59524.2023.10476794 ↗
35. A time-frequency bins selection pipeline for direction-of-arrival estimation using a single Acoustic Vector Sensor10.1109/JSEN.2022.3180157 ↗
36. Jin, Y., Zou, Y., and Ritz, C.H. (2017, January 5–9). Robust speaker DOA estimation based on the inter-sensor data ratio model and binary mask estimation in the bispectrum domain. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7952760 ↗
37. Zou, Y., Liu, Z., and Ritz, C.H. (2018). Enhancing target speech based on nonlinear soft masking using a single Acoustic Vector Sensor. Appl. Sci., 8.10.3390/app8091436 ↗
38. Wang, D., and Zou, Y. (2018, January 2–6). Joint noise and reverberation adaptive learning for robust speaker DOA estimation with an Acoustic Vector Sensor. Proceedings of the Interspeech 2018, Hyderabad, India.10.21437/Interspeech.2018-1135 ↗
39. Learning soft mask with DNN and DNN-SVM for multi-speaker DOA estimation using an acoustic vector sensor10.1016/j.jfranklin.2017.05.002 ↗
40. Bredin, H. (2023, January 20–24). Pyannote.audio 2.1 Speaker Diarization Pipeline: Principle, Benchmark, and Recipe. Proceedings of the Interspeech 2023, Dublin, Ireland.10.21437/Interspeech.2023-105 ↗
41. Bain, M., Huh, J., Han, T., and Zisserman, A. (2023, January 20–24). WhisperX: Time-accurate speech transcription of long-form audio. Proceedings of the Interspeech 2023, Dublin, Ireland.10.21437/Interspeech.2023-78 ↗
42. Fahy, F. (1995). Sound Intensity, E & F.N. Spon. [2nd ed.].
43. The Rich Transcription 2005 Spring Meeting Recognition Evaluation
44. Calibration of Acoustic Vector Sensor based on MEMS microphones for DOA estimation10.1016/j.apacoust.2018.07.025 ↗