journal article·2024-10-22Open access

Data Augmentation Method Using Room Transfer Function for Monitoring of Domestic Activities

Minhan Kim, Seokjin Lee

Applied Sciences· Volume 14· Issue 21

Abstract

Monitoring domestic activities helps us to understand user behaviors in indoor environments, which has garnered interest as it aids in understanding human activities in context-aware computing. In the field of acoustics, this goal has been achieved through studies employing machine learning techniques, which are widely used for classification tasks involving sound recognition and other objectives. Machine learning typically achieves better performance with large amounts of high-quality training data. Given the high cost of data collection, development datasets often suffer from imbalanced data or lack high-quality samples, leading to performance degradations in machine learning models. The present study aims to address this data issue through data augmentation techniques. Specifically, since the proposed method targets indoor activities in domestic activity detection, room transfer functions were used for data augmentation. The results show that the proposed method achieves a 0.59% improvement in the F1-Score (micro) from that of the baseline system for the development dataset. Additionally, test data including microphones that were not used during training achieved an F1-Score improvement of 0.78% over that of the baseline system. This demonstrates the enhanced model generalization performance of the proposed method on samples having different room transfer functions to those of the trained dataset.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Context-Aware Activity Recognition Systems · Speech and Audio Processing · Anomaly Detection Techniques and Applications

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Electronic and Electrical Engineering, Kyungpook National University, Daegu 41566, Republic of Korea

School of Electronics Engineering, Kyungpook National University, Daegu 41566, Republic of Korea

Funding

Ministry of Trade, Industry & Energy (MOTIE, Korea) · Korea Research Institute for defense Technology planning and advancement (KRIT) · Korea government

References (30)

Show source-provided references
1. Schilit, B., Adams, N., and Want, R. (1994, January 8–9). Context-aware computing applications. Proceedings of the 1994 First Workshop on Mobile Computing Systems and Applications, Santa Cruz, CA, USA.10.1109/WMCSA.1994.16 ↗
2. Ciregan, D., Meier, U., and Schmidhuber, J. (2012, January 16–21). Multi-column deep neural networks for image classification. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.10.1109/CVPR.2012.6248110 ↗
3. Chan, W., Jaitly, N., Le, Q., and Vinyals, O. (2016, January 20–25). Listen, attend and spell: A neural network for large vocabulary conversational speech recognition. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7472621 ↗
4. Acoustic scene classification: Classifying environments from the sounds they produce10.1109/MSP.2014.2326181 ↗
5. Context-dependent sound event detection10.1186/1687-4722-2013-1 ↗
6. Dekkers, G., Vuegen, L., van Waterschoot, T., Vanrumste, B., and Karsmakers, P. (2018). DCASE 2018 Challenge—Task 5: Monitoring of Domestic aCtivities Based on Multi-Channel Acoustics. arXiv.
7. Valenti, M., Squartini, S., Diment, A., Parascandolo, G., and Virtanen, T. (2017, January 14–19). A convolutional neural network approach for acoustic scene classification. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.10.1109/IJCNN.2017.7966035 ↗
8. Graves, A. (2012). Sequence transduction with recurrent neural networks. arXiv.10.1007/978-3-642-24797-2 ↗
9. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
10. Hidden markov models10.1016/S0959-440X(96)80056-X ↗
11. An introduction to kernel and nearest-neighbor nonparametric regression10.1080/00031305.1992.10475879 ↗
12. Analyzing overfitting under class imbalance in neural networks for image segmentation10.1109/TMI.2020.3046692 ↗
13. Inoue, T., Vinayavekhin, P., Wang, S., Wood, D., Greco, N., and Tachibana, R. (2018, January 19–20). Domestic activities classification based on CNN using shuffling and mixing data augmentation. Proceedings of the DCASE 2018 Challenge, Surrey, UK.
14. Park, D.S., Chan, W., Zhang, Y., Chiu, C.C., Zoph, B., Cubuk, E.D., and Le, Q.V. (2019). Specaugment: A simple data augmentation method for automatic speech recognition. arXiv.10.21437/Interspeech.2019-2680 ↗
15. Wang, S., Rohdin, J., Plchot, O., Burget, L., Yu, K., and Černockỳ, J. (2020, January 4–8). Investigation of specaugment for deep speaker embedding learning. Proceedings of the ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9053481 ↗
16. Kim, B., Yang, S., Kim, J., and Chang, S. (2022). QTI Submission to DCASE 2021: Residual Normalization for Device-Imbalanced Acoustic Scene Classification with Efficient Design. arXiv.
17. Koyama, Y., Shigemi, K., Takahashi, M., Shimada, K., Takahashi, N., Tsunoo, E., Takahashi, S., and Mitsufuji, Y. (2022, January 22–27). Spatial data augmentation with simulated room impulse responses for sound event localization and detection. Proceedings of the ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9746754 ↗
18. Overview of geometrical room acoustic modeling techniques10.1121/1.4926438 ↗
19. Dekkers, G., Lauwereins, S., Thoen, B., Adhana, M.W., Brouckxon, H., Van den Bergh, B., Van Waterschoot, T., Vanrumste, B., Verhelst, M., and Karsmakers, P. (2017, January 16–17). The SINS database for detection of daily activities in a home environment using an acoustic sensor network. Proceedings of the Detection and Classification of Acoustic Scenes and Events 2017, Munich, Germany.
20. Turpault, N., Serizel, R., Shah, A.P., and Salamon, J. (2019, January 25–26). Sound event detection in domestic environments with weakly labeled data and soundscape synthesis. Proceedings of the Workshop on Detection and Classification of Acoustic Scenes and Events, New York, NY, USA.10.33682/006b-jx26 ↗
21. Werner, N. (2024, October 15). audiolabs/rir-generator: Version 0.1.0. Available online: https://zenodo.org/records/4133078.
22. Grondin, F., Lauzon, J.S., Michaud, S., Ravanelli, M., and Michaud, F. (2020). BIRD: Big Impulse Response Dataset. arXiv.
23. A central limit theorem and a strong mixing condition10.1073/pnas.42.1.43 ↗
24. Record values of the ratio of Rayleigh random variables
25. Image method for efficiently simulating small-room acoustics10.1121/1.382599 ↗
26. Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv.
27. Zhang, H., Cisse, M., Dauphin, Y.N., and Lopez-Paz, D. (2017). mixup: Beyond empirical risk minimization. arXiv.10.1007/978-1-4899-7687-1_79 ↗
28. Grandini, M., Bagli, E., and Visani, G. (2020). Metrics for multi-class classification: An overview. arXiv.
29. Computational bioacoustics with deep learning: A review and roadmap10.7717/peerj.13152 ↗
30. Cornell, S., Ebbers, J., Douwes, C., Martín-Morató, I., Harju, M., Mesaros, A., and Serizel, R. (2024). DCASE 2024 Task 4: Sound Event Detection with Heterogeneous Data and Missing Labels. arXiv.