An Initial Machine Learning-Based Victim’s Scream Detection Analysis for Burning Sites
Source abstract
Abstract
Fire incidents are responsible for severe damage and thousands of deaths every year all over the world. Extreme temperatures, low visibility, toxic gases, and unknown locations of victims create difficulties and delays in rescue operations, escalating the risk of injury or death. It is time-critical to detect the victims trapped inside the burning sites for facilitating the rescue operations. This research work presents an audio-based automated system for victim detection in fire emergencies, investigating two machine learning (ML) methods: support vector machines (SVM) and long short-term memory (LSTM). The performance of these two ML techniques has been evaluated based on a variety of performance metrics. Our analyses show that both ML methods provide superior scream detection performance, with SVM slightly overperforming LSTM. Because of its lower complexity, SVM is a better candidate for real-time implementation in our autonomous embedded system vehicle (AESV).
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Fire Detection and Safety Systems · Anomaly Detection Techniques and Applications · Time Series Analysis and Forecasting
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Ingram School of Engineering, Texas State University, San Marcos, TX 78666, USA
Electrical & Computer Engineering, Northeastern University, Boston, MA 02115, USA
References (64)
Show source-provided references
1. (2021, August 09). National Fire Protection Association. Available online: https://www.nfpa.org/.
2. (2021, August 09). U.S Fire Administration, Available online: https://www.usfa.fema.gov/.
3. (2021, August 20). Firefighter Deaths. Available online: https://www.nfpa.org/News-and-Research/Data-research-and-tools/Emergency-Responders/Firefighter-fatalities-in-the-United-States/Firefighter-deaths.
4. Dubel, W., Gongora, H., Bechtold, K., and Diaz, D. (2003). An Autonomous Firefighting Robot, Dep. Electr. Comput. Eng. Fla. Int. Univ.
5. Firefighting robot stereo infrared vision and radar sensor fusion for imaging through smoke10.1007/s10694-014-0413-6 ↗
6. Bhattarai, M., Jensen-Curtis, A.R., and Martínez-Ramón, M. (2020, January 14–17). An embedded deep learning system for augmented reality in firefighting applications. Proceedings of the 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA), Miami, FL, USA.10.1109/ICMLA51294.2020.00193 ↗
7. Khana, N., Leea, D., Alia, A.K., and Parka, C. (2020, January 27–28). Artificial Intelligence and Blockchain-based Inspection Data Recording System for Portable Firefighting Equipment. Proceedings of the International Symposium on Automation and Robotics in Construction, Kitakyushu, Japan.10.22260/ISARC2020/0130 ↗
8. Liu, J. (2009, January 1–3). Case-based reasoning intelligent decision approach for firefighting tactics. Proceedings of the 2009 Second International Conference on Intelligent Networks and Intelligent Systems, Tianjian, China.10.1109/ICINIS.2009.118 ↗
9. New concept for indoor fire fighting robot10.1016/j.sbspro.2015.06.191 ↗
10. Design of fire detectors capable of self-adjusting by ignition
11. Thermal sensors for performance evaluation of protective clothing against heat and fire: A review10.1177/0040517514542864 ↗
12. Pinales, A., and Valles, D. (2018, January 1–3). Autonomous embedded system vehicle design on environmental, mapping and human detection data acquisition for firefighting situations. Proceedings of the 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference, Vancouver, BC, Canada.10.1109/IEMCON.2018.8615022 ↗
13. Jaradat, F.B., and Valles, D. (2020, January 6–8). A Victims Detection Approach for Burning Building Sites Using Convolutional Neural Networks. Proceedings of the 2020 10th Annual Computing and Communication Workshop and Conference, Las Vegas, NV, USA.10.1109/CCWC47524.2020.9031275 ↗
14. Baum, E., Harper, M., Alicea, R., and Ordonez, C. (February, January 31). Sound identification for fire-fighting mobile robots. Proceedings of the 2018 Second IEEE International Conference on Robotic Computing, Laguna Hills, CA, USA.10.1109/IRC.2018.00020 ↗
15. (2021, August 09). Getting Started with Jetson Nano Developer Kit. Available online: https://developer.nvidia.com/embedded/learn/get-started-jetson-nano-devkit.
16. Liu, P., Yu, H., Cang, S., and Vladareanu, L. (2016, January 7–8). Robot-assisted smart firefighting and interdisciplinary perspectives. Proceedings of the 2016 22nd International Conference on Automation and Computing (ICAC), Colchester, UK.10.1109/IConAC.2016.7604952 ↗
17. Hamins, A.P., Bryner, N.P., Jones, A.W., and Koepke, G.H. (2015). Research Roadmap for Smart Fire Fighting, National Institute of Standards and Technology.
18. (2021, August 09). Carbon Monoxide Poisoning and FIRE Fighters. Available online: https://www.iaff.org/carbon-monoxide/.
19. (2021, August 09). ROS. Available online: https://www.ros.org/.
20. Nandwana, M.K., and Hansen, J.H.L. (2014, January 14–18). Analysis and identification of human scream: Implications for speaker recognition. Proceedings of the 15th Annual Conference of the International Speech Communication Association, Singapore.10.21437/Interspeech.2014-249 ↗
21. Chan, C.-F., and Eric, W.M. (2010, January 23–27). An abnormal sound detection and classification system for surveillance applications. Proceedings of the 2010 18th European Signal Processing Conference, Aalborg, Denmark.
22. Pillai, A., and Kaushik, P. (2018). AC: An Audio Classifier to Classify Violent Extensive Audios. Speech and Language Processing for Human-Machine Communications, Springer.10.1007/978-981-10-6626-9_1 ↗
23. Acoustic signal based abnormal event detection in indoor environment using multiclass adaboost10.1109/TCE.2013.6626247 ↗
24. Kuklyte, J., Kelly, P., Ó Conaire, C., O’Connor, N.E., and Xu, L.-Q. (2009, January 9–11). Anti-social behavior detection in audio-visual surveillance systems. Proceedings of the The Workshop on Pattern Recognition and Artificial Intelligence for Human Behaviour Analysis, Reggio Emilia, Italy.
25. Laffitte, P., Sodoyer, D., Tatkeu, C., and Girin, L. (2016, January 20–25). Deep neural networks for automatic detection of screams and shouted speech in subway trains. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, China.10.1109/ICASSP.2016.7472921 ↗
26. Rabaoui, A., Davy, M., Rossignol, S., Lachiri, Z., and Ellouze, N. (2007, January 5–7). Improved one-class SVM classifier for sounds classification. Proceedings of the 2007 IEEE Conference on Advanced Video and Signal Based Surveillance, London, UK.10.1109/AVSS.2007.4425296 ↗
27. Noise-robust scream detection using band-limited spectral entropy10.1016/j.aeue.2017.04.002 ↗
28. Reliable detection of audio events in highly noisy environments10.1016/j.patrec.2015.06.026 ↗
29. Valenzise, G., Gerosa, L., Tagliasacchi, M., Antonacci, F., and Sarti, A. (2007, January 5–7). Scream and gunshot detection and localization for audio-surveillance systems. Proceedings of the 2007 IEEE Conference on Advanced Video and Signal Based Surveillance, London, UK.10.1109/AVSS.2007.4425280 ↗
30. Huang, W., Chiew, T.K., Li, H., Kok, T.S., and Biswas, J. (2010, January 15–17). Scream detection for home applications. Proceedings of the 2010 5th IEEE Conference on Industrial Electronics and Applications, Taichung, Taiwan.
31. Screaming sound detection based on UBM-GMM10.14257/ijgdc.2017.10.11.02 ↗
32. Two-stage supervised learning-based method to detect screams and cries in urban environments10.1109/TASLP.2015.2506264 ↗
33. Robust scream sound detection via sound event partitioning10.1007/s11042-015-2555-z ↗
34. Mak, M.-W., and Kung, S.-Y. (2012, January 25–30). Low-power SVM classifiers for sound event classification on mobile devices. Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Kyoto, Japan.10.1109/ICASSP.2012.6288296 ↗
35. Liao, W.-H., and Lin, Y.-K. (2009, January 11–14). Classification of non-speech human sounds: Feature selection and snoring sound analysis. Proceedings of the 2009 IEEE International Conference on Systems, Man and Cybernetics, San Antonio, TX, USA.10.1109/ICSMC.2009.5346556 ↗
36. Kao, C.-C., Wang, W., Sun, M., and Wang, C. (2018, January 2–6). R-CRNN: Region-based convolutional recurrent neural network for audio event detection. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, Hyderabad, India.10.21437/Interspeech.2018-2323 ↗
37. Lim, H., Park, J., and Han, Y. (2017, January 16). Rare sound event detection using 1D convolutional recurrent neural networks. Proceedings of the Detection and Classification of Acoustic Scenes and Events 2017 Workshop, Munich, Germany.
38. Colangelo, F., Battisti, F., Carli, M., Neri, A., and Calabró, F. (September, January 29). Enhancing audio surveillance with hierarchical recurrent neural networks. Proceedings of the 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Lecce, Italy.10.1109/AVSS.2017.8078496 ↗
39. Mun, S., Shon, S., Kim, W., Han, D.K., and Ko, H. (2017, January 5–9). Deep neural network based learning and transferring mid-level audio features for acoustic scene classification. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7952265 ↗
40. Zaheer, M.Z., Kim, J.Y., Kim, H.-G., and Na, S.Y. (2015, January 24–27). A preliminary study on deep-learning based screaming sound detection. Proceedings of the 2015 5th International Conference on IT Convergence and Security (ICITCS), Kuala Lumpur, Malaysia.10.1109/ICITCS.2015.7292925 ↗
41. Human screams occupy a privileged niche in the communication soundscape10.1016/j.cub.2015.06.043 ↗
42. (2021, August 09). MIVIA Dataset. Available online: https://mivia.unisa.it/datasets/audio-analysis/mivia-audio-events/.
43. Barker, J., Watanabe, S., Vincent, E., and Trmal, J. (2018, January 2–6). The fifth‘CHiME’speech separation and recognition challenge: Dataset, task and baselines. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, Hyderabad, India.10.21437/Interspeech.2018-1768 ↗
44. (2021, August 09). Freesound. Available online: https://freesound.org/.
45. Zarazaga, P.P., Das, S., Bäckström, T., Vegesna, V.V.R., and Vuppala, A.K. (2019, January 15–19). Sound Privacy: A Conversational Speech Corpus for Quantifying the Experience of Privacy. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2019, Graz, Austria.10.21437/Interspeech.2019-1172 ↗
46. Visualizing data using t-SNE
47. (2021, August 09). How to Apply Machine Learning and Deep Learning Methods to Audio Analysis. Available online: https://towardsdatascience.com/how-to-apply-machine-learning-and-deep-learning-methods-to-audio-analysis-615e286fcbbc.
48. McFee, B., Raffel, C., Liang, D., Ellis, D.P.W., McVicar, M., Battenberg, E., and Nieto, O. (2015, January 6–12). librosa: Audio and music signal analysis in python. Proceedings of the 14th Python in Science Conference (SciPy 2015), Austin, Texas.10.25080/Majora-7b98e3ed-003 ↗
49. A Review on Scream Classification for Situation Understanding
50. Muda, L., Begam, M., and Elamvazuthi, I. (2010). Voice recognition algorithms using mel frequency cepstral coefficient (MFCC) and dynamic time warping (DTW) techniques. arXiv.
51. (2021, August 09). Zero Crossing Rate. Available online: https://www.sciencedirect.com/topics/engineering/zero-crossing-rate.
52. (2021, August 09). Spectral Centroid. Available online: https://www.sciencedirect.com/topics/engineering/spectral-centroid.
53. Assessing the performances of different neural network architectures for the detection of screams and shouts in public transportation10.1016/j.eswa.2018.08.052 ↗
54. Support-vector networks10.1007/BF00994018 ↗
55. Support vector machines for multi-class pattern recognition
56. Scikit-learn: Machine learning in Python
57. Raschka, S. (2015). Python Machine Learning, Packt publishing Ltd.
58. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
59. Chollet, F. (2017). Deep learning with Python, Manning Publication Co.
60. Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., and Isard, M. (2016, January 2–4). Tensorflow: A system for large-scale machine learning. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), Savannah, GA, USA.
61. (2021, August 09). Zero Padding in the Time Domain. Available online: https://www.dsprelated.com/freebooks/sasp/Zero_Padding_Time_Domain.html.
62. (2021, August 09). Permutation Importance. Available online: https://www.kaggle.com/dansbecker/permutation-importance.
63. Dropout: A simple way to prevent neural networks from overfitting
64. Time difference of arrival estimation of speech source in a noisy and reverberant environment10.1016/j.sigpro.2004.09.014 ↗