Using Vocal-Based Emotions as a Human Error Prevention System with Convolutional Neural Networks
Source abstract
Abstract
Human error is a mark assigned to an event that has negative effects or does not produce a desired result, with emotions playing an important role in how humans think and behave. If we detect feelings early, it may decrease human error. The human voice is one of the most powerful tools that can be used for emotion recognition. This study aims to reduce human error by building a system that detects positive or negative emotions of a user like (happy, sad, fear, and anger) through the analysis of the proposed vocal emotion component using Convolutional Neural Networks. By applying the proposed method to an emotional voice database (RAVDESS) using Librosa for voice processing and PyTorch, with the emotion classification of (happy/angry), the results show a better accuracy (98%) in comparison to the literature with regard to making a decision to deny or allow a user to access sensitive operations or send a warning to the system administrator prior to accessing system resources.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: AI in Service Interactions
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of IT, Faculty of Computing and IT, AIST Research Center, University of Tabuk, Tabuk 71491, Saudi Arabia
Funding
Artificial Intelligence and Sensing Technologies Research Center at the University of Tabuk
References (32)
Show source-provided references
1. Effects of seafarers’ emotion on human performance using bridge simulation10.1016/j.oceaneng.2018.10.021 ↗
2. Manjunath, K., Anu, V., Walia, G., and Bradshaw, G. (2018, January 15–18). Training Industry Practitioners to Investigate the Human Error Causes of Requirements Faults. Proceedings of the 2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), Memphis, TN, USA.10.1109/ISSREW.2018.00-31 ↗
3. Critical Success Factors for Controlling and Managing Hospital Errors10.1080/10686967.2004.11919099 ↗
4. Causal relationships of human errors in building maintenance: Findings from Sri Lanka10.1080/09613218.2023.2238850 ↗
5. Alkhaiwani, A.H., and Almalki, G.A. (2021, January 27–28). Saudi Human Awareness needs. A survey in how human causes errors and mistakes leads to leak confidential data with proposed solutions in Saudi Arabia. Proceedings of the 2021 National Computing Colleges Conference (NCCC), Taif, Saudi Arabia.10.1109/NCCC49330.2021.9428790 ↗
6. Automatically Analyzing Facial-Feature Movements to Identify Human Errors10.1109/MIS.2009.106 ↗
7. Human Error: A Concept Analysis
8. (2020, July 01). Colab Website. Available online: https://colab.research.google.com/.
9. An argument for basic emotions10.1080/02699939208411068 ↗
10. Hossain, N., and Naznin, M. (2018, January 4–7). Sensing Emotion from Voice Jitter. Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems, New York, NY, USA.10.1145/3274783.3275182 ↗
11. Manasa, C., Dheeraj, D., and Deepthi, V.S. (2019., January 19–20). Statistical Analysis of Voice Based Emotion Recognition using Similarity Measures. Proceedings of the 2019 1st International Conference on Advanced Technologies in Intelligent Control, Environment, Computing & Communication Engineering (ICATIECE), Bangalore, India,.10.1109/ICATIECE45860.2019.9063783 ↗
12. Kumbhakarn, M., and Sathe-Pathak, B. (2015, January 26–27). Analysis of Emotional State of a Person and Its Effect on Speech Features Using PRAAT Software. Proceedings of the 2015 International Conference on Computing Communication Control and Automation, Pune, India.10.1109/ICCUBEA.2015.152 ↗
13. Mande, A., Telang, S., Dani, S., and Shao, Z. (2019). Emotion Detection Using Audio Data Samples. Int. J. Adv. Res. Comput. Sci., 10.10.26483/ijarcs.v10i6.6489 ↗
14. (2020, July 01). Librosa Website. Available online: https://librosa.org/.
15. Analysis of Emotion Recognition System through Speech Signal Using KNN & GMM Classifier
16. Samantaray, A.K., Mahapatra, K., Kabi, B., and Routray, A. (2015, January 9–11). A novel approach of speech emotion recognition with prosody, quality and derived features using SVM classifier for a class of North-Eastern Languages. Proceedings of the 2015 IEEE 2nd International Conference on Recent Trends in Information Systems (ReTIS), Kolkata, India.10.1109/ReTIS.2015.7232907 ↗
17. Challenges in Automatic Emotion Recognition Process10.26483/ijarcs.v9i3.6035 ↗
18. Speech Emotion Recognition Using Convolutional Neural Network (CNN)
19. Han, K., Yu, D., and Tashev, I. (2014, January 14–18). Speech Emotion Recognition Using Deep Neural Network and Extreme Learning Machine. Proceedings of the Interspeech, Singapore.10.21437/Interspeech.2014-57 ↗
20. Evaluating deep learning architectures for Speech Emotion Recognition10.1016/j.neunet.2017.02.013 ↗
21. Deep learning and SVMbased emotion recognition from Chinese speech for smart affective services10.1002/spe.2487 ↗
22. Speech emotion recognition based on feature selection and extreme learning machine decision tree10.1016/j.neucom.2017.07.050 ↗
23. Emotion recognition from speech signals via a probabilistic echo-state network10.1016/j.patrec.2014.10.015 ↗
24. Niu, Y., Zou, D., Niu, Y., He, Z., and Tan, H. (2017). A breakthrough in Speech emotion recognition using Deep Retinal Convolution Neural Networks. arXiv.10.1145/3194452.3194460 ↗
25. A Novel Approach for Classification of Speech Emotions Based on Deep and Acoustic Features10.1109/ACCESS.2020.3043201 ↗
26. Almehmadi, A., and El-Khatib, K. (2013, January 26–28). Authorized! Access Denied, Unauthorized! Access Granted. Proceedings of the SIN ’13: The 6th International Conference on Security of Information and Networks, New York, NY, USA.10.1145/2523514.2523612 ↗
27. Bobulski, J. (2012). Access Control System Using Face Image, Systems Research Institute of the Polish Academy.
28. (2020, July 05). Ravdess, Website. Available online: https://smartlaboratory.org/ravdess/.
29. Livingstone, S.R., and Russo, F.A. (2018). The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English. PLoS ONE, 13.10.1371/journal.pone.0196391 ↗
30. (2020, July 01). Pytorch, Website. Available online: https://pytorch.org/.
31. Emotion Detection Using MFCC and Cepstrum Features10.1016/j.procs.2015.10.020 ↗
32. Investigation of Combined use of MFCC and LPC Features in Speech Recognition Systems