journal article·2025-09-03Open access

Mathematical Analysis and Performance Evaluation of CBAM-DenseNet121 for Speech Emotion Recognition Using the CREMA-D Dataset

Zineddine Sarhani Kahhoul, Nadjiba Terki, Ilyes Benaissa, Khaled Aldwoah, E. I. Hassan, Osman Osman, Djamel Eddine Boukhari

Applied Sciences· Volume 15· Issue 17

Abstract

Emotion recognition from speech is essential for human–computer interaction (HCI) and affective computing, with applications in virtual assistants, healthcare, and education. Although deep learning has made significant advancements in Automatic Speech Emotion Recognition (ASER), the challenge still exists in the task given variation in speakers, subtle emotional expressions, and environmental noise. Practical deployment in this context depends on a strong, fast, scalable recognition system. This work introduces a new framework combining DenseNet121, especially fine-tuned for the crowd-sourced emotional multimodal actors dataset (CREMA-D), with the convolutional block attention module (CBAM). While DenseNet121’s effective feature propagation captures rich, hierarchical patterns in the speech data, CBAM improves the focus of the model on emotionally significant elements by applying both spatial and channel-wise attention. Furthermore, enhancing the input spectrograms and strengthening resistance against environmental noise is an advanced preprocessing pipeline including log-Mel spectrogram transformation and normalization. The proposed model demonstrates superior performance. To make sure the evaluation is strong even if there is a class imbalance, we point out important metrics like an Unweighted Average Recall (UAR) of 71.01% and an F1 score of 71.25%. The model also gets a test accuracy of 71.26% and a precision of 71.30%. These results establish the model as a promising solution for real-world speech emotion detection, highlighting its strong generalization capabilities, computational efficiency, and focus on emotion-specific features compared to recent work. The improvements demonstrate practical flexibility, enabling the integration of established image recognition techniques and allowing for substantial adaptability in various application contexts.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Emotion and Mood Recognition · Speech and Audio Processing · Music and Audio Processing

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

IL3CUB Laboratory, University of Mohamed Khider Biskra, Biskra 07000, Algeria

VSC Laboratory, University of Mohamed Khider Biskra, Biskra 07000, Algeria

Department of Mathematics, Faculty of Science, Islamic University of Madinah, Medinah 42351, Saudi Arabia

Department of Mathematics and Statistics, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh 13318, Saudi Arabia

Department of Mathematics, College of Science, Qassim University, Buraydah 51452, Saudi Arabia

Scientific and Technical Research Centre for Arid Areas, CRSTRA, Biskra 07000, Algeria

Funding

Qassim University

References (41)

Show source-provided references
1. Sawaf, H., El-Beltagy, S., Zaghouani, W., Magdy, W., Abdelali, A., Tomeh, N., Abu Farha, I., Habash, N., Khalifa, S., and Keleg, A. (2023). On Enhancing Fine-Tuning for Pre-trained Language Models. Proceedings of ArabicNLP 2023, Hybrid.
2. Habash, N., Bouamor, H., Eskander, R., Tomeh, N., Abu Farha, I., Abdelali, A., Touileb, S., Hamed, I., Onaizan, Y., and Alhafni, B. Functional Text Dimensions for Arabic Text Classification. Proceedings of the Second Arabic Natural Language Processing Conference.
3. A review on five recent and near-future developments in computational processing of emotion in the human voice10.1177/1754073919898526 ↗
4. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
5. Early prediction of learners at risk in self-paced education: A neural network approach10.1016/j.eswa.2022.118868 ↗
6. A survey of speech emotion recognition in natural environment10.1016/j.dsp.2020.102951 ↗
7. Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences10.1109/TASSP.1980.1163420 ↗
8. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2025, August 28). Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst., Available online: https://proceedings.neurips.cc/paper/2012/hash/c399862d3b9d6b76c8436e924a68c45b-Abstract.html.
9. Simonyan, K. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
10. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision And pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
11. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
12. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. Adv. Neural Inf. Process. Syst., 30, Available online: https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html.
13. Lin, Z., Feng, M., Santos, C.N.d., Yu, M., Xiang, B., Zhou, B., and Bengio, Y. (2017). A structured self-attentive sentence embedding. arXiv.
14. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., and Hovy, E. (2016, January 12–17). Hierarchical attention networks for document classification. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA.10.18653/v1/N16-1174 ↗
15. A deep learning model for depression detection based on MFCC and CNN generated spectrogram features10.1016/j.bspc.2023.105898 ↗
16. Survey on speech emotion recognition: Features, classification schemes, and databases10.1016/j.patcog.2010.09.020 ↗
17. IEMOCAP: Interactive emotional dyadic motion capture database10.1007/s10579-008-9076-6 ↗
18. Chen, Y.W., Hirschberg, J., and Tsao, Y. (2023). Noise robust speech emotion recognition with signal-to-noise ratio adapting speech enhancement. arXiv.
19. Gong, Y., Chung, Y.A., and Glass, J. (2021). Ast: Audio spectrogram transformer. arXiv.10.21437/Interspeech.2021-698 ↗
20. Logan, B. (2000, January 23–25). Mel frequency cepstral coefficients for music modeling. Proceedings of the Ismir, Plymouth, MA, USA.
21. Machine Learning, Deep Learning, and Data Preprocessing Techniques for Detecting, Predicting, and Monitoring Stress and Stress-Related Mental Disorders: Scoping Review10.2196/53714 ↗
22. Automated segmentation of dental restorations using deep learning: Exploring data augmentation techniques10.1007/s11282-024-00794-y ↗
23. Risks of feature leakage and sample size dependencies in deep feature extraction for breast mass classification10.1002/mp.14678 ↗
24. Schuller, B., and Batliner, A. (2013). Computational Paralinguistics: Emotion, Affect and Personality in Speech and Language Processing, John Wiley & Sons.10.1002/9781118706664 ↗
25. Emotional speech recognition: Resources, features, and methods10.1016/j.specom.2006.04.003 ↗
26. Kahhoul, Z., Terki, N., Benaissa, I., and Baarir, Z.E. (2025, August 28). SpectoResNet: Advancing Speech Emotion Recognition Through Deep Learning and Data Augmentation on the CREMA-D Dataset. Available online: https://sciforum.net/paper/view/20789.
27. Tarantino, L., Garner, P.N., and Lazaridis, A. (2019, January 15–19). Self-attention for speech emotion recognition. Proceedings of the 9th International on Speech Emotion Recognition and Understanding (IS-SERU), Graz, Austria.10.21437/Interspeech.2019-2822 ↗
28. Zhao, Z., Bao, Z., Zhao, Y., Zhang, Z., Cummins, N., Schuller, B., and Tao, J. (February, January 27). Speech emotion recognition using deep 3D convolutional neural networks with a bidirectional LSTM network. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
29. Ristea, N.C., and Ionescu, R.T. (2021). Self-paced ensemble learning for speech and audio classification. arXiv.10.21437/Interspeech.2021-155 ↗
30. Lin, Y.C., Chou, H.C., and yi Lee, H. (2025). Mitigating Subgroup Disparities in Multi-Label Speech Emotion Recognition: A Pseudo-Labeling and Unsupervised Learning Approach. arXiv, Available online: http://arxiv.org/abs/2505.14449.
31. TMNet: Transformer-fused multimodal framework for emotion recognition via EEG and speech10.1016/j.icte.2025.04.007 ↗
32. Huang, X., Acero, A., Hon, H.W., and Reddy, R. (2001). Spoken Language Processing: A Guide to Theory, Algorithm, and System Development, Prentice Hall PTR.
33. Accuracy enhancement method for speech emotion recognition from spectrogram using temporal frequency correlation and positional information learning through knowledge transfer
34. Tareq, I., Elbagoury, B., El-Regaily, S., and El-Horbaty, E.S. (2022). Analysis of ToN-IoT, UNW-NB15, and Edge-IIoT Datasets Using DL in Cybersecurity for IoT. Appl. Sci., 12.10.3390/app12199572 ↗
35. Zeiler, M.D., and Fergus, R. (2014, January 6–12). Visualizing and understanding convolutional networks. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland. Proceedings, Part I 13.
36. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (2016, January 27–30). Learning deep features for discriminative localization. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.319 ↗
37. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
38. Crema-d: Crowd-sourced emotional multimodal actors dataset10.1109/TAFFC.2014.2336244 ↗
39. librosa: Audio and music signal analysis in python10.25080/Majora-7b98e3ed-003 ↗
40. Ristea, N., Ionescu, R., and Khan, F. (2022). SepTr: Separable Transformer for Audio Spectrogram Processing. arXiv.10.21437/Interspeech.2022-249 ↗
41. Learning Rate Curriculum10.1007/s11263-024-02186-5 ↗