A Depression Recognition Method Based on the Alteration of Video Temporal Angle Features
Source abstract
Abstract
In recent years, significant progress has been made in the auxiliary diagnosis system for depression. However, most of the research has focused on combining features from multiple modes to enhance classification accuracy. This approach results in increased space-time overhead and feature synchronization problems. To address this issue, this paper presents a single-modal framework for detecting depression based on changes in facial expressions. Firstly, we propose a robust method for extracting angle features from facial landmarks. Theoretical evidence is provided to demonstrate the translation and rotation invariance of these features. Additionally, we introduce a flip correction method to mitigate angle deviations caused by head flips. The proposed method not only preserves the spatial topological relationship of facial landmarks, but also maintains the temporal correlation between frames preceding and following the facial landmarks. Finally, the GhostNet network is employed for depression detection, and the effectiveness of various modal data is compared. In the depression binary classification task using the DAIC-WOZ dataset, our proposed framework significantly improves the classification performance, achieving an F1 value of 0.80 for depression detection. Experimental results demonstrate that our method outperforms other existing depression detection models based on a single modality.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Emotion and Mood Recognition · Mental Health via Writing · Mental Health Research Topics
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China
Computer Intelligent System Laboratory, Zhejiang University of Technology, Hangzhou 310023, China
School of Information Science and Engineering, Hangzhou Normal University, Hangzhou 311121, China
Funding
The “Pioneer” and “Leading Goose” R&D Program of Zhejiang Province · National Natural Science Foundation of China
References (54)
Show source-provided references
1. Ma, X., Yang, H., Chen, Q., Huang, D., and Wang, Y. (2016, January 16). Depaudionet: An efficient deep model for audio based depression classification. Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge, Amsterdam, The Netherlands.10.1145/2988257.2988267 ↗
2. Psychometric testing of the Depressive Cognition Scale in Korean adults10.1016/j.apnr.2011.04.003 ↗
3. Global, regional, and national incidence, prevalence, and years lived with disability for 328 diseases and injuries for 195 countries, 1990–2016: A systematic analysis for the Global Burden of Disease Study 201610.1016/S0140-6736(17)32154-2 ↗
4. A structured interview guide for the Hamilton Depression Rating Scale10.1001/archpsyc.1988.01800320058007 ↗
5. A Self-Rating Depression Scale10.1001/archpsyc.1965.01720310065008 ↗
6. The PHQ-9: Validity of a brief depression severity measure10.1046/j.1525-1497.2001.016009606.x ↗
7. Beck depression inventory-II
8. An augmented artificial intelligence approach for chronic diseases prediction10.3389/fpubh.2022.860396 ↗
9. Williamson, J.R., Quatieri, T.F., Helfer, B.S., Horwitz, R., Yu, B., and Mehta, D.D. (2013, January 21). Vocal biomarkers of depression based on motor incoordination. Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge, Barcelona, Spain.10.1145/2512530.2512531 ↗
10. Visually Interpretable representation learning for depression recognition from facial images10.1109/TAFFC.2018.2828819 ↗
11. Suhara, Y., Xu, Y., and Pentland, A.S. (2017, January 3–7). DeepMood: Forecasting Depressed Mood Based on Self-Reported Histories via Recurrent Neural Networks. Proceedings of the 26th International Conference. International World Wide Web Conferences Steering Committee, Perth, Australia.10.1145/3038912.3052676 ↗
12. Gratch, J., Artstein, R., Lucas, G.M., Stratou, G., Scherer, S., Nazarian, A., and Morency, L.P. (2014, January 26–31). The distress analysis interview corpus of human and computer interviews. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland.
13. Intelligent Identification of Hate Speeches to address the increased rate of Individual Mental Degeneration10.1016/j.procs.2023.01.444 ↗
14. A Model of Normality Inspired Deep Learning Framework for Depression Relapse Prediction Using Audiovisual Data10.1016/j.cmpb.2022.107132 ↗
15. Mehrabian, A. (2017). Communication Theory, Routledge.
16. Meng, H., Huang, D., Wang, H., Yang, H., Ai-Shuraifi, M., and Wang, Y. (2013, January 21). Depression recognition based on dynamic facial and vocal expression features using partial least square regression. Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge, Barcelona, Spain.10.1145/2512530.2512532 ↗
17. Pampouchidou, A., Simantiraki, O., Fazlollahi, A., Pediaditis, M., Manousos, D., Roniotis, A., and Tsiknakis, M. (2016, January 16). Depression assessment by fusing high and low level features from audio, video, and text. Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge, Amsterdam, The Netherlands.10.1145/2988257.2988266 ↗
18. Syed, Z.S., Sidorov, K., and Marshall, D. (2017, January 23–27). Depression severity prediction based on biomarkers of psychomotor retardation. Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, Mountain View, CA, USA.10.1145/3133944.3133947 ↗
19. Mehrabian, A., and Russell, J.A. (1974). An Approach to Environmental Psychology, MIT Press.
20. Effect of nature prescriptions on cardiometabolic and mental health, and physical activity: A systematic review10.1016/S2542-5196(23)00025-6 ↗
21. Motor and cognitive aspects of motor retardation in depression10.1016/S0165-0327(99)00068-3 ↗
22. Cohn, J.F., Kruez, T.S., Matthews, I., Yang, Y., Nguyen, M.H., Padilla, M.T., and De la Torre, F. (2009, January 10–12). Detecting depression from facial actions and vocal prosody. Proceedings of the 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, Amsterdam, The Netherlands.10.1109/ACII.2009.5349358 ↗
23. Mcintyre, G., Gocke, R., Hyett, M., Green, M., and Breakspear, M. (2009, January 10–12). An approach for automatically measuring facial activity in depressed subjects. Proceedings of the International Conference on Affective Computing and Intelligent Interaction and Workshops, Amsterdam, The Netherlands.10.1109/ACII.2009.5349593 ↗
24. Automated Facial Action Coding System for dynamic analysis of facial expressions in neuropsychiatric disorders10.1016/j.jneumeth.2011.06.023 ↗
25. Coupled HMM-based multimodal fusion for mood disorder detection through elicited audio-visual signals10.1007/s12652-016-0395-y ↗
26. Gupta, R., Malandrakis, N., Xiao, B., Guha, T., Van Segbroeck, M., Black, M., and Narayanan, S. (2014, January 3–7). Multimodal Prediction of Affective Dimensions and Depression in Human-Computer Interactions. Proceedings of the International Workshop on Audio/Visual Emotion Challenge ACM, Orlando, FL, USA.10.1145/2661806.2661810 ↗
27. Nasir, M., Jati, A., Shivakumar, P.G., Nallan Chakravarthula, S., and Georgiou, P.l. (2016, January 16). Multimodal and multiresolution depression detection from speech and facial landmark features. Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge, Amsterdam, The Netherlands.10.1145/2988257.2988261 ↗
28. Wang, Y., Ma, J., Hao, B., Wang, X., Mei, J., and Li, S. (2020, January 9–7). Automatic depression detection via facial expressions using multiple instance learning. Proceedings of the 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), Iowa City, IA, USA.10.1109/ISBI45749.2020.9098396 ↗
29. An automatic diagnostic network using skew-robust adversarial discriminative domain adaptation to evaluate the severity of depression10.1016/j.cmpb.2019.01.006 ↗
30. Baltrusaitis, T., Robinson, P., and Morency, L.P. (2016, January 7–10). OpenFace: An open source facial behavior analysis toolkit. Proceedings of the 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Placid, NY, USA.10.1109/WACV.2016.7477553 ↗
31. Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., and Xu, C. (2020, January 13–19). GhostNet: More features from cheap operations. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00165 ↗
32. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
33. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18–23). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00474 ↗
34. A Method Based on Knowledge Distillation for Fish School Stress State Recognition in Intensive Aquaculture
35. A comparative assessment of machine learning algorithms with the Least Absolute Shrinkage and Selection Operator for breast cancer detection and prediction10.1016/j.dajour.2023.100245 ↗
36. Erokhina, O.V., Borisenko, B.B., and Fadeev, A.S. (July, January 30). Analysis of the Multilayer Perceptron Parameters Impact on the Quality of Network Attacks Identification. Proceedings of the 2021 Systems of Signal Synchronization, Generating and Processing in Telecommunications, Kaliningrad, Russia.10.1109/SYNCHROINFO51390.2021.9488344 ↗
37. Applying deep learning for epilepsy seizure detection and brain mapping visualization10.1145/3241056 ↗
38. The PHQ-8 as a measure of current depression in the general population10.1016/j.jad.2008.06.026 ↗
39. Differential evolution algorithm with strategy adaptation for global numerical optimization10.1109/TEVC.2008.927706 ↗
40. Zhang, Z. (2018, January 4–6). Improved adam optimizer for deep neural networks. Proceedings of the 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS), Banff, AB, Canada.10.1109/IWQoS.2018.8624183 ↗
41. Manoret, P., Chotipurk, P., Sunpaweravong, S., Jantrachotechatchawan, C., and Duangrattanalert, K. (2021). Automatic Detection of Depression from Stratified Samples of Audio Data. arXiv.
42. MFCC-based recurrent neural network for automatic clinical depression recognition and assessment from speech10.1016/j.bspc.2021.103107 ↗
43. Dinkel, H., Wu, M., and Yu, K. (2019). Text-based depression detection on sparse data. arXiv.
44. Arioz, U., Smrke, U., Plohl, N., and Mlakar, I. (2022). Scoping Review on the Multimodal Classification of Depression and Experimental Study on Existing Multimodal Models. Diagnostics, 12.10.3390/diagnostics12112683 ↗
45. Lam, G., Dongyan, H., and Lin, W. (2019, January 12–17). Context-aware deep learning for multi-modal depression detection. Proceedings of the ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8683027 ↗
46. Song, S., Shen, L., and Valstar, M. (2018, January 15–19). Human behaviour-based automatic depression analysis using hand-crafted statistics and deep learned spectral features. Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi’an, China.10.1109/FG.2018.00032 ↗
47. Wei, P.-C., Peng, K., Roitberg, A., Yang, K., Zhang, J., and Stiefelhagen, R. (2022). Multi-modal depression estimation based on sub-attentional fusion. arXiv.10.1007/978-3-031-25075-0_42 ↗
48. Haque, A., Guo, M., Miner, A.S., and Fei-Fei, L. (2018). Measuring depression symptom severity from spoken language and 3D facial expressions. arXiv.
49. Integrating Deep and Shallow Models for Multi-Modal Depression Analysis-Hybrid Architectures10.1109/TAFFC.2018.2870398 ↗
50. Guo, Y., Zhu, C., Hao, S., and Hong, R. (2022). Automatic depression detection via learning and fusing features from visual cues. IEEE Trans. Comput. Soc. Syst., 1–8.10.1109/TCSS.2022.3202316 ↗
51. Assessment of brain tumor due to the usage of MATLAB performance10.1166/jmihi.2017.2187 ↗
52. Chen, L., Yang, Y., Wang, Z., Zhang, J., Zhou, S., and Wu, L. (2023). Lightweight Underwater Target Detection Algorithm Based on Dynamic Sampling Transformer and Knowledge-Distillation Optimization. J. Mar. Sci. Eng., 11.10.3390/jmse11020426 ↗
53. An efficient Apriori algorithm for frequent pattern in human intoxication data10.1007/s11334-022-00523-w ↗
54. DISNet: A sequential learning framework to handle occlusion in human action recognition with video acquisition sensors10.1016/j.dsp.2022.103763 ↗