Are You Depressed? Analyze User Utterances to Detect Depressive Emotions Using DistilBERT
Source abstract
Abstract
This paper introduces the Are u Depressed (AuD) model, which aims to detect depressive emotional intensity and classify detailed depressive symptoms expressed in user utterances. The study includes the creation of a BWS dataset using a tool for the Best-Worst Scaling annotation task and a DSM-5 dataset containing nine types of depression annotations based on major depressive disorder (MDD) episodes in the Diagnostic and Statistical Manual of Mental Disorders (DSM-5). The proposed model employs the DistilBERT model for both tasks and demonstrates superior performance compared to other machine learning and deep learning models. We suggest using our model for real-time depressive emotion detection tasks that demand speed and accuracy. Overall, the AuD model significantly advances the accurate detection of depressive emotions in user utterances.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Mental Health via Writing · Sentiment Analysis and Opinion Mining · Emotion and Mood Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Artificial Intelligence Convergence, Sungkyunkwan University, Seoul 03063, Republic of Korea
College of Computing and Informatics, Sungkyunkwan University, Seoul 03063, Republic of Korea
Funding
Korean government (MSIT)
References (48)
Show source-provided references
1. (2022, December 28). A Red Light for Modern Mental Health and Stress Management Are Essential. Available online: http://www.medical-tribune.co.kr/news/articleView.html?idxno=100431.
2. Guangyao, S., Jiang, J., Liqiang, N., Fuli, F., Cunjun, Z., Tianrui, H., Tat-Seng, C., and Wenwu, Z. (2017, January 19–25). Depression detection via harvesting social media: A multimodal dictionary learning solution. Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI'17), Sidney, Australia.
3. Cohan, A., Desmet, B., Yates, A., Soldaini, L., MacAvaney, S., and Goharian, N. (2018, January 20–26). SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions. Proceedings of the 27th International Conference on Computational Linguistics, Association for Computational Linguistics, Santa Fe, NM, USA.
4. Depression and suicide analysis using machine learning and NLP
5. A lexicon-based approach to examine depression detection in social media: The case of Twitter and university community10.1057/s41599-022-01313-2 ↗
6. Lin, L., Chen, X., Shen, Y., and Zhang, L. (2020). Towards Automatic Depression Detection: A BiLSTM/1D CNN-Based Model. Appl. Sci., 10.10.3390/app10238701 ↗
7. Louviere, J.J., Flynn, T.N., and Marley, A.A.J. (2015). Best-Worst Scaling: Theory, Methods and Applications, Cambridge University Press.10.1017/CBO9781107337855 ↗
8. American Psychiatric Association (2013). Diagnostic And Statistical Manual of Mental Disorders, American Psychiatric Association. [5th ed.].10.1176/appi.books.9780890425596 ↗
9. (2023, April 20). BWS Tagging Tool Github. Available online: https://github.com/Jaedong95/BWS-Tagging.
10. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv.
11. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019, January 2–7). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MI, USA.
12. Clark, K., Luong, M.-T., Le Quoc, V., and Christopher, D. (2020). Manning: Electra: Pre-training text encoders as discriminators rather than generators. arXiv.
13. (2023, January 10). Major Depressive Disorder (Diagnosis). Available online: https://chsciowa.org/sites/chsciowa.org/files/resource/files/7_-_depression_dsm-5_checklist.pdf.
14. Yates, A., Cohan, A., and Goharian, N. (2017, January 9–11). Depression and Self-Harm Risk Assessment in Online Forums. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.10.18653/v1/D17-1322 ↗
15. Predicting Depression via Social Media10.1609/icwsm.v7i1.14432 ↗
16. Detection of Types of Mental Illness through the Social Network Using Ensembled Deep Learning Model
17. Amanat, A., Rizwan, M., Javed, A.R., Abdelhaq, M., Alsaqour, R., Pandya, S., and Uddin, M. (2022). Deep learning for depression detection from textual data. Electronics, 11.10.3390/electronics11050676 ↗
18. DEPTWEET: A typology for social media texts to detect depression severities10.1016/j.chb.2022.107503 ↗
19. Analysis of depression in social media texts through the Patient Health Questionnaire-9 and natural language processing10.1177/20552076221114204 ↗
20. Ji, S., Zhang, T., Ansari, L., Fu, J., Tiwari, P., and Cambria, E. (2022, January 20–25). MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare. Proceedings of the Thirteenth Language Resources and Evaluation Conference, Marseille, France.
21. Rodrigues Makiuchi, M., Warnita, T., Uto, K., and Shinoda, K. (2019, January 21). Multimodal Fusion of BERT-CNN and Gated CNN Representations for Depression Detection. Proceedings of the 9th International on Audio/Visual Emotion Challenge and Workshop (AVEC’19), Nice, France.10.1145/3347320.3357694 ↗
22. Afef, S., Othman, S.B., and Saoud, S.B. (2020, January 15–18). Hybrid CNN-SVM classifier for efficient depression detection system. Proceedings of the 4th International Conference on Advanced Systems and Emergent Technologies (IC_ASET), Hammamet, Tunisia.
23. Li, Y., Su, H., Shen, X., Li, W., Cao, Z., and Niu, S. (December, January 27). DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset. Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Taipei, Taiwan.
24. (2023, January 08). Subreddit r/Depression. Available online: https://www.reddit.com/r/depression/.
25. (2023, January 08). Reddit Archive Data. Available online: https://files.pushshift.io/reddit/.
26. (2023, March 05). Papago API. Available online: https://developers.naver.com/docs/papago/papago-detectlangs-overview.md.
27. (2023, February 28). Bert-Base-NER Model. Available online: https://huggingface.co/dslim/bert-base-NER.
28. Mohammad, S., and Bravo-Marquez, F. (2017, January 3–4). Emotion Intensities in Tweets. Proceedings of the 6th Joint Conference on Lexical and Computational Semantics (*SEM 2017), Vancouver, BC, Canada.10.18653/v1/S17-1007 ↗
29. (2023, March 10). Tweet Emotion Intensity Dataset Webpage. Available online: https://saifmohammad.com/WebPages/TweetEmotionIntensity-dataviz.html.
30. Kiritchenko, S., and Saif, M.M. (2017, January 12–17). Capturing Reliable Fine-Grained Sentiment Associations by Crowdsourcing and Best–Worst Scaling. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA.10.18653/v1/N16-1095 ↗
31. (2023, March 14). MaxDiff Analysis: Simple Counting, Individual-Level Logit, and HB. Available online: https://sawtoothsoftware.com/resources/technical-papers/maxdiff-analysis-simple-counting-individual-level-logit-and-hb.
32. (2023, March 14). Min-Max Scale Using Sklearn. Available online: https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html.
33. (2023, February 19). How to Scale into the 0–1 Range Using Min-Max Normalization. Available online: https://androidkt.com/how-to-scale-data-to-range-using-minmax-normalization/.
34. Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv.
35. Clark, K., Khandelwal, U., Levy, O., and Manning. C, D. (2019, January 1). What Does BERT Look at? An Analysis of BERT’s Attention. Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, Florence, Italy.10.18653/v1/W19-4828 ↗
36. (2023, February 22). About BERT. Available online: https://heekangpark.github.io/nlp/huggingface-bert.
37. (2023, February 24). BertViz Github. Available online: https://github.com/jessevig/bertviz.
38. He, P., Liu, X., Gao, J., and Chen, W. (2020). Deberta: Decoding-enhanced bert with disentangled attention. arXiv.
39. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv.
40. Mariette, A., Khanna, R., Awad, M., and Khanna, R. (2015). Efficient Learning Machines: Theories, Concepts, and Applications for Engineers and System Designers, Springer.
41. K-nearest neighbor10.4249/scholarpedia.1883 ↗
42. Leung, K.M. (2007). Naive Bayesian Classifier, Polytechnic University Department of Computer Science/Finance and Risk Engineering.
43. Random Forests10.1023/A:1010933404324 ↗
44. Siino, M., Di Nuovo, E., Tinnirello, I., and La Cascia, M. (2022). Fake News Spreaders Detection: Sometimes Attention Is Not All You Need. Information, 13.10.3390/info13090426 ↗
45. Principal component analysis10.1002/wics.101 ↗
46. (2023, April 24). About t-SNE. Available online: https://lvdmaaten.github.io/tsne/.
47. (2022, December 25). PCA vs. t-SNE. Available online: https://skyeong.net/284.
48. Visualizing Data using t-SNE