journal article·2022-08-28Open access

Automated Detection of the Competency of Delivering Guided Self-Help for Anxiety via Speech and Language Processing

Dalia Attas, Niall Power, Jessica Smithies, Charlotte Bee, Vikki Aadahl, Stephen Kellett, Chris Blackmore, Heidi Christensen

Applied Sciences· Volume 12· Issue 17

Abstract

Speech and language play an essential role in automatically assessing several psychotherapeutic qualities. These automation procedures require translating the manual rating qualities to speech and language features that accurately capture the assessed psychotherapeutic quality. Speech features can be determined by analysing recordings of psychotherapeutic conversations (acoustics), while language-based analyses rely on the transcriptions of such psychotherapeutic conversations (linguistics). Guided self-help is a psychotherapeutic intervention that mainly relay on therapeutic competency of practitioners. This paper investigates the feasibility of automatically analysing guided self-help sessions for mild-to-moderate anxiety to detect and predict practitioner competence. This analysis is performed on sessions drawn from a patient preference randomised controlled trial using actual patient-practitioner conversations manually rated using a valid and reliable measure of competency. The results show the efficacy and potential of automatically detecting practitioners’ competence using a system based on acoustic and linguistic features extracted from transcripts generated by an automatic speech recogniser. Feature extraction, feature selection and classification or regression have been implemented as blocks of the prediction model. The Lasso regression model achieved the best prediction results with an R of 0.92 and lower error rates with an MAE of 1.66 and RMSE of 2.25.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Mental Health via Writing · Digital Mental Health Interventions · Emotion and Mood Recognition

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Computer Science, University of Sheffield, Sheffield S1 4DP, UK

Derbyshire Community Health Services NHS Foundation Trust, Chesterfield DE45 1AD, UK

Improving Access to Psychological Therapies, Pennine Care NHS Foundation Trust, Ashton-under-Lyne OL6 7SR, UK

Rotherham Doncaster and South Humber NHS Foundation Trust and Clinical and Applied Psychology Unit, Department of Psychology, University of Sheffield, Sheffield DN4 8QN, UK

School of Health and Related Research (ScHARR), University of Sheffield, Sheffield S1 4DA, UK

Funding

Saudi Ministry of Higher Education · Association of Cognitive Analytic Therapists and Catalyse

References (57)

Show source-provided references
1. Wickramasinghe, N., and Geisler, E. (2008). Encyclopedia of Healthcare Information Systems, IGI Global.10.4018/978-1-59904-889-5 ↗
2. Repeatability of commonly used speech and language features for clinical applications10.1159/000511671 ↗
3. Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence10.1037/0022-006X.61.4.620 ↗
4. Benchmarking treatment adherence and therapist competence in individual cognitive-behavioral treatment for youth anxiety disorders10.1080/15374416.2017.1381914 ↗
5. Ringeval, F., Schuller, B., Valstar, M., Cowie, R., Kaya, H., Schmitt, M., Amiriparian, S., Cummins, N., Lalanne, D., and Michaud, A. (2018, January 22). AVEC 2018 workshop and challenge: Bipolar disorder and cross-cultural affect recognition. Proceedings of the 2018 on Audio/Visual Emotion Challenge and Workshop, Seoul, Korea.10.1145/3266302.3266316 ↗
6. Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge10.1016/j.specom.2011.01.011 ↗
7. A survey on big data-driven digital phenotyping of mental health10.1016/j.inffus.2019.04.001 ↗
8. Therapist competence in global mental health: Development of the ENhancing Assessment of Common Therapeutic factors (ENACT) rating scale10.1016/j.brat.2015.03.009 ↗
9. Defining the Assessment and Treatment Competencies to Deliver Low-Intensity Cognitive Behavior Therapy: A Multi-Center Validation Study10.1016/j.beth.2020.01.006 ↗
10. Kellett, S., Simmonds-Buckley, M., Limon, E., Hague, J., Hughes, L., Stride, C., and Millings, A. (2021). Low Intensity Cognitive Behavioural Competency Scale Manual, Unpublished document.
11. Therapist competence, therapy quality, and therapist training10.1016/j.brat.2011.03.005 ↗
12. Educating psychotherapy supervisors10.1176/appi.psychotherapy.2012.66.3.279 ↗
13. A review of therapist characteristics and techniques positively impacting the therapeutic alliance10.1016/S0272-7358(02)00146-0 ↗
14. Therapist competence and therapeutic alliance are important in the treatment of health anxiety (hypochondriasis)10.1016/j.psychres.2015.03.042 ↗
15. Attas, D., Kellett, S., Blackmore, C., and Christensen, H. (2021, January 15–17). Automatic Time-Continuous Prediction of Emotional Dimensions during Guided Self Help for Anxiety Disorders. Proceedings of the FRIAS Junior Researcher Conference: Human Perspectives on Spoken Human-Machine Interaction (SpoHuMa21), Online.
16. Beneath the surface of the therapeutic interaction: The psychoanalytic method in modern dress10.1177/000306510705500412 ↗
17. Linking verbal and non-verbal representations: Computer analysis of referential activity10.1348/000711299160040 ↗
18. Nasir, M., Baucom, B., Georgiou, P., and Narayanan, S. (2017). Predicting couple therapy outcomes based on speech acoustic features. PLoS ONE, 12.10.1371/journal.pone.0185123 ↗
19. Amir, N., Mixdorff, H., Amir, O., Rochman, D., Diamond, G., Pfitzinger, H., Levi-Isserlish, T., and Abramson, S. (2010, January 10–14). Unresolved anger: Prosodic analysis and classification of speech from a therapeutic setting. Proceedings of the Speech Prosody 2010-Fifth International Conference, Chicago, IL, USA.10.21437/SpeechProsody.2010-88 ↗
20. Sümer, Ö., Beyan, C., Ruth, F., Kramer, O., Trautwein, U., and Kasneci, E. (2021). Estimating Presentation Competence using Multimodal Nonverbal Behavioral Cues. arXiv.
21. Ringeval, F., Marchi, E., Grossard, C., Xavier, J., Chetouani, M., Cohen, D., and Schuller, B. (2016, January 8–12). Automatic analysis of typical and atypical encoding of spontaneous emotion in the voice of children. Proceedings of the INTERSPEECH 2016, 17th Annual Conference of the International Speech Communication Association (ISCA), San Francisco, CA, USA.10.21437/Interspeech.2016-766 ↗
22. An emotional modulation model as signature for the identification of children developmental disorders10.1038/s41598-018-32454-7 ↗
23. Gideon, J., Schatten, H., McInnis, M., and Provost, E. (2019, January 15–19). Emotion recognition from natural phone conversations in individuals with and without recent suicidal ideation. Proceedings of the INTERSPEECH, Graz, Austria.10.21437/Interspeech.2019-1830 ↗
24. Zhang, Z., Lin, W., Liu, M., and Mahmoud, M. (2020, January 16–20). Multimodal deep learning framework for mental disorder recognition. Proceedings of the 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), Buenos Aires, Argentina.10.1109/FG47880.2020.00033 ↗
25. Linguistic features of the therapeutic alliance in the first session: A psychotherapy process study
26. Assessing key linguistic dimensions of ruptures in the therapeutic alliance10.1007/s10936-021-09768-1 ↗
27. Deep learning-based methods for sentiment analysis on Nepali COVID-19-related tweets10.1155/2021/2158184 ↗
28. Recognizing hotspots in Brief Eclectic Psychotherapy for PTSD by text and audio mining10.1080/20008198.2020.1726672 ↗
29. Tavabi, L., Stefanov, K., Zhang, L., Borsari, B., Woolley, J., Scherer, S., and Soleymani, M. (2020, January 25–29). Multimodal Automatic Coding of Client Behavior in Motivational Interviewing. Proceedings of the 2020 International Conference on Multimodal Interaction, Virtual.10.1145/3382507.3418853 ↗
30. Bhardwaj, V., Ben Othman, M.T., Kukreja, V., Belkhier, Y., Bajaj, M., Goud, B.S., Rehman, A.U., Shafiq, M., and Hamam, H. (2022). Automatic Speech Recognition (ASR) Systems for Children: A Systematic Literature Review. Appl. Sci., 12.10.3390/app12094419 ↗
31. A systematic comparison of contemporary automatic speech recognition engines for conversational clinical speech
32. Xiao, B., Imel, Z., Georgiou, P., Atkins, D., and Narayanan, S. (2015). “Rate my therapist”: Automated detection of empathy in drug and alcohol counseling via speech and language processing. PLoS ONE, 10.10.1371/journal.pone.0143055 ↗
33. Chen, Z., Flemotomos, N., Ardulov, V., Creed, T., Imel, Z., Atkins, D., and Narayanan, S. (2021, January 1–5). Feature fusion strategies for end-to-end evaluation of cognitive behavior therapy sessions. Proceedings of the 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Virtual Conference.10.1109/EMBC46164.2021.9629694 ↗
34. Automated evaluation of psychotherapy skills using speech and language technologies10.3758/s13428-021-01623-4 ↗
35. A pragmatic patient preference trial of cognitive behavioural versus cognitive analytic guided self-help for anxiety disorders
36. Therapist effects and moderators of effectiveness and efficiency in psychological wellbeing practitioners: A multilevel modelling analysis10.1016/j.brat.2015.04.001 ↗
37. Transcriber: Development and use of a tool for assisting speech corpora production10.1016/S0167-6393(00)00067-4 ↗
38. Renals, S., and Swietojanski, P. (2017). Distant speech recognition experiments using the AMI Corpus. New Era for Robust Speech Recognition, Springer.10.1007/978-3-319-64680-0_16 ↗
39. Wang, Y. (2020). Automatic Speech Recognition Model for Swedish Using Kaldi, KTH School of Electrical Engineering and Computer Science.
40. Povey, D., Ghoshal, A., Boulianne, G., Burget, L., Glembek, O., Goel, N., Hannemann, M., Motlicek, P., Qian, Y., and Schwarz, P. (2011, January 11–15). The Kaldi speech recognition toolkit. Proceedings of the IEEE 2011 Workshop on Automatic Speech Recognition and Understanding, Waikoloa, HI, USA.
41. Mirheidari, B., Blackburn, D., O’Malley, R., Venneri, A., Walker, T., Reuber, M., and Christensen, H. (2020, January 25–29). Improving Cognitive Impairment Classification by Generative Neural Network-Based Feature Augmentation. Proceedings of the INTERSPEECH, Shanghai, China.10.21437/Interspeech.2020-2433 ↗
42. Mirheidari, B., Pan, Y., Blackburn, D., O’Malley, R., and Christensen, H. (September, January 30). Identifying Cognitive Impairment Using Sentence Representation Vectors. Proceedings of the INTERSPEECH 2021, Brno, Czechia.10.21437/Interspeech.2021-915 ↗
43. Neonatal bowel sound detection using convolutional neural network and Laplace hidden semi-Markov model10.1109/TASLP.2022.3178225 ↗
44. The Geneva minimalistic acoustic parameter set (GeMAPS) for voice research and affective computing10.1109/TAFFC.2015.2457417 ↗
45. Eyben, F., Wöllmer, M., and Schuller, B. (2010, January 25–29). Opensmile: The munich versatile and fast open-source audio feature extractor. Proceedings of the 18th ACM International Conference on Multimedia, Firenze, Italy.10.1145/1873951.1874246 ↗
46. Valstar, M., Gratch, J., Schuller, B., Ringeval, F., Lalanne, D., Torres Torres, M., Scherer, S., Stratou, G., Cowie, R., and Pantic, M. (2016, January 16). AVEC 2016: Depression, mood, and emotion recognition workshop and challenge. Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge, Amsterdam, The Netherlands.10.1145/2964284.2980532 ↗
47. OpenXBOW: Introducing the passau open-source crossmodal Bag-of-Words Toolkit
48. Overview of computer measures of the referential process10.1007/s10936-021-09761-8 ↗
49. Tocatly, K., Bucci, W., and Maskit, B. (2019). Developing a Preliminary Measure of the Arousal Function of the Referential Process, City University of New York. [Poster presentation]; Research Day Colloquium at the City College of New York’s Clinical Psychology Doctoral Program.
50. Maskit, B. (2021, September 22). The Discourse Attributes Analysis Program (DAAP) (Series 8) [Computer Software]. Available online: http://www.thereferentialprocess.org/dictionary-measures-and-computer-programs.
51. Murphy, S., Maskit, B., and Bucci, W. (2015, January 5). Putting feelings into words: Cross-linguistic markers of the referential process. Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality, Denver, CO, USA.10.3115/v1/W15-1210 ↗
52. Gene selection for cancer classification using support vector machines10.1023/A:1012487302797 ↗
53. Scikit-learn: Machine learning in Python
54. Mirheidari, B., Blackburn, D., O’Malley, R., Walker, T., Venneri, A., Reuber, M., and Christensen, H. (2019, January 12–17). Computational Cognitive Assessment: Investigating the Use of an Intelligent Virtual Agent for the Detection of Early Signs of Dementia. Proceedings of the ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK.10.1109/ICASSP.2019.8682423 ↗
55. Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. (2015, January 19–24). Librispeech: An ASR corpus based on public domain audio books. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, QLD, Australia.10.1109/ICASSP.2015.7178964 ↗
56. Acoustic speech markers for schizophrenia-spectrum disorders: A diagnostic and symptom-recognition tool
57. Corrales-Astorgano, M., Martínez-Castilla, P., Escudero-Mancebo, D., Aguilar, L., González-Ferreras, C., and Carde noso-Payo, V. (2019). Automatic assessment of prosodic quality in down syndrome: Analysis of the impact of speaker heterogeneity. Appl. Sci., 9.10.3390/app9071440 ↗