The Impact of Pause and Filler Word Encoding on Dementia Detection with Contrastive Learning
Source abstract
Abstract
Dementia is primarily caused by neurodegenerative diseases like Alzheimer’s disease (AD). It affects millions worldwide, making detection and monitoring crucial. This study focuses on the detection of dementia from speech transcripts of controls and dementia groups. We propose encoding in-text pauses and filler words (e.g., “uh” and “um”) in text-based language models and thoroughly evaluating their impact on performance (e.g., accuracy). Additionally, we suggest using contrastive learning to improve performance in a multi-task framework. Our results demonstrate the effectiveness of our approaches in enhancing the model’s performance, achieving 87% accuracy and an 86% f1-score. Compared to the state of the art, our approach has similar performance despite having significantly fewer parameters. This highlights the importance of pause and filler word encoding on the detection of dementia.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Emotion and Mood Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electrical and Computer Engineering, North Carolina State University, Engineering Bldg II, 890 Oval Dr, Raleigh, NC 27606, USA
Division of Speech and Hearing Sciences, Department of Allied Health Sciences, University of North Carolina at Chapel Hill, Chapel Hill, NC 27559, USA
Funding
National Science Foundation
References (42)
Show source-provided references
1. A.D. International (2024, May 25). Dementia Statistics. Available online: https://www.alz.co.uk/research/statistics.
2. Current and future treatments in Alzheimer disease: An update10.1177/1179573520907397 ↗
3. Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv.
4. Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv.
5. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020). Language Models are Few-Shot Learners. arXiv.
6. Semantic and sentiment analysis of selected Bhagavad Gita translations using BERT-based language framework10.1109/ACCESS.2022.3152266 ↗
7. Qu, C., Yang, L., Qiu, M., Croft, W.B., Zhang, Y., and Iyyer, M. (2019, January 21–25). BERT with history answer embedding for conversational question answering. Proceedings of the 42nd international ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France.10.1145/3331184.3331341 ↗
8. Hakala, K., and Pyysalo, S. (2019, January 4). Biomedical named entity recognition with multilingual BERT. Proceedings of the 5th Workshop on BioNLP Open Shared Tasks, Hong Kong, China.10.18653/v1/D19-5709 ↗
9. Thomas, C., Keselj, V., Cercone, N., Rockwood, K., and Asp, E. (August, January 29). Automatic detection and rating of dementia of Alzheimer type through lexical analysis of spontaneous speech. Proceedings of the IEEE International Conference Mechatronics and Automation, Niagara Falls, ON, Canada.
10. Analysis of brief language tests in the detection of cognitive decline and dementia10.1590/S1980-57642008DN10100007 ↗
11. Lexical acquisition in progressive aphasia and frontotemporal dementia10.1080/02643290600890657 ↗
12. Yuan, J., Bian, Y., Cai, X., Huang, J., Ye, Z., and Church, K. (2020, January 25–29). Disfluencies and Fine-Tuning Pre-Trained Language Models for Detection of Alzheimer’s Disease. Proceedings of the Interspeech, Shanghai, China.10.21437/Interspeech.2020-2516 ↗
13. Valsaraj, A., Madala, I., Garg, N., and Baths, V. (2021, January 26–27). Alzheimer’s dementia detection using acoustic & linguistic features and pre-trained BERT. Proceedings of the 2021 8th International Conference on Soft Computing & Machine Intelligence (ISCMI), Cairo, Egypt.10.1109/ISCMI53840.2021.9654804 ↗
14. Transformer-based deep neural network language models for Alzheimer’s disease risk assessment from targeted speech10.1186/s12911-021-01456-3 ↗
15. Nambiar, A.S., Likhita, K., Pujya, K.V.S.S., Gupta, D., Vekkot, S., and Lalitha, S. (2022, January 24–26). Comparative study of Deep Classifiers for Early Dementia Detection using Speech Transcripts. Proceedings of the 2022 IEEE 19th India Council International Conference (INDICON), Kochi, India.10.1109/INDICON56171.2022.10039705 ↗
16. Cai, H., Huang, X., Liu, Z., Liao, W., Dai, H., Wu, Z., Zhu, D., Ren, H., Li, Q., and Liu, T. (2023, January 1–3). Multimodal Approaches for Alzheimer’s Detection Using Patients’ Speech and Transcript. Proceedings of the International Conference on Brain Informatics, Hoboken, NJ, USA.10.1007/978-3-031-43075-6_34 ↗
17. Guo, Z., Liu, Z., Ling, Z., Wang, S., Jin, L., and Li, Y. (2020, January 8–13). Text classification by contrastive learning and cross-lingual data augmentation for alzheimer’s disease detection. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain.10.18653/v1/2020.coling-main.542 ↗
18. Saltz, P., Lin, S.Y., Cheng, S.C., and Si, D. (2021, January 9–12). Dementia Detection using Transformer-Based Deep Learning and Natural Language Processing Models. Proceedings of the 2021 IEEE 9th International Conference on Healthcare Informatics (ICHI), Victoria, BC, Canada.10.1109/ICHI52183.2021.00094 ↗
19. Pan, Y., Mirheidari, B., Reuber, M., Venneri, A., Blackburn, D., and Christensen, H. (2019, January 15–19). Automatic hierarchical attention neural network for detecting AD. Proceedings of the Interspeech 2019. International Speech Communication Association (ISCA), Graz, Austria.10.21437/Interspeech.2019-1799 ↗
20. A data augmentation-based framework to handle class imbalance problem for Alzheimer’s stage detection10.1109/ACCESS.2019.2932786 ↗
21. Mirheidari, B., Blackburn, D., O’Malley, R., Venneri, A., Walker, T., Reuber, M., and Christensen, H. (2020, January 25–29). Improving Cognitive Impairment Classification by Generative Neural Network-Based Feature Augmentation. Proceedings of the INTERSPEECH, Shanghai, China.10.21437/Interspeech.2020-2433 ↗
22. A novel AI-based system for detection and severity prediction of dementia using MRI10.1109/ACCESS.2021.3127394 ↗
23. Feng, S.Y., Gangal, V., Wei, J., Chandar, S., Vosoughi, S., Mitamura, T., and Hovy, E. (2021). A survey of data augmentation approaches for NLP. arXiv.10.18653/v1/2021.findings-acl.84 ↗
24. Soleimani, R., Guo, S., Haley, K., Jacks, A., and Lobaton, E. (2024, August 09). Dementia Detection by In-Text Pause Encoding. Preprint. Available online: https://www.preprints.org/manuscript/202408.0727/v1.10.20944/preprints202408.0727.v1 ↗
25. Luz, S., Haider, F., de la Fuente, S., Fromm, D., and MacWhinney, B. (2020, January 25–29). Alzheimer’s Dementia Recognition through Spontaneous Speech: The ADReSS Challenge. Proceedings of the INTERSPEECH 2020, Shanghai, China.10.21437/Interspeech.2020-2571 ↗
26. The Natural History of Alzheimer’s Disease: Description of Study Cohort and Accuracy of Diagnosis10.1001/archneur.1994.00540180063015 ↗
27. Zhou, X., Koltun, V., and Krähenbühl, P. (2020). Tracking objects as points. Proceedings of the European Conference on Computer Vision, Springer.10.1007/978-3-030-58548-8_28 ↗
28. Multitask learning and benchmarking with clinical time series data10.1038/s41597-019-0103-9 ↗
29. Some theorems on distribution functions
30. Jaiswal, A., Babu, A.R., Zadeh, M.Z., Banerjee, D., and Makedon, F. (2020). A survey on contrastive self-supervised learning. Technologies, 9.10.3390/technologies9010002 ↗
31. Chen, Q., Zhang, R., Zheng, Y., and Mao, Y. (2022). Dual Contrastive Learning: Text Classification via Label-Aware Data Augmentation. arXiv.
32. Vigo, I., Coelho, L., and Reis, S. (2022). Speech-and language-based classification of Alzheimer’s disease: A systematic review. Bioengineering, 9.10.3390/bioengineering9010027 ↗
33. Automated Classification of Cognitive Decline and Probable Alzheimer’s Dementia Across Multiple Speech and Language Domains10.1044/2023_AJSLP-22-00403 ↗
34. Radford, A., Kim, J.W., Xu, T., Brockman, G., McLeavey, C., and Sutskever, I. (2022). Robust Speech Recognition via Large-Scale Weak Supervision. arXiv.
35. Baevski, A., Zhou, H., Mohamed, A., and Auli, M. (2020). wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations. arXiv.
36. (2024, May 16). Hugging Face. Available online: https://huggingface.co/google-bert/bert-base-uncased.
37. Guo, S. (2024, May 16). Enhancing Dementia Detection in Text Data through NLP by Encoding Silent Pauses. Available online: https://repository.lib.ncsu.edu/items/7d00284d-35b0-4a9d-8b26-845157739f0f.
38. Kingma, D.P., and Ba, J. (2017). Adam: A Method for Stochastic Optimization. arXiv.
39. Karlekar, S., Niu, T., and Bansal, M. (2018). Detecting linguistic characteristics of Alzheimer’s dementia by interpreting neural models. arXiv.10.18653/v1/N18-2110 ↗
40. Variation and change in the use of hesitation markers in Germanic languages10.1163/22105832-00602001 ↗
41. Uh and um as sociolinguistic markers in British English10.1075/ijcl.16.2.02tot ↗
42. Kobayashi, S. (2018). Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations. arXiv.10.18653/v1/N18-2072 ↗