journal article·2024-09-04Open access

Innovative Speech-Based Deep Learning Approaches for Parkinson’s Disease Classification: A Systematic Review

Lisanne van Gelderen, Cristian Tejedor-García

Applied Sciences· Volume 14· Issue 17

Abstract

Parkinson’s disease (PD), the second most prevalent neurodegenerative disorder worldwide, frequently presents with early-stage speech impairments. Recent advancements in Artificial Intelligence (AI), particularly deep learning (DL), have significantly enhanced PD diagnosis through the analysis of speech data. Nevertheless, the progress of research is restricted by the limited availability of publicly accessible speech-based PD datasets, primarily due to privacy concerns. The goal of this systematic review is to explore the current landscape of speech-based DL approaches for PD classification, based on 33 scientific works published between January 2020 and March 2024. We discuss their available resources, capabilities, and potential limitations, and issues related to bias, explainability, and privacy. Furthermore, this review provides an overview of publicly accessible speech-based datasets and open-source material for PD. The DL approaches identified are categorized into end-to-end (E2E) learning, transfer learning (TL), and deep acoustic feature extraction (DAFE). Among E2E approaches, Convolutional Neural Networks (CNNs) are prevalent, though Transformers are increasingly popular. E2E approaches face challenges such as limited data and computational resources, especially with Transformers. TL addresses these issues by providing more robust PD diagnosis and better generalizability across languages. DAFE aims to improve the explainability and interpretability of results by examining the specific effects of deep features on both other DL approaches and more traditional machine learning (ML) methods. However, it often underperforms compared to E2E and TL approaches.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Voice and Speech Disorders · Speech Recognition and Synthesis · Music and Audio Processing

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Centre for Language Studies, Radboud University, 6500 HD Nijmegen, The Netherlands

Funding

NWO research programme NGF AiNed Fellowship

References (91)

Show source-provided references
1. Ngo, Q.C., Motin, M.A., Pah, N.D., Drotár, P., Kempster, P., and Kumar, D. (2022). Computerized analysis of speech and voice for Parkinson’s disease: A systematic review. Comput. Methods Programs Biomed., 226.10.1016/j.cmpb.2022.107133 ↗
2. Toye, A.A., and Kompalli, S. (2021). Comparative Study of Speech Analysis Methods to Predict Parkinson’s Disease. arXiv.
3. Hireš, M., Gazda, M., Vavrek, L., and Drotár, P. (2022, January 2–5). Voice-Specific Augmentations for Parkinson’s Disease Detection Using Deep Convolutional Neural Network. Proceedings of the 2022 IEEE 20th Jubilee World Symposium on Applied Machine Intelligence and Informatics (SAMI), Poprad, Slovakia.10.1109/SAMI54271.2022.9780856 ↗
4. A forced gaussians based methodology for the differential evaluation of Parkinson’s Disease by means of speech processing10.1016/j.bspc.2018.10.020 ↗
5. Hireš, M., Gazda, M., Drotár, P., Pah, N.D., Motin, M.A., and Kumar, D.K. (2022). Convolutional Neural Network Ensemble for Parkinson’s Disease Detection from Voice Recordings. Comput. Biol. Med., 141.10.1016/j.compbiomed.2021.105021 ↗
6. Speech and Language Therapy Treatment on Hypokinetic Dysarthria in Parkinson Disease: Systematic Review and Meta-Analysis10.1177/0269215520976267 ↗
7. Study of the Performance of Automatic Speech Recognition Systems in Speakers with Parkinson’s Disease
8. Junaid, M., Ali, S., Eid, F., El-Sappagh, S., and Abuhmed, T. (2023). Explainable Machine Learning Models Based on Multimodal Time-Series Data for the Early Detection of Parkinson’s Disease. Comput. Methods Programs Biomed., 234.10.1016/j.cmpb.2023.107495 ↗
9. Jeancolas, L., Petrovska-Delacrétaz, D., Mangone, G., Benkelfat, B.E., Corvol, J.C., Vidailhet, M., and Benali, H. (2021). X-Vectors: New Quantitative Biomarkers for Early Parkinson’s Disease Detection from Speech. Front. Neuroinform., 15.10.3389/fninf.2021.578369 ↗
10. A Systematic Review of Artificial Intelligence (AI) Based Approaches for the Diagnosis of Parkinson’s Disease10.1007/s11831-022-09710-1 ↗
11. Khojasteh, P., Viswanathan, R., Aliahmad, B., Ragnav, S., Zham, P., and Kumar, D.K. (2018, January 28–30). Parkinson’s Disease Diagnosis Based on Multivariate Deep Features of Speech Signal. Proceedings of the 2018 IEEE Life Sciences Conference (LSC), Montreal, QC, Canada.10.1109/LSC.2018.8572136 ↗
12. Ferrante, C., and Scotti, V. (2023). Cross-Lingual Transferability of Voice Analysis Models: A Parkinson’s Disease Case Study. Booklet of Abstracts–Spoken Language in the Medical Field: Linguistic Analysis, Technological Applications and Clinical Tools, Politecnico di Milano University.
13. Favaro, A., Tsai, Y.T., Butala, A., Thebaud, T., Villalba, J., Dehak, N., and Moro-Velázquez, L. (2023). Interpretable Speech Features vs. DNN Embeddings: What to Use in the Automatic Assessment of Parkinson’s Disease in Multi-Lingual Scenarios. Comput. Biol. Med., 166.10.1016/j.compbiomed.2023.107559 ↗
14. Escobar-Grisales, D., Ríos-Urrego, C.D., and Orozco-Arroyave, J.R. (2023). Deep Learning and Artificial Intelligence Applied to Model Speech and Language in Parkinson’s Disease. Diagnostics, 13.10.3390/diagnostics13132163 ↗
15. End-to-End Speech Recognition: A Survey10.1109/TASLP.2023.3328283 ↗
16. Taye, M.M. (2023). Understanding of Machine Learning with Deep Learning: Architectures, Workflow, Applications and Future Directions. Computers, 12.10.3390/computers12050091 ↗
17. The potential for artificial intelligence to transform healthcare: Perspectives from international health leaders10.1038/s41746-024-01097-6 ↗
18. Artificial Intelligence in Bladder Cancer Diagnosis: Current Applications and Future Perspectives10.3390/biomedinformatics3010008 ↗
19. Artificial intelligence in drug discovery: Recent advances and future perspectives10.1080/17460441.2021.1909567 ↗
20. Porumb, M., Stranges, S., Pescapè, A., and Pecchia, L. (2020). Precision medicine and artificial intelligence: A pilot study on deep learning for hypoglycemic events detection based on ECG. Sci. Rep., 10.10.1038/s41598-019-56927-5 ↗
21. End-to-end deep learning approach for Parkinson’s disease detection from speech signals10.1016/j.bbe.2022.04.002 ↗
22. Rios-Urrego, C.D., Moreno-Acevedo, S.A., Nöth, E., and Orozco-Arroyave, J.R. (2022). End-to-end Parkinson’s disease detection using a deep convolutional recurrent network. International Conference on Text, Speech, and Dialogue, Springer International Publishing.10.1007/978-3-031-16270-1_27 ↗
23. Parkinson Classification Neural Network with Mass Algorithm for Processing Speech Signals10.1007/s00521-024-09596-z ↗
24. Robust Automated Parkinson Disease Detection Based on Voice Signals with Transfer Learning10.1016/j.eswa.2021.115013 ↗
25. Transfer Learning Helps to Improve the Accuracy to Classify Patients with Different Speech Disorders in Different Languages10.1016/j.patrec.2021.04.011 ↗
26. Garg, D., Rodrigues, J.J.P.C., Gupta, S.K., Cheng, X., Sarao, P., and Patel, G.S. (2024). Transfer Learning Approach for Differentiating Parkinson’s Syndromes Using Voice Recordings. Advanced Computing, Springer.10.1007/978-3-031-56703-2 ↗
27. A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness10.1561/116.00000084 ↗
28. Detecting Parkinson Disease Using a Web-Based Speech Task: Observational Study10.2196/26305 ↗
29. Moher, D., Liberati, A., Tetzlaff, J., and Altman, D.G.T.G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med., 6.10.1371/journal.pmed.1000097 ↗
30. The detection of Parkinson’s disease from speech using voice source information10.1109/TASLP.2021.3078364 ↗
31. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NE, USA.
32. Er, M.B., Isik, E., and Isik, I. (2021). Parkinson’s Detection Based on Combined CNN and LSTM Using Enhanced Speech Signals with Variational Mode Decomposition. Biomed. Signal Process. Control, 70.10.1016/j.bspc.2021.103006 ↗
33. Bhati, S., Velazquez, L.M., Villalba, J., and Dehak, N. (2019, January 11–14). LSTM Siamese Network for Parkinson’s Disease Detection from Speech. Proceedings of the 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Ottawa, ON, Canada.10.1109/GlobalSIP45357.2019.8969430 ↗
34. Long Short-Term Memory10.1162/neco.1997.9.8.1735 ↗
35. Khaskhoussy, R., and Ayed, Y.B. (2021, January 14–16). Detecting Parkinson’s Disease According to Gender Using Speech Signals. Proceedings of the Knowledge Science, Engineering and Management: 14th International Conference, KSEM 2021, Tokyo, Japan. Proceedings, Part III.10.1007/978-3-030-82153-1_34 ↗
36. Orozco-Arroyave, J.R., Arias-Londoño, J.D., Vargas-Bonilla, J.F., Gonzalez-Rátiva, M.C., and Nöth, E. (2014). New Spanish speech corpus database for the analysis of people suffering from Parkinson’s disease. LREC, European Language Resources Association (ELRA).
37. Grad-CAM: Visual explanations from deep networks via gradient-based localization10.1007/s11263-019-01228-7 ↗
38. Using AUC and accuracy in evaluating learning algorithms10.1109/TKDE.2005.50 ↗
39. Multi-variate vocal data analysis for detection of Parkinson disease using deep learning10.1007/s00521-020-05233-7 ↗
40. CNN and LSTM for the classification of parkinson’s disease based on the GTCC and MFCC10.35784/acs-2023-11 ↗
41. Faragó, P., Ștefănigă, S.A., Cordoș, C.G., Mihăilă, L.I., Hintea, S., Peștean, A.S., and Ileșan, R.R. (2023). CNN-Based Identification of Parkinson’s Disease from Continuous Speech in Noisy Environments. Bioengineering, 10.10.3390/bioengineering10050531 ↗
42. Diagnosing Parkinson’s disease with speech signal based on convolutional neural network10.1504/IJCAT.2020.110415 ↗
43. Sarlas, A., Kalafatelis, A., Alexandridis, G., Kourtis, M.A., and Trakadas, P. (September, January 29). Exploring Federated Learning for Speech-Based Parkinson’s Disease Detection. Proceedings of the 18th International Conference on Availability, Reliability and Security, Benevento, Italy.
44. Janbakhshi, P., and Kodrasi, I. (October, January 29). Supervised Speech Representation Learning for Parkinson’s Disease Classification. Proceedings of the Speech Communication; 14th ITG Conference, Online.
45. Sakar, C., Serbes, G., Gunduz, A., Nizam, H., and Sakar, B. Parkinson’s Disease Classification. UC Irvine Machine Learning Repository., 2018.
46. Hlavnička, J., Čmejla, R., Klempíř, J., Růžička, E., and Rusz, J. (Figshare, 2019). Synthetic Vowels of Speakers with Parkinson’s Disease and Parkinsonism [Dataset], Figshare.
47. Mallela, J., Illa, A., Suhas, B.N., Udupa, S., Belur, Y., Atchayaram, N., and Ghosh, P.K. (2020, January 4–8). Voice Based Classification of Patients with Amyotrophic Lateral Sclerosis, Parkinson’s Disease and Healthy Controls with CNN-LSTM Using Transfer Learning. Proceedings of the ICASSP 2020—2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.10.1109/ICASSP40776.2020.9053682 ↗
48. Raw Speech Waveform Based Classification of Patients with ALS, Parkinson’s Disease and Healthy Controls Using CNN-BLSTM
49. Attention Is All You Need
50. Chronowski, M., Klaczynski, M., Dec-Cwiek, M., and Porebska, K. (2022). Parkinson’s disease diagnostics using AI and natural language knowledge transfer. arXiv.
51. Malekroodi, H.S., Madusanka, N., Lee, B.I., and Yi, M. (2024). Leveraging Deep Learning for Fine-Grained Categorization of Parkinson’s Disease Progression Levels Through Analysis of Vocal Acoustic Patterns. Bioengineering, 11.10.3390/bioengineering11030295 ↗
52. Hemmerling, D., Wodzinski, M., Orozco-Arroyave, J.R., Sztaho, D., Daniol, M., Jemiolo, P., and Wojcik-Pedziwiatr, M. (2023, January 24–27). Vision Transformer for Parkinson’s Disease Classification Using Multilingual Sustained Vowel Recordings. Proceedings of the 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Sydney, Australia.10.1109/EMBC40787.2023.10340478 ↗
53. Nijhawan, R., Kumar, M., Arya, S., Mendirtta, N., Kumar, S., Towfek, S.K., and Abdelhamid, A.A. (2023). A Novel Artificial-Intelligence-Based Approach for Classification of Parkinson’s Disease Using Complex and Large Vocal Features. Biomimetics, 8.10.3390/biomimetics8040351 ↗
54. wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
55. Peng, X., Xu, H., Liu, J., Wang, J., and He, C. (2023). Voice Disorder Classification Using Convolutional Neural Network Based on Deep Transfer Learning. Sci. Rep., 13.10.1038/s41598-023-34461-9 ↗
56. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). Imagenet: A Large-Scale Hierarchical Image Database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
57. On the Inter-Dataset Generalization of Machine Learning Approaches to Parkinson’s Disease Detection from Voice10.1016/j.ijmedinf.2023.105237 ↗
58. Barry, W.J., and Putzer, M. (2024, July 13). Saarbruecken Voice Database. Available online: https://stimmdb.coli.uni-saarland.de/help_en.php4.
59. Venegas, D. (2024, July 13). Vowels Dataset. Available online: https://www.kaggle.com/datasets/darubiano57/dataset-of-vowels.
60. Imprecise vowel articulation as a potential early marker of Parkinson’s disease: Effect of speaking task10.1121/1.4816541 ↗
61. Assessment of Speech Intelligibility in Parkinson’s Disease Using a Speech-To-Text System10.1109/ACCESS.2017.2762475 ↗
62. Viswanathan, R., Khojasteh, P., Aliahmad, B., Arjunan, S.P., Ragnav, S., Kempster, P., Wong, K., Nagao, J., and Kumar, D. (2018, January 28–30). Efficiency of voice features based on consonant for detection of Parkinson’s disease. Proceedings of the 2018 IEEE Life Sciences Conference (LSC), Montreal, QC, Canada.10.1109/LSC.2018.8572266 ↗
63. Vasquez-Correa, J.C., Arias-Vergara, T., Rios-Urrego, C.D., Schuster, M., Rusz, J., Orozco-Arroyave, J.R., and Nöth, E. (2019, January 28–31). Convolutional Neural Networks and a Transfer Learning Strategy to Classify Parkinson’s Disease from Speech in Three Different Languages. Proceedings of the Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications: 24th Iberoamerican Congress, CIARP 2019, Havana, Cuba. Proceedings 24.10.1007/978-3-030-33904-3_66 ↗
64. Automatic Detection of Parkinson’s Disease in Running Speech Spoken in Three Different Languages10.1121/1.4939739 ↗
65. Arasteh, S.T., Rios-Urrego, C.D., Noeth, E., Maier, A., Yang, S.H., Rusz, J., and Orozco-Arroyave, J.R. (2023). Federated Learning for Secure Development of AI Models for Parkinson’s Disease Detection Using Speech from Different Languages. arXiv.
66. Karan, B., Sahu, S.S., and Mahto, K. (2020, January 10–12). Stacked auto-encoder based Time-frequency features of Speech signal for Parkinson disease prediction. Proceedings of the 2020 International Conference on Artificial Intelligence and Signal Processing (AISP), Amaravati, India.10.1109/AISP48273.2020.9073595 ↗
67. Mendes-Laureano, J., Gómez-García, J.A., Guerrero-López, A., Luque-Buzo, E., Arias-Londoño, J.D., Grandas-Pérez, F.J., and Godino-Llorente, J.I. (2024). NeuroVoz: A Castillian Spanish corpus of parkinsonian speech [Dataset]. Zenodo.
68. Ma, J., Zhang, Y., Li, Y., Zhou, L., Qin, L., Zeng, Y., and Lei, Y. (2021). Deep Dual-Side Learning Ensemble Model for Parkinson Speech Recognition. Biomed. Signal Process. Control, 69.10.1016/j.bspc.2021.102849 ↗
69. Tsanas, A. (UCI Machine Learning Repository, 2014). LSVT Voice Rehabilitation [Dataset], UCI Machine Learning Repository.
70. Parkinson’s Disease Detection Based on Running Speech Data from Phone Calls10.1109/TBME.2021.3116935 ↗
71. Fully automated assessment of the severity of Parkinson’s disease from speech10.1016/j.csl.2013.12.001 ↗
72. Collection and analysis of a Parkinson speech dataset with multiple types of sound recordings10.1109/JBHI.2013.2245674 ↗
73. Automatic estimation of parkinson’s disease severity from diverse speech tasks
74. Considering sex and gender in Alzheimer disease and other dementias10.31887/DCNS.2016.18.4/cepperson ↗
75. Gender differences in Parkinson’s disease: Clinical characteristics and cognition10.1002/mds.23388 ↗
76. Sex differences in Parkinson’s disease10.1016/j.yfrne.2014.02.002 ↗
77. Attention Guided CAM: Visual Explanations of Vision Transformer Guided by Self-Attention
78. Abnar, S., and Zuidema, W. (2020). Quantifying attention flow in transformers. arXiv.10.18653/v1/2020.acl-main.385 ↗
79. Application of explainable artificial intelligence in medical health: A systematic review of interpretability methods10.1016/j.imu.2023.101286 ↗
80. An analysis of explainability methods for convolutional neural networks10.1016/j.engappai.2022.105606 ↗
81. Paissan, F., Ravanelli, M., and Subakan, C. (2024). Listenable Maps for Audio Classifiers. arXiv.
82. A unified approach to interpreting model predictions
83. A German Database Of Patterns Of Pathological Vocal Fold Vibration
84. Kursun, O., Sakar, B., Isenkul, M., Sakar, C., Sertbas, A., and Gurgen, F. Parkinson’s Speech with Multiple Types of Sound Recordings [Dataset], UCI Machine Learning Repository.
85. Dimauro, G., and Girardi, F. (IEEE Dataport, 2019). Italian Parkinson’s Voice and Speech [Dataset], IEEE Dataport.
86. A comparative analysis of speech signal processing algorithms for Parkinson’s disease classification and the use of the tunable Q-factor wavelet transform10.1016/j.asoc.2018.10.022 ↗
87. Acoustic Tracking of Pitch, Modal, and Subharmonic Vibrations of Vocal Folds in Parkinson’s Disease and Parkinsonism10.1109/ACCESS.2019.2945874 ↗
88. Detecting Effect of Levodopa in Parkinson’s Disease Patients Using Sustained Phonemes10.1109/JTEHM.2021.3066800 ↗
89. Kumar, D., Kempster, P., Raghav, S., Viswanthan, R., Zham, P., and Arjunan, S. (2020). Screening Parkinson’s Diseases Using Sustained Phonemes, RMIT University.
90. Mendes-Laureano, J., Gómez-García, J.A., Guerrero-López, A., Luque-Buzo, E., Arias-Londoño, J.D., Grandas-Pérez, F.J., and Godino-Llorente, J.I. (2024). NeuroVoz: A Castillian Spanish corpus of Parkinsonian Speech. arXiv.
91. Mahmood, A., Mehroz Khan, M., Imran, M., Alhajlah, O., Dhahri, H., and Karamat, T. (2023). End-to-end deep learning method for detection of invasive Parkinson’s disease. Diagnostics, 13.10.3390/diagnostics13061088 ↗