journal article·2024-12-26Open access

Resilience of Machine Learning Models in Anxiety Detection: Assessing the Impact of Gaussian Noise on Wearable Sensors

Abdulrahman Alkurdi, Jean Clore, Richard Sowers, Elizabeth T. Hsiao-Wecksler, Manuel E. Hernandez

Applied Sciences· Volume 15· Issue 1

Abstract

The resilience of machine learning models for anxiety detection through wearable technology was explored. The effectiveness of feature-based and end-to-end machine learning models for anxiety detection was evaluated under varying conditions of Gaussian noise. By adding synthetic Gaussian noise to a well-known open access affective states dataset collected with commercially available wearable devices (WESAD), a performance baseline was established using the original dataset. This was followed by an examination of the impact of noise on model accuracy to better understand model performance (F1-score and Accuracy) changes as a function of noise. The results of the analysis revealed that with the increase in noise, the performance of feature-based models dropped from a high of 90% F1-score and 92% accuracy to 65% and 70%, respectively; while end-to-end models showed a decrease from an 85% F1-score and 87% accuracy to below 60% and 65%, respectively. This indicated a proportional decline in performance across both feature-based and end-to-end models as noise levels increased, challenging initial assumptions about model resilience. This analysis highlights the need for more robust algorithms capable of maintaining accuracy in noisy, real-world environments and emphasizes the importance of considering environmental factors in the development of wearable anxiety detection systems.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Emotion and Mood Recognition · Mental Health Research Topics · Heart Rate Variability and Autonomic Control

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Mechanical Science & Engineering, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA

Department of Psychiatry and Behavioral Medicine, University of Illinois College of Medicine at Peoria, Peoria, IL 61605, USA

Department of Industrial & Enterprise Systems Engineering, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA

Department of Biomedical and Translational Sciences, Carle Illinois College of Medicine, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA

Neuroscience Program, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA

Department of Health and Kinesiology, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA

Department of Bioengineering, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA

Beckman Institute, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA

Funding

Jump ARCHES endowment through the Health Care Engineering Systems Center · ACCESS MATCH program made possible by the U.S. National Science Foundation

References (57)

Show source-provided references
1. Grand Challenges in Global Mental Health10.1038/475027a ↗
2. Design and Field Procedures in the US National Comorbidity Survey Replication Adolescent Supplement (NCS-A)10.1002/mpr.279 ↗
3. Prevalence of DSM-5 Anxiety Disorders, Comorbidity, and Persistence of Symptoms in Spanish Early Adolescents10.1007/s00787-018-1207-z ↗
4. The Size and Burden of Mental Disorders and Other Disorders of the Brain in Europe 201010.1016/j.euroneuro.2011.07.018 ↗
5. Detecting Stress during Real-World Driving Tasks Using Physiological Sensors10.1109/TITS.2005.848368 ↗
6. Elgendi, M., Galli, V., Ahmadizadeh, C., and Menon, C. (2022). Dataset of Psychological Scales and Physiological Signals Collected for Anxiety Assessment Using a Portable Device. Data, 7.10.3390/data7090132 ↗
7. Haouij, N.E., Poggi, J.M., Sevestre-Ghalila, S., Ghozi, R., and Jadane, M. (2018, January 9–13). AffectiveROAD System and Database to Assess Driver’s Attention. Proceedings of the ACM Symposium on Applied Computing, New York, NY, USA.10.1145/3167132.3167395 ↗
8. Schmidt, P., Reiss, A., Duerichen, R., and Laerhoven, K. (2018, January 16–20). Van Introducing WeSAD, a Multimodal Dataset for Wearable Stress and Affect Detection. Proceedings of the ICMI 2018—Proceedings of the 2018 International Conference on Multimodal Interaction, Boulder, CO, USA.10.1145/3242969.3242985 ↗
9. Stretchable On-Skin Touchless Screen Sensor Enabled by Ionic Hydrogel10.1007/s12274-023-6365-8 ↗
10. Stretchable Piezoelectric Biocrystal Thin Films10.1038/s41467-023-42184-8 ↗
11. Recent Advances in Smart Wearable Sensors for Continuous Human Health Monitoring10.1016/j.talanta.2024.125817 ↗
12. A Review on Flexible Wearables-Recent Developments in Non-Invasive Continuous Health Monitoring10.1016/j.sna.2023.114993 ↗
13. Advanced and Personalized Healthcare through Integrated Wearable Sensors (Versatile)10.1039/D3MA00657C ↗
14. Machine Learning, Deep Learning, and Data Preprocessing Techniques for Detecting, Predicting, and Monitoring Stress and Stress-Related Mental Disorders: Scoping Review10.2196/53714 ↗
15. Integration of Artificial Intelligence and Wearable Internet of Things for Mental Health Detection
16. Gomes, N., Pato, M., Lourenco, A.R., and Datia, N. (2023). A Survey on Wearable Sensors for Mental Health Monitoring. Sensors, 23.10.3390/s23031330 ↗
17. Spielberger, C.D. (1966). Theory and Research on Anxiety, Academic Press Inc.10.1016/B978-1-4832-3131-0.50006-8 ↗
18. Notes and Comments Trait-State Anxiety and Motor Behavior10.1080/00222895.1971.10734907 ↗
19. Hackfort, D., and Spielberger, C.D. (2021). Sport-Related Anxiety: Current Trends in Theory and Research, Academic Press Inc.10.4324/9781315781594-21 ↗
20. The Continuing Shortage of Child and Adolescent Psychiatrists10.1097/01.chi.0000225353.16831.5d ↗
21. County-Level Estimates of Mental Health Professional Shortage in the United States10.1176/ps.2009.60.10.1323 ↗
22. Child and Adolescent Psychiatry Workforce: A Critical Shortage and National Challenge10.1176/appi.ap.27.4.277 ↗
23. Projected Workforce of Psychiatrists in the United States: A Population Analysis10.1176/appi.ps.201700344 ↗
24. Psychological Stress and the Human Immune System: A Meta-Analytic Study of 30 Years of Inquiry10.1037/0033-2909.130.4.601 ↗
25. Effects of Work Stress on Ambulatory Blood Pressure, Heart Rate, and Heart Rate Variability10.1161/01.HYP.35.4.880 ↗
26. Anxiety Disorders and Cardiovascular Disease10.1007/s11920-016-0739-5 ↗
27. An Analysis of Mental Workload in Pilots During Flight Using Multiple Psychophysiological Measures10.1207/S15327108IJAP1201_2 ↗
28. Information Bias in Health Research: Definition, Pitfalls, and Adjustment Methods10.2147/JMDH.S104807 ↗
29. Measures of Anxiety10.1002/acr.20561 ↗
30. Ecological Momentary Assessment10.1146/annurev.clinpsy.3.022806.091415 ↗
31. Relative Roles of the Sympathetic and Parasympathetic Nervous Systems in the Reflex Control of Heart Rate10.1161/01.RES.16.4.363 ↗
32. Impaired Cardiovascular Recovery Following Stress Predicts 3-Year Increases in Blood Pressure10.1097/01.hjh.0000160208.66405.a8 ↗
33. Psychophysiological Stress and Emg Activity of the Trapezius Muscle10.1207/s15327558ijbm0104_5 ↗
34. Waxenbaum, J.A., and Varacallo, M. (2024, December 24). Anatomy, Autonomic Nervous System, StatPearls [Internet], Available online: https://pubmed.ncbi.nlm.nih.gov/30969667/.
35. Study of the Stress Response: Role of Anxiety, Cortisol and DHEAs10.1177/107385840200800209 ↗
36. The “Trier Social Stress Test”—A Tool for Investigating Psychobiological Stress Responses in a Laboratory Setting10.1159/000119004 ↗
37. DEAP: A Database for Emotion Analysis; Using Physiological Signals10.1109/T-AFFC.2011.15 ↗
38. The Cold Pressor Test and Autonomic Function: A Review and Integration10.1111/j.1469-8986.1975.tb01289.x ↗
39. Dziezyc, M., Gjoreski, M., Kazienko, P., Saganowski, S., and Gams, M. (2020). Can We Ditch Feature Engineering? End-to-End Deep Learning for Affect Recognition from Physiological Sensor Data. Sensors, 20.10.3390/s20226535 ↗
40. Ancillon, L., Elgendi, M., and Menon, C. (2022). Machine Learning for Anxiety Detection Using Biosignals: A Review. Diagnostics, 12.10.3390/diagnostics12081794 ↗
41. Review on Psychological Stress Detection Using Biosignals10.1109/TAFFC.2019.2927337 ↗
42. Autonomic Nervous System Activity in Emotion: A Review10.1016/j.biopsycho.2010.03.010 ↗
43. Machine Learning in Mental Health: A Scoping Review of Methods and Applications10.1017/S0033291719000151 ↗
44. Development and Validation of Brief Measures of Positive and Negative Affect: The PANAS Scales10.1037/0022-3514.54.6.1063 ↗
45. The State-Trait Anxiety Inventory
46. Validation of a Short Stress State Questionnaire10.1177/154193120404801107 ↗
47. Eliciting Positive, Negative and Mixed Emotional States: A Film Library for Affective Scientists10.1080/02699931.2015.1031089 ↗
48. An Efficient Algorithm for Automatic Peak Detection in Noisy Periodic and Quasi-Periodic Signals10.3390/a5040588 ↗
49. Chen, T., and Guestrin, C. (2016, January 13–17). XGBoost. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.10.1145/2939672.2939785 ↗
50. Scikit-Learn: Machine Learning in Python
51. Representation Learning: A Review and New Perspectives10.1109/TPAMI.2013.50 ↗
52. Multi-Class Pattern Classification Using Neural Networks10.1016/j.patcog.2006.04.041 ↗
53. In Defense of One-vs-All Classification
54. A Comparison of Methods for Multiclass Support Vector Machines10.1109/72.991427 ↗
55. Zaman, M.S., and Morshed, B.I. (2020, January 20–24). Estimating Reliability of Signal Quality of Physiological Data from Data Statistics Itself for Real-Time Wearables. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada.10.1109/EMBC44109.2020.9175317 ↗
56. Wearable Artificial Intelligence for Detecting Anxiety: Systematic Review and Meta-Analysis10.2196/48754 ↗
57. Efficient KNN Classification with Different Numbers of Nearest Neighbors10.1109/TNNLS.2017.2673241 ↗