Wearable-Based Intelligent Emotion Monitoring in Older Adults during Daily Life Activities
Source abstract
Abstract
We present a system designed to monitor the well-being of older adults during their daily activities. To automatically detect and classify their emotional state, we collect physiological data through a wearable medical sensor. Ground truth data are obtained using a simple smartphone app that provides ecological momentary assessment (EMA), a method for repeatedly sampling people’s current experiences in real time in their natural environments. We are making the resulting dataset publicly available as a benchmark for future comparisons and methods. We are evaluating two feature selection methods to improve classification performance and proposing a feature set that augments and contrasts domain expert knowledge based on time-analysis features. The results demonstrate an improvement in classification accuracy when using the proposed feature selection methods. Furthermore, the feature set we present is better suited for predicting emotional states in a leave-one-day-out experimental setup, as it identifies more patterns.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Emotion and Mood Recognition · Mental Health Research Topics · Digital Mental Health Interventions
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Centre for Applied Autonomous Sensor Systems (AASS), Örebro University, 702 81 Örebro, Sweden
Funding
Wallenberg AI, Autonomous Systems and Software Program (WASP) · Knut and Alice Wallenberg Foundation
References (59)
Show source-provided references
1. United Nations (2021, May 14). Population Division. Available online: https://www.un.org/development/desa/pd/.
2. Guest-Editorial Computer-Based Intelligent Technologies for Improving the Quality of Life10.1109/JBHI.2014.2350651 ↗
3. EUROSTAT (2021, May 12). EUROSTAT Statistics Explained. Available online: https://ec.europa.eu/eurostat/statistics-explained.
4. Depression, self-esteem and anxiety in the elderly: A comparative study
5. World Health Organization (2021, May 14). Mental Health. Available online: https://www.who.int.
6. The effects of social isolation on well-being and life satisfaction during pandemic10.1057/s41599-021-00710-3 ↗
7. McCollam, A., O’Sullivan, C., Mukkala, M., Stengård, E., and Rowe, P. (2016). Mental Health in the EU—Key Facts, Figures, and Activities, European Communities.
8. Mental Health Europe (2021, May 14). Ageing and Mental Health—A Forgotten Matter. Available online: https://www.mhe-sme.org/ageing-and-mental-health-a-forgotten-matter/.
9. Organisation for Economic Co-Operation and Development (2021, May 14). Health Workforce. Available online: https://www.oecd.org/health/health-systems/workforce.htm.
10. Toward machine emotional intelligence: Analysis of affective physiological state10.1109/34.954607 ↗
11. Are they different? Affect, feeling, emotion, sentiment, and opinion detection in text10.1109/TAFFC.2014.2317187 ↗
12. (2022, March 01). Empatica. Available online: https://www.empatica.com.
13. A circumplex model of affect10.1037/h0077714 ↗
14. Bautista-Salinas, D., González, J.R., Méndez, I., and Mozos, O.M. (2019, January 23–27). Monitoring and Prediction of Mood in Elderly People during Daily Life Activities. Proceedings of the 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Berlin, Germany.10.1109/EMBC.2019.8857847 ↗
15. Kosti, R., Alvarez, J.M., Recasens, A., and Lapedriza, A. (2017, January 21–26). Emotion recognition in context. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.212 ↗
16. Prediction of loneliness in older adults using natural language processing: Exploring sex differences in speech10.1016/j.jagp.2020.09.009 ↗
17. Bhakre, S.K., and Bang, A. (2016, January 21–24). Emotion recognition on the basis of audio signal using Naive Bayes classifier. Proceedings of the 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI), Jaipur, India.10.1109/ICACCI.2016.7732408 ↗
18. Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review10.1016/j.inffus.2020.01.011 ↗
19. Personalized machine learning of depressed mood using wearables10.1038/s41398-021-01445-0 ↗
20. A machine learning model for emotion recognition from physiological signals10.1016/j.bspc.2019.101646 ↗
21. Shu, L., Yu, Y., Chen, W., Hua, H., Li, Q., Jin, J., and Xu, X. (2020). Wearable emotion recognition using heart rate data from a smart bracelet. Sensors, 20.10.3390/s20030718 ↗
22. Fernández, A.P., Leenders, C., Aerts, J.M., and Berckmans, D. (2023). Emotional States versus Mental Heart Rate Component Monitored via Wearables. Appl. Sci., 13.10.3390/app13020807 ↗
23. Pollreisz, D., and TaheriNejad, N. (2017, January 11–15). A simple algorithm for emotion recognition, using physiological signals of a smart watch. Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju Island, Republic of Korea.10.1109/EMBC.2017.8037328 ↗
24. Li, R., and Liu, Z. (2020). Stress detection using deep neural networks. BMC Med. Inform. Decis. Mak., 20.10.1186/s12911-020-01299-4 ↗
25. Schmidt, P., Reiss, A., Duerichen, R., Marberger, C., and Van Laerhoven, K. (2018, January 16–20). Introducing wesad, a multimodal dataset for wearable stress and affect detection. Proceedings of the 20th ACM International Conference on Multimodal Interaction, Boulder, CO, USA.10.1145/3242969.3242985 ↗
26. Daher, K., Fuchs, M., Mugellini, E., Lalanne, D., and Abou Khaled, O. (2020, January 27–29). Reduce stress through empathic machine to improve HCI. Proceedings of the International Conference on Human Interaction and Emerging Technologies, Virtual.10.1007/978-3-030-44267-5_35 ↗
27. Multiclass emotion prediction using heart rate and virtual reality stimuli10.1186/s40537-020-00401-x ↗
28. Zhao, B., Wang, Z., Yu, Z., and Guo, B. (2018, January 8–12). EmotionSense: Emotion recognition based on wearable wristband. Proceedings of the 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), Guangzhou, China.10.1109/SmartWorld.2018.00091 ↗
29. Toward Emotion Recognition From Physiological Signals in the Wild: Approaching the Methodological Issues in Real-Life Data Collection10.3389/fpsyg.2020.01111 ↗
30. Stressing the accuracy: Wrist-worn wearable sensor validation over different conditions10.1111/psyp.13441 ↗
31. Real-Life Stress Level Monitoring Using Smart Bands in the Light of Contextual Information10.1109/JSEN.2020.2984644 ↗
32. Monitoring Changes in Depression Severity Using Wearable and Mobile Sensors10.3389/fpsyt.2020.584711 ↗
33. Time Series Feature Extraction on basis of Scalable Hypothesis tests (tsfresh—A Python package)10.1016/j.neucom.2018.03.067 ↗
34. Wu, J., Zhou, T., and Li, T. (2020). Detecting epileptic seizures in EEG signals with complementary ensemble empirical mode decomposition and extreme gradient boosting. Entropy, 22.10.3390/e22020140 ↗
35. Machine learning for automated EEG-based biomarkers of cognitive impairment during Deep Brain Stimulation screening in patients with Parkinson’s Disease10.1016/j.clinph.2021.01.021 ↗
36. Spathis, D., Servia-Rodriguez, S., Farrahi, K., Mascolo, C., and Rentfrow, J. (2019, January 20–23). Passive mobile sensing and psychological traits for large scale mood prediction. Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare, Trento, Italy.10.1145/3329189.3329213 ↗
37. Machine Learning for Real-Time Heart Disease Prediction10.1109/JBHI.2021.3066347 ↗
38. Zangróniz, R., Martínez-Rodrigo, A., Pastor, J.M., López, M.T., and Fernández-Caballero, A. (2017). Electrodermal Activity Sensor for Classification of Calm/Distress Condition. Sensors, 17.10.3390/s17102324 ↗
39. Vandecasteele, K., Lázaro, J., Cleeren, E., Claes, K., Van Paesschen, W., Van Huffel, S., and Hunyadi, B. (2018, January 19–21). Artifact Detection of Wrist Photoplethysmograph Signals. Proceedings of the BIOSIGNALS, 2018, Funchal, Madeira, Portugal.10.5220/0006594301820189 ↗
40. Ecological Momentary Assessment10.1146/annurev.clinpsy.3.022806.091415 ↗
41. Ecological momentary assessment: A research method for studying the daily lives of teachers10.1080/1743727X.2010.484548 ↗
42. Using machine learning and smartphone and smartwatch data to detect emotional states and transitions: Exploratory study10.2196/17818 ↗
43. Giannakaki, K., Giannakakis, G., Farmaki, C., and Sakkalis, V. (2017, January 22–24). Emotional state recognition using advanced machine learning techniques on EEG data. Proceedings of the 2017 IEEE 30th International Symposium on Computer-Based Medical Systems (CBMS), Thessaloniki, Greece.10.1109/CBMS.2017.156 ↗
44. Likamwa, R., Liu, Y., Lane, N.D., and Zhong, L. (2013, January 25–28). MoodScope: Building a Mood Sensor from Smartphone Usage Patterns. Proceedings of the 11th Annual International Conference on Mobile Systems, Applications and Services, Taipei, Taiwan.10.1145/2462456.2483967 ↗
45. Cognition in emotion: Concept and action
46. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
47. Plarre, K., Raij, A., Hossain, S.M., Ali, A.A., Nakajima, M., Al’Absi, M., Ertin, E., Kamarck, T., Kumar, S., and Scott, M. (2011, January 4–6). Continuous inference of psychological stress from sensory measurements collected in the natural environment. Proceedings of the 10th ACM/IEEE International Conference on Information Processing in Sensor Networks, Milano, Italy.
48. Detecting stress during real-world driving tasks using physiological sensors10.1109/TITS.2005.848368 ↗
49. Healey, J., Nachman, L., Subramanian, S., Shahabdeen, J., and Morris, M. (2010, January 17–20). Out of the lab and into the fray: Towards modeling emotion in everyday life. Proceedings of the International Conference on Pervasive Computing, Helsinki, Finland.10.1007/978-3-642-12654-3_10 ↗
50. Christ, M., Kempa-Liehr, A., and Feindt, M. (2016). Distributed and parallel time series feature extraction for industrial big data applications. arXiv.
51. Frijda, N.H., Mesquita, B., Sonnemans, J., and Van Goozen, S. (1991). The Duration of Affective Phenomena or Emotions, Sentiments and Passions, Wiley.
52. Theory of support vector machines
53. The Kolmogorov-Smirnov test for goodness of fit10.1080/01621459.1951.10500769 ↗
54. A new measure of rank correlation10.1093/biomet/30.1-2.81 ↗
55. Schaffernicht, E., and Gross, H.M. (2011, January 14–17). Weighted mutual information for feature selection. Proceedings of the International Conference on Artificial Neural Networks, Espoo, Finland.10.1007/978-3-642-21738-8_24 ↗
56. Estimating mutual information10.1103/PhysRevE.69.066138 ↗
57. Bishop, C.M., and Nasrabadi, N.M. (2006). Pattern Recognition and Machine Learning, Springer.
58. Two-level domain adaptation neural network for EEG-based emotion recognition10.3389/fnhum.2020.605246 ↗
59. Hernandez, J., Morris, R.R., and Picard, R.W. (2011, January 9–12). Call center stress recognition with person-specific models. Proceedings of the Affective Computing and Intelligent Interaction: 4th International Conference, ACII 2011, Memphis, TN, USA.