Real-Time Machine Learning for Human Activities Recognition Based on Wrist-Worn Wearable Devices
Source abstract
Abstract
Wearable technologies have slowly invaded our lives and can easily help with our day-to-day tasks. One area where wearable devices can shine is in human activity recognition, as they can gather sensor data in a non-intrusive way. We describe a real-time activity recognition system based on a common wearable device: a smartwatch. This is one of the most inconspicuous devices suitable for activity recognition as it is very common and worn for extensive periods of time. We propose a human activity recognition system that is extensible, due to the wide range of sensing devices that can be integrated, and that provides a flexible deployment system. The machine learning component recognizes activity based on plot images generated from raw sensor data. This service is exposed as a Web API that can be deployed locally or directly in the cloud. The proposed system aims to simplify the human activity recognition process by exposing such capabilities via a web API. This web API can be consumed by small-network-enabled wearable devices, even with basic processing capabilities, by leveraging a simple data contract interface and using raw data. The system replaces extensive pre-processing by leveraging high performance image recognition based on plot images generated from raw sensor data. We have managed to obtain an activity recognition rate of 94.89% and to implement a fully functional real-time human activity recognition system.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Context-Aware Activity Recognition Systems · IoT and Edge/Fog Computing · Non-Invasive Vital Sign Monitoring
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
North University Center of Baia Mare, Technical University of Cluj-Napoca, 400114 Cluj-Napoca, Romania
Faculty of Informatics, University of Debrecen, 4032 Debrecen, Hungary
References (42)
Show source-provided references
1. Peppas, K., Tsolakis, A.C., Krinidis, S., and Tzovaras, D. (2020). Real-Time Physical Activity Recognition on Smart Mobile Devices Using Convolutional Neural Networks. Appl. Sci., 10.10.3390/app10238482 ↗
2. Kolkar, R., and Geetha, V. (2021, January 20–21). Human Activity Recognition in Smart Home using Deep Learning Techniques. Proceedings of the 2021 13th International Conference on Information & Communication Technology and System (ICTS), Surabaya, Indonesia.10.1109/ICTS52701.2021.9609044 ↗
3. Deep learning of smartphone sensor data for personal health assistance10.1016/j.mejo.2018.01.015 ↗
4. Sensor-based and vision-based human activity recognition: A comprehensive survey10.1016/j.patcog.2020.107561 ↗
5. Kim, B., Kim, S., Lee, M., Chang, H., Park, E., and Han, T. (2022). Application of an Internet of Medical Things (IoMT) to Communications in a Hospital Environment. Appl. Sci., 12.10.3390/app122312042 ↗
6. Human activity recognition based on hybrid learning algorithm for wearable sensor data
7. A hybrid DL with the Internet of Things to monitor human activities using wearable sensors
8. Using recurrent neural networks for continuous authentication through gait analysis10.1016/j.patrec.2021.03.010 ↗
9. Machine Learning Based Activity Recognition To Identify Wasteful Activities In Production10.1016/j.promfg.2020.04.090 ↗
10. Human activity recognition based on smartphone and wearable sensors using multiscale DCNN ensemble10.1016/j.neucom.2020.04.151 ↗
11. Prasad, A., Tyagi, A.K., Althobaiti, M.M., Almulihi, A., Mansour, R.F., and Mahmoud, A.M. (2021). Human Activity Recognition Using Cell Phone-Based Accelerometer and Convolutional Neural Network. Appl. Sci., 11.10.3390/app112412099 ↗
12. Weiss, G. (2023, February 01). WISDM Smartphone and Smartwatch Activity and Biometrics Dataset. Available online: https://archive.ics.uci.edu/ml/datasets/WISDM+Smartphone+and+Smartwatch+Activity+and+Biometrics+Dataset+.
13. Real-time human activity recognition from accelerometer data using Convolutional Neural Networks10.1016/j.asoc.2017.09.027 ↗
14. Bragança, H., Colonna, J.G., Oliveira, H.A.B.F., and Souto, E. (2022). How Validation Methodology Influences Human Activity Recognition Mobile Systems. Sensors, 22.10.3390/s22062360 ↗
15. Najeh, H., Lohr, C., and Leduc, B. (2022). Dynamic Segmentation of Sensor Events for Real-Time Human Activity Recognition in a Smart Home Context. Sensors, 22.10.3390/s22145458 ↗
16. Helmi, A.M., Al-qaness, M.A.A., Dahou, A., Damaševičius, R., Krilavičius, T., and Elaziz, M.A. (2021). A Novel Hybrid Gradient-Based Optimizer and Grey Wolf Optimizer Feature Selection Method for Human Activity Recognition Using Smartphone Sensors. Entropy, 23.10.3390/e23081065 ↗
17. Mekruksavanich, S., and Jitpattanakul, A. (2021). Deep Learning Approaches for Continuous Authentication Based on Activity Patterns Using Mobile Sensing. Sensors, 21.10.3390/s21227519 ↗
18. Gul, M.A., Yousaf, M.H., Nawaz, S., Ur Rehman, Z., and Kim, H. (2020). Patient Monitoring by Abnormal Human Activity Recognition Based on CNN Architecture. Electronics, 9.10.3390/electronics9121993 ↗
19. Magdin, M., Benc, J., Koprda, Š., Balogh, Z., and Tuček, D. (2022). Comparison of Multilayer Neural Network Models in Terms of Success of Classifications Based on EmguCV, ML.NET and Tensorflow.NET. Appl. Sci., 12.10.3390/app12083730 ↗
20. Online active learning for human activity recognition from sensory data streams10.1016/j.neucom.2019.08.092 ↗
21. Real-Time Human Activity Recognition Using Conditionally Parametrized Convolutions on Mobile and Wearable Devices10.1109/JSEN.2022.3149337 ↗
22. Cengiz, A.B., Birant, K.U., Cengiz, M., Birant, D., and Baysari, K.T. (2022). Improving the Performance and Explainability of Indoor Human Activity Recognition in the Internet of Things Environment. Symmetry, 14.10.3390/sym14102022 ↗
23. Mekruksavanich, S., Jitpattanakul, A., Youplao, P., and Yupapin, P. (2020). Enhanced Hand-Oriented Activity Recognition Based on Smartwatch Sensor Data Using LSTMs. Symmetry, 12.10.3390/sym12091570 ↗
24. Bragança, H., Colonna, J.G., Lima, W.S., and Souto, E. (2020). A Smartphone Lightweight Method for Human Activity Recognition Based on Information Theory. Sensors, 20.10.3390/s20071856 ↗
25. Farag, M.M. (2022). Matched Filter Interpretation of CNN Classifiers with Application to HAR. Sensors, 22.10.3390/s22208060 ↗
26. Mairittha, N., Mairittha, T., and Inoue, S. (2019). On-Device Deep Learning Inference for Efficient Activity Data Collection. Sensors, 19.10.3390/s19153434 ↗
27. Issa, M.E., Helmi, A.M., Al-Qaness, M.A.A., Dahou, A., Abd Elaziz, M., and Damaševičius, R. (2022). Human Activity Recognition Based on Embedded Sensor Data Fusion for the Internet of Healthcare Things. Healthcare, 10.10.3390/healthcare10061084 ↗
28. Wang, X., and Shang, J. (2023). Human Activity Recognition Based on Two-Channel Residual-Channel Residual–GRU–ECA Module with Two Types of Sensors. Electronics, 12.10.3390/electronics12071622 ↗
29. WiFi-Based Human Activity Recognition Using Attention-Based BiLSTM
30. Smartphone and Smartwatch-Based Biometrics Using Activities of Daily Living10.1109/ACCESS.2019.2940729 ↗
31. Alexan, A., Alexan, A., and Oniga, Ş. (2022, January 16–18). Smart watch activity recognition using plot image analysis. Proceedings of the 2022 IEEE 2nd Conference on Information Technology and Data Science (CITDS), Debrecen, Hungary.10.1109/CITDS54976.2022.9914230 ↗
32. Harden, S.W. (2023, February 01). ScottPlot Library for .NET Source Code. Available online: https://github.com/ScottPlot/ScottPlot.
33. Alexan, A. (2023, February 01). Deployed Web App. Available online: https://mlprocessorwebapi.azure-api.net.
34. Weiss, G. (2023, February 01). WISDM Smartphone and Smartwatch Activity and Biometrics Dataset. Available online: https://archive-beta.ics.uci.edu/dataset/507/wisdm+smartphone+and+smartwatch+activity+and+biometrics+dataset.
35. Harden, S.W. (2023, February 01). ScottPlot Library for .NET. Available online: https://scottplot.net/.
36. Harden, S.W. (2023, February 01). ScottPlot Library for .NET NuGet Package. Available online: https://www.nuget.org/packages/ScottPlot/.
37. NVIDIA (2023, February 01). CUDA Toolkit. Available online: https://developer.nvidia.com/cuda-toolkit.
38. NVIDIA (2023, February 01). NVIDIA cuDNN. Available online: https://developer.nvidia.com/cudnn.
39. Microsoft (2023, February 01). ML.NET Model Builder GPU Support. Available online: https://marketplace.visualstudio.com/items?itemName=MLNET.ModelBuilderGPU2022.
40. Microsoft (2023, February 01). ML.NET Model Builder. Available online: https://marketplace.visualstudio.com/items?itemName=MLNET.ModelBuilder2022.
41. Roeder, L. (2023, February 01). Netron Web App. Available online: https://netron.app/.
42. Alexan, A. (2023, February 01). Source Code. Available online: https://bitbucket.org/alexandruAlexan/publicwisdmsmartwatchhar/src/master/.