journal article·2022-05-15Open access

A Hybrid Deep Residual Network for Efficient Transitional Activity Recognition Based on Wearable Sensors

Sakorn Mekruksavanich, Narit Hnoohom, Anuchit Jitpattanakul

Applied Sciences· Volume 12· Issue 10

Abstract

Numerous learning-based techniques for effective human behavior identification have emerged in recent years. These techniques focus only on fundamental human activities, excluding transitional activities due to their infrequent occurrence and short period. Nevertheless, postural transitions play a critical role in implementing a system for recognizing human activity and cannot be ignored. This study aims to present a hybrid deep residual model for transitional activity recognition utilizing signal data from wearable sensors. The developed model enhances the ResNet model with hybrid Squeeze-and-Excitation (SE) residual blocks combining a Bidirectional Gated Recurrent Unit (BiGRU) to extract deep spatio-temporal features hierarchically, and to distinguish transitional activities efficiently. To evaluate recognition performance, the experiments are conducted on two public benchmark datasets (HAPT and MobiAct v2.0). The proposed hybrid approach achieved classification accuracies of 98.03% and 98.92% for the HAPT and MobiAct v2.0 datasets, respectively. Moreover, the outcomes show that the proposed method is superior to the state-of-the-art methods in terms of overall accuracy. To analyze the improvement, we have investigated the effects of combining SE modules and BiGRUs into the deep residual network. The findings indicates that the SE module is efficient in improving transitional activity recognition.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Context-Aware Activity Recognition Systems · Human Pose and Action Recognition · Non-Invasive Vital Sign Monitoring

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Computer Engineering, School of Information and Communication Technology, University of Phayao, Phayao 56000, Thailand

Image Information and Intelligence Laboratory, Department of Computer Engineering, Faculty of Engineering, Mahidol University, Nakhon Pathom 73170, Thailand

Department of Mathematics, Faculty of Applied Science, King Mongkut’s University of Technology North Bangkok, Bangkok 10800, Thailand

Intelligent and Nonlinear Dynamic Innovations Research Center, Science and Technology Research Institute, King Mongkut’s University of Technology North Bangkok, Bangkok 10800, Thailand

Funding

University of Phayao · Thailand Science Research and Innovation Fund

References (56)

Show source-provided references
1. A Hybrid Hierarchical Framework for Gym Physical Activity Recognition and Measurement Using Wearable Sensors10.1109/JIOT.2018.2846359 ↗
2. Online human activity recognition employing hierarchical hidden Markov models10.1007/s12652-019-01380-5 ↗
3. Sensor-based and vision-based human activity recognition: A comprehensive survey10.1016/j.patcog.2020.107561 ↗
4. A Review on Human Activity Recognition Using Vision-Based Method
5. Vitrià, J., Sanches, J.M., and Hernández, M. (2011). Human Activity Recognition from Accelerometer Data Using a Wearable Device. Proceedings of the Pattern Recognition and Image Analysis, Springer.10.1007/978-3-642-21257-4 ↗
6. Vision-based human activity recognition: A survey10.1007/s11042-020-09004-3 ↗
7. Segmentation and Recognition of Basic and Transitional Activities for Continuous Physical Human Activity10.1109/ACCESS.2019.2905575 ↗
8. Ferreira, P.J.S., Cardoso, J.M.P., and Mendes-Moreira, J. (2020). kNN Prototyping Schemes for Embedded Human Activity Recognition with Online Learning. Computers, 9.10.3390/computers9040096 ↗
9. Palaniappan, A., Bhargavi, R., and Vaidehi, V. (2012, January 19–21). Abnormal human activity recognition using SVM based approach. Proceedings of the 2012 International Conference on Recent Trends in Information Technology, Chennai, India.10.1109/ICRTIT.2012.6206829 ↗
10. Fan, L., Wang, Z., and Wang, H. (2013, January 13–15). Human Activity Recognition Model Based on Decision Tree. Proceedings of the 2013 International Conference on Advanced Cloud and Big Data, Nanjing, China.10.1109/CBD.2013.19 ↗
11. Random Forest for Human Daily Activity Recognition10.1088/1742-6596/1655/1/012087 ↗
12. Cheng, L., Guan, Y., Zhu, K., and Li, Y. (2017, January 9–11). Recognition of human activities using machine learning methods with wearable sensors. Proceedings of the 2017 IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA.10.1109/CCWC.2017.7868369 ↗
13. Mekruksavanich, S., and Jitpattanakul, A. (2022). Deep Residual Network for Smartwatch-Based User Identification through Complex Hand Movements. Sensors, 22.10.3390/s22083094 ↗
14. Mekruksavanich, S., and Jitpattanakul, A. (2021). Deep Convolutional Neural Network with RNNs for Complex Activity Recognition Using Wrist-Worn Wearable Sensor Data. Electronics, 10.10.3390/electronics10141685 ↗
15. On Improving the accuracy with Auto-Encoder on Conjunctivitis10.1016/j.asoc.2019.105489 ↗
16. Mekruksavanich, S., and Jitpattanakul, A. (2021). LSTM Networks Using Smartphone Data for Sensor-Based Human Activity Recognition in Smart Homes. Sensors, 21.10.3390/s21051636 ↗
17. Analysis and design of a recurrent neural network for linear programming10.1109/81.244913 ↗
18. Albawi, S., Mohammed, T.A., and Al-Zawi, S. (2017, January 21–23). Understanding of a convolutional neural network. Proceedings of the 2017 International Conference on Engineering and Technology (ICET), Antalya, Turkey.10.1109/ICEngTechnol.2017.8308186 ↗
19. LSTM: A Search Space Odyssey10.1109/TNNLS.2016.2582924 ↗
20. Describing the User Experience of Wearable Fitness Technology through Online Product Reviews10.1177/1541931213601248 ↗
21. Deep Learning Models for Real-time Human Activity Recognition with Smartphones10.1007/s11036-019-01445-x ↗
22. Deep-Learning-Enhanced Human Activity Recognition for Internet of Healthcare Things10.1109/JIOT.2020.2985082 ↗
23. WiFi CSI Based Passive Human Activity Recognition Using Attention Based BLSTM10.1109/TMC.2018.2878233 ↗
24. A Novel Semisupervised Deep Learning Method for Human Activity Recognition10.1109/TII.2018.2889315 ↗
25. InnoHAR: A Deep Neural Network for Complex Human Activity Recognition10.1109/ACCESS.2018.2890675 ↗
26. A Survey on Human Activity Recognition using Wearable Sensors10.1109/SURV.2012.110112.00192 ↗
27. Classification and characterization of postural transitions using instrumented shoes10.1007/s11517-017-1778-8 ↗
28. Acampora, G., Minopoli, G., Musella, F., and Staffa, M. (2020). Classification of Transition Human Activities in IoT Environments via Memory-Based Neural Networks. Electronics, 9.10.3390/electronics9030409 ↗
29. Uddin, M.T., Billah, M.M., and Hossain, M.F. (2016, January 13–14). Random forests based recognition of human activities and postural transitions on smartphone. Proceedings of the 2016 5th International Conference on Informatics, Electronics and Vision (ICIEV), Dhaka, Bangladesh.10.1109/ICIEV.2016.7760005 ↗
30. Adaptive sliding window segmentation for physical activity recognition using a single tri-axial accelerometer10.1016/j.pmcj.2016.09.009 ↗
31. Transition-Aware Human Activity Recognition Using Smartphones10.1016/j.neucom.2015.07.085 ↗
32. Three-layer Activity Recognition Combining Domain Knowledge and Meta- classification Author list10.5405/jmbe.1321 ↗
33. Feature Selection and Activity Recognition System Using a Single Triaxial Accelerometer10.1109/TBME.2014.2307069 ↗
34. Transition-Aware Housekeeping Task Monitoring Using Single Wrist-Worn Sensor10.1109/JSEN.2018.2868278 ↗
35. A robust human activity recognition system using smartphone sensors and deep learning10.1016/j.future.2017.11.029 ↗
36. Gusain, K., Gupta, A., and Popli, B. (2018). Transition-aware human activity recognition using eXtreme gradient boosted decision trees. Advanced Computing and Communication Technologies, Springer.10.1007/978-981-10-4603-2_5 ↗
37. Ahmed, N., Rafiq, J.I., and Islam, M.R. (2020). Enhanced Human Activity Recognition Based on Smartphone Sensor Data Using Hybrid Feature Selection Model. Sensors, 20.10.3390/s20010317 ↗
38. Shi, J., Zuo, D., and Zhang, Z. (2020). Transition Activity Recognition System Based on Standard Deviation Trend Analysis. Sensors, 20.10.3390/s20113117 ↗
39. Setterquist, D. (2018). Using a Smartphone to Detect the Standing-to-Kneeling and Kneeling-to-Standing Postural Transitions. [Master’s Thesis, School of Electrical Engineering and Computer Science (EECS), KTH Royal Institute of Technology].
40. HiHAR: A Hierarchical Hybrid Deep Learning Architecture for Wearable Sensor-Based Human Activity Recognition10.1109/ACCESS.2021.3122298 ↗
41. Wearable sensor-based human activity recognition using hybrid deep learning techniques10.1155/2020/2132138 ↗
42. Irfan, S., Anjum, N., Masood, N., Khattak, A.S., and Ramzan, N. (2021). A Novel Hybrid Deep Learning Model for Human Activity Recognition Based on Transitional Activities. Sensors, 21.10.3390/s21248227 ↗
43. Röcker, C., O’Donoghue, J., Ziefle, M., Helfert, M., and Molloy, W. (2017). Human daily activity and fall recognition using a smartphone’s acceleration sensor. Proceedings of the Information and Communication Technologies for Ageing Well and e-Health, Springer International Publishing.10.1007/978-3-319-62704-5 ↗
44. A Smartphone-Enabled Fall Detection Framework for Elderly People in Connected Home Healthcare10.1109/MNET.001.1900100 ↗
45. Anguita, D., Ghio, A., Oneto, L., Parra, X., and Reyes-Ortiz, J.L. (2013, January 24–26). A Public Domain Dataset for Human Activity Recognition using Smartphones. Proceedings of the International Conference on European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Bruges, Belgium.
46. Rong, L., Jianzhong, Z., Ming, L., and Xiangfeng, H. (2007, January 23–25). A Wearable Acceleration Sensor System for Gait Recognition. Proceedings of the 2007 2nd IEEE Conference on Industrial Electronics and Applications, Harbin, China.10.1109/ICIEA.2007.4318894 ↗
47. Window Size Impact in Human Activity Recognition10.3390/s140406474 ↗
48. Chung, J., Gulcehre, C., Cho, K., and Bengio, Y. (2014, January 8–13). Empirical evaluation of gated recurrent neural networks on sequence modeling. Proceedings of the NIPS 2014 Workshop on Deep Learning, Montreal, QC, Canada.
49. Bidirectional recurrent neural networks10.1109/78.650093 ↗
50. HRAN: Hybrid Residual Attention Network for Single Image Super-Resolution10.1109/ACCESS.2019.2942346 ↗
51. Lin, M., Chen, Q., and Yan, S. (2014). Network in Network. arXiv.
52. USE-Net: Incorporating Squeeze-and-Excitation blocks into U-Net for prostate zonal segmentation of multi-institutional MRI datasets10.1016/j.neucom.2019.07.006 ↗
53. Kingma, D., and Ba, J. (2014, January 14–16). Adam: A method for stochastic optimization. Proceedings of the International Conference on Learning Representations, Banff, AB, Canada.
54. Janocha, K., and Czarnecki, W. (2017). On Loss Functions for Deep Neural Networks in Classification. arXiv.10.4467/20838476SI.16.004.6185 ↗
55. Xie, S., Girshick, R., Dollár, P., Tu, Z., and He, K. (2017, January 21–26). Aggregated Residual Transformations for Deep Neural Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.634 ↗
56. LSTM-CNN Architecture for Human Activity Recognition10.1109/ACCESS.2020.2982225 ↗