journal article·2022-10-10Open access

Human Posture Detection Using Image Augmentation and Hyperparameter-Optimized Transfer Learning Algorithms

Roseline Oluwaseun Ogundokun, Rytis Maskeliūnas, Robertas Damaševičius

Applied Sciences· Volume 12· Issue 19

Abstract

With the advancement in pose estimation techniques, human posture detection recently received considerable attention in many applications, including ergonomics and healthcare. When using neural network models, overfitting and poor performance are prevalent issues. Recently, convolutional neural networks (CNNs) were successfully used for human posture recognition from human images due to their superior multiscale high-level visual representations over hand-engineering low-level characteristics. However, calculating millions of parameters in a deep CNN requires a significant number of annotated examples, which prohibits many deep CNNs such as AlexNet and VGG16 from being used on issues with minimal training data. We propose a new three-phase model for decision support that integrates CNN transfer learning, image data augmentation, and hyperparameter optimization (HPO) to address this problem. The model is used as part of a new decision support framework for the optimization of hyperparameters for AlexNet, VGG16, CNN, and multilayer perceptron (MLP) models for accomplishing optimal classification results. The AlexNet and VGG16 transfer learning algorithms with HPO are used for human posture detection, while CNN and Multilayer Perceptron (MLP) were used as standard classifiers for contrast. The HPO methods are essential for machine learning and deep learning algorithms because they directly influence the behaviors of training algorithms and have a major impact on the performance of machine learning and deep learning models. We used an image data augmentation technique to increase the number of images to be used for model training to reduce model overfitting and improve classification performance using the AlexNet, VGG16, CNN, and MLP models. The optimal combination of hyperparameters was found for the four models using a random-based search strategy. The MPII human posture datasets were used to test the proposed approach. The proposed models achieved an accuracy of 91.2% using AlexNet, 90.2% using VGG16, 87.5% using CNN, and 89.9% using MLP. The study is the first HPO study executed on the MPII human pose dataset.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Hand Gesture Recognition Systems · Human Pose and Action Recognition · Advanced Technologies in Various Fields

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Faculty of Informatics, Department of Multimedia Engineering, Kaunas University of Technology, 51368 Kaunas, Lithuania

References (59)

Show source-provided references
1. Deng, L. (2011, January 19–21). An overview of deep-structured learning for information processing. Proceedings of the Asia-Pacific Signal and Information Processing Annual Summit Conference (APSIPA-ASC), Xi’an, China.
2. Dropout: A simple way to prevent neural networks from overfitting
3. Imagenet large-scale visual recognition challenge10.1007/s11263-015-0816-y ↗
4. Imagenet classification with deep convolutional neural networks
5. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
6. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
7. NAS-HRIS: Automatic design and architecture search of neural network for semantic segmentation in remote sensing images
8. Ogundokun, R.O., Maskeliunas, R., Misra, S., and Damaševičius, R. (2022, January 4–7). Improved CNN Based on Batch Normal-ization and Adam Optimizer. Proceedings of the International Conference on Computational Science and Its Applications, Malaga, Spain.10.1007/978-3-031-10548-7_43 ↗
9. A review of deep learning techniques for 2D and 3D human pose estimation10.1016/j.imavis.2021.104282 ↗
10. Human pose estimation and its application to action recognition: A survey10.1016/j.jvcir.2021.103055 ↗
11. Involvement of machine learning tools in healthcare decision making10.1155/2021/6679512 ↗
12. Artificial intelligence applied to support medical decisions for the automatic analysis of echocardiogram images: A systematic review10.1016/j.artmed.2021.102165 ↗
13. Application of radiomics and decision support systems for breast MR differential diagnosis10.1155/2018/7417126 ↗
14. Deep learning aided decision support for pulmonary nodules diagnosing: A review10.21037/jtd.2018.02.57 ↗
15. An automated diagnostic system for heart disease prediction based on χ2 statistical model and optimally configured deep neural network10.1109/ACCESS.2019.2904800 ↗
16. An intelligent and reliable hyperparameter optimization machine learning model for early heart disease assessment using imperative risk attributes10.1155/2022/9882288 ↗
17. Evaluation of hyperparameter optimization in machine and deep learning methods for decoding imagined speech eeg10.3390/s20164629 ↗
18. Understanding the effect of hyperparameter optimization on machine learning models for structure design problems10.1016/j.cad.2021.103013 ↗
19. Chollet, F., and Allaire, J.J. (2018). Deep Learning mit R und Keras: Das Praxis-Handbuch von den Entwicklern von Keras und Rstudio, MITP-Verlags GmbH Co. KG.
20. Elgendy, M. (2020). Deep Learning for Vision Systems, Simon and Schuster.
21. A survey on image data augmentation for deep learning10.1186/s40537-019-0197-0 ↗
22. Semantic segmentation on small datasets of satellite images using convolutional neural networks10.1117/1.JRS.13.046510 ↗
23. Dual sparse learning via data augmentation for robust facial image classification10.1007/s13042-020-01067-w ↗
24. Abayomi-Alli, O.O., Damaševicius, R., Maskeliunas, R., and Misra, S. (2021). Few-shot learning with a novel voronoi tessellation-based image augmentation method for facial palsy detection. Electronics, 10.10.3390/electronics10080978 ↗
25. Cassava disease recognition from low-quality images using enhanced data augmentation model and deep learning10.1111/exsy.12746 ↗
26. Malignant skin melanoma detection using image augmentation by oversampling in nonlinear lower-dimensional embedding manifold10.3906/elk-2101-133 ↗
27. A novel data augmentation convolutional neural network for detecting malaria parasite in blood smear images10.1080/08839514.2022.2033473 ↗
28. Wang, Z., Yang, J., Jiang, H., and Fan, X. (2020). CNN training with twenty samples for crack detection via data augmentation. Sensors, 20.10.3390/s20174849 ↗
29. Hutter, F., Hoos, H., and Leyton-Brown, K. (2014, January 21–26). An efficient approach for assessing hyperparameter importance. Proceedings of the International Conference on Machine Learning, Beijing, China.
30. Hutter, F., Kotthoff, L., and Vanschoren, J. (2019). Automated Machine Learning: Methods, Systems, Challenges, Springer Nature.10.1007/978-3-030-05318-5 ↗
31. A meta-learning recommender system for hyperparameter tuning10.1016/j.ins.2019.06.005 ↗
32. Neary, P. (2018, January 2–7). Automatic hyperparameter tuning in deep convolutional neural networks using asynchronous reinforcement learning. Proceedings of the 2018 IEEE International Conference on Cognitive Computing (ICCC), San Francisco, CA, USA.10.1109/ICCC.2018.00017 ↗
33. Tuning of reinforcement learning parameters applied to sop using the Scott–Knott method10.1007/s00500-019-04206-w ↗
34. Hyperparameter tuning and performance assessment of statistical and machine-learning algorithms using spatial data10.1016/j.ecolmodel.2019.06.002 ↗
35. Hyperparameter tuning deep learning for diabetic retinopathy fundus image classification10.1109/ACCESS.2020.3005152 ↗
36. Pawara, P., Okafor, E., Schomaker, L., and Wiering, M. (2017, January 18–21). Data augmentation for plant classification. Proceedings of the International Conference on Advanced Concepts for Intelligent Vision Systems, Antwerp, Belgium.10.1007/978-3-319-70353-4_52 ↗
37. Analysis of the impact of data augmentation on target recognition for UAV-based transmission line inspection10.1155/2020/3107450 ↗
38. CovidXrayNet: Optimizing data augmentation and CNN hyperparameters for improved COVID-19 detection from CXR10.1016/j.compbiomed.2021.104375 ↗
39. Optimization of SVM parameters for recognition of regulatory DNA sequences10.1007/s11750-010-0152-x ↗
40. Neural network hyperparameter optimization for prediction of real estate prices in helsinki10.7717/peerj-cs.444 ↗
41. A heuristic approach to the hyperparameters in training spiking neural networks using spike-timing-dependent plasticity10.1007/s00521-021-06824-8 ↗
42. Lawal, M.O. (2021). Tomato Detection Based on Modified YOLOv3 Framework, Springer Science and Business Media LLC.10.1038/s41598-021-81216-5 ↗
43. Zhang, K., Robinson, N., Lee, S.-W., and Guan, C. (2021). Adaptive Transfer Learning for EEG Motor Imagery Classification with Deep Convolutional Neural Network, Elsevier BV.10.1016/j.neunet.2020.12.013 ↗
44. Roy, A.M. (2022). Adaptive Transfer Learning-Based Multiscale Feature Fused Deep Convolutional Neural Network for EEG MI Multiclassification in Brain–Computer Interface, Elsevier BV.10.1016/j.engappai.2022.105347 ↗
45. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
46. Deep learning10.1038/nature14539 ↗
47. Andriluka, M., Pishchulin, L., Gehler, P., and Schiele, B. (2014, January 23–28). 2d human pose estimation: New benchmark and state-of-the-art analysis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.471 ↗
48. Pattern recognition by affine moment invariants10.1016/0031-3203(93)90098-H ↗
49. ImageNet Classification with Deep Convolutional Neural Networks10.1145/3065386 ↗
50. Ogundokun, R.O., Misra, S., Douglas, M., Damaševičius, R., and Maskeliūnas, R. (2022). Medical Internet-of-Things Based Breast Cancer Diagnosis Using Hyperparameter-Optimized Neural Networks. Future Internet, 14.10.3390/fi14050153 ↗
51. Montgomery, D.C. (2017). Design and Analysis of Experiments, John Wiley Sons.
52. Multi-task deep learning for real-time 3D human pose estimation and action recognition
53. SimpleCut: A simple and strong 2D model for multi-person pose estimation10.1016/j.cviu.2022.103509 ↗
54. Lightweight human pose estimation: CVC-net10.1007/s11042-022-12245-z ↗
55. Wang, R., Geng, F., and Wang, X. (2022). MTPose: Human pose estimation with high-resolution multi-scale transformers. Neural Process. Lett., 1–24.10.1007/s11063-022-10794-w ↗
56. UULPN: An ultra-lightweight network for human pose estimation based on unbiased data processing10.1016/j.neucom.2021.12.083 ↗
57. Wu, Y., Ma, S., Zhang, D., Huang, W., and Chen, Y. (2022). An improved mixture density network for 3D human pose estimation with ordinal ranking. Sensors, 22.10.3390/s22134987 ↗
58. Lightweight densely connected residual network for human pose estimation10.1007/s11554-020-01025-3 ↗
59. EfficientPose: Efficient human pose estimation with neural architecture search10.1007/s41095-021-0214-z ↗