Enhancing Human Key Point Identification: A Comparative Study of the High-Resolution VICON Dataset and COCO Dataset Using BPNET
Source abstract
Abstract
Accurately identifying human key points is crucial for various applications, including activity recognition, pose estimation, and gait analysis. This study introduces a high-resolution dataset formed via the VICON motion capture system and three diverse 2D cameras. It facilitates the training of neural networks to estimate 2D key joint positions from images and videos. The study involved 25 healthy adults (17 males, 8 females), executing normal gait for 2 to 3 s. The VICON system captured 3D ground truth data, while the three 2D cameras collected images from different perspectives (0°, 45°, and 135°). The dataset was used to train the Body Pose Network (BPNET), a popular neural network model developed by NVIDIA TAO. Additionally, a comparison entails another BPNET model trained on the COCO 2017 dataset, featuring over 118,000 annotated images. Notably, the proposed dataset exhibited a higher level of accuracy (14.5%) than COCO 2017, despite comprising one-fourth of the image count (23,741 annotated image). This substantial reduction in data size translates to improvements in computational efficiency during model training. Furthermore, the unique dataset’s emphasis on gait and precise prediction of key joint positions during normal gait movements distinguish it from existing alternatives. This study has implications ranging from gait-based person identification, and non-invasive concussion detection through sports temporal analysis, to pathologic gait pattern identification.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Gait Recognition and Analysis · Video Surveillance and Tracking Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Engineering, Grand Valley State University, Grand Rapids, MI 49503, USA
Department of Physical Therapy and Athletic Training, Grand Valley State University, Grand Rapids, MI 49503, USA
Department of Electrical and Computer Engineering, University of Michigan, Dearborn, MI 48128, USA
Funding
ADVANCE Grant Program · Grand Valley State University, the Center for Scholarly and Creative Excellence (CSCE), Research Grant
References (52)
Show source-provided references
1. Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A. (2011, January 21–23). Real-time human pose recognition in parts from single depth images. Proceedings of the CVPR 2011, Colorado Springs, CO, USA.10.1109/CVPR.2011.5995316 ↗
2. Gait Analysis in Rehabilitation Medicine: A Brief Report: 110.1097/00002060-199905000-00019 ↗
3. A Review of the Evolution of Vision-Based Motion Analysis and the Integration of Advanced Computer Vision Methods Towards Developing a Markerless System10.1186/s40798-018-0139-y ↗
4. Zheng, C., Wu, W., Chen, C., Yang, T., Zhu, S., Shen, J., Kehtarnavaz, N., and Shah, M. (2022). Deep Learning-Based Human Pose Estimation: A Survey. arXiv, Available online: http://arxiv.org/abs/2012.13392.
5. Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., and Sheikh, Y. (2019). OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. arXiv, Available online: http://arxiv.org/abs/1812.08008.
6. Güler, R.A., Neverova, N., and Kokkinos, I. (2018, January 18–22). DensePose: Dense Human Pose Estimation in The Wild. Proceedings of the CVPR 2018, Salt Lake City, UT, USA.10.1109/CVPR.2018.00762 ↗
7. AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time10.1109/TPAMI.2022.3222784 ↗
8. Sun, K., Geng, Z., Meng, D., Xiao, B., Liu, D., Zhang, Z., and Wang, J. (2020). Bottom-Up Human Pose Estimation by Ranking Heatmap-Guided Adaptive Keypoint Estimates. arXiv, Available online: http://arxiv.org/abs/2006.15480.
9. Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: A survey10.1007/s10462-018-09679-z ↗
10. (2022, May 10). Vicon Nexus 2.14. Vicon Motion Systems Ltd. Available online: https://www.vicon.com/software/nexus.
11. (2022, May 10). Nvidia Tao Toolkit (Bpnet). NVIDIA Corporation. Available online: https://developer.nvidia.com/tao-toolkit.
12. (2022, May 10). NVIDIA Corporation. NVIDIA TAO Toolkit. Available online: https://developer.nvidia.com/tao-toolkit.
13. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft COCO: Common Objects in Context. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
14. Sun, K., Xiao, B., Liu, D., and Wang, J. (2019, January 15–20). Deep High-Resolution Representation Learning for Human Pose Estimation. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.00584 ↗
15. Influence of a Marker-Based Motion Capture System on the Performance of Microsoft Kinect v2 Skeleton Algorithm10.1109/JSEN.2018.2876624 ↗
16. Gaertner, S., Do, M., Asfour, T., Dillmann, R., Simonidis, C., and Seemann, W. (2010, January 7–9). Generation of Human-like Motion for Humanoid Robots Based on Marker-based Motion Capture Data. Proceedings of the ISR 2010 (41st International Symposium on Robotics) and ROBOTIK 2010 (6th German Conference on Robotics), Munich, Germany.
17. Wirth, M.A., Fischer, G., Verdú, J., Reissner, L., Balocco, S., and Calcagni, M. (2019). Comparison of a New Inertial Sensor Based System with an Optoelectronic Motion Capture System for Motion Analysis of Healthy Human Wrist Joints. Sensors, 19.10.3390/s19235297 ↗
18. Kadam, R., and Pawar, S.N. (2020, January 11–13). Development of Cost Effective Motion Capture System based on Arduino. Proceedings of the 2020 Fourth International Conference on Computing Methodologies and Communication (ICCMC), Erode, India.10.1109/ICCMC48092.2020.ICCMC-0001 ↗
19. Ka-Chun Siu, Current Low-Cost Video-Based Motion Analysis Options for Clinical Rehabilitation: A Systematic Review10.1093/ptj/pzz097 ↗
20. Sharma, S., Verma, S., Kumar, M., and Sharma, L. (2019, January 14–16). Use of Motion Capture in 3D Animation: Motion Capture Systems, Challenges, and Recent Trends. Proceedings of the 2019 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon), Faridabad, India.10.1109/COMITCon.2019.8862448 ↗
21. Arai, K., and Kapoor, S. (2020). Advances in Computer Vision, Proceedings of the CVC 2019. Advances in Intelligent Systems and Computing, Las Vegas, NV, USA, 25–26 April 2019, Springer.10.1007/978-3-030-17795-9 ↗
22. Deep learning for smart manufacturing: Methods and applications10.1016/j.jmsy.2018.01.003 ↗
23. Deep learning for finance: Deep portfolios10.1002/asmb.2209 ↗
24. Deep learning in the construction industry: A review of present status and future innovations10.1016/j.jobe.2020.101827 ↗
25. BIM, machine learning and computer vision techniques in underground construction: Current status and future perspectives10.1016/j.tust.2020.103677 ↗
26. Deep Learning in Medicine—Promise, Progress, and Challenges10.1001/jamainternmed.2018.7117 ↗
27. Campedelli, G.M. (2022). Machine Learning for Criminology and Crime Research: At the Crossroads. N.p., Taylor & Francis.10.4324/9781003217732 ↗
28. Deep Learning for Computer Vision: A Brief Review10.1155/2018/7068349 ↗
29. Zemouri, R., Noureddine, Z., and Daniel, R. (2019). Deep Learning in the Biomedical Applications: Recent and Future Status. Appl. Sci., 9.10.3390/app9081526 ↗
30. Deep learning-enabled medical computer vision10.1038/s41746-020-00376-2 ↗
31. Chen, K.-Y., Zheng, W.-Z., Lin, Y.-Y., Tang, S.-T., Chou, L.-W., and Lai, Y.-H. (2020, January 20–24). Deep-learning-based human motion tracking for rehabilitation applications using 3D image features. Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada.10.1109/EMBC44109.2020.9176120 ↗
32. Assessment of a novel deep learning-based marker-less motion capture system for gait study10.1016/j.gaitpost.2022.03.008 ↗
33. Human gait analysis for osteoarthritis prediction: A framework of deep learning and kernel extreme learning machine10.1007/s40747-020-00244-2 ↗
34. Deep Learning Based Advanced Spatio-Temporal Extraction Model in Medical Sports Rehabilitation for Motion Analysis and Data Processing10.1109/ACCESS.2020.3003652 ↗
35. PyTorch (2022, June 15). Available online: https://pytorch.org/get-started/previous-versions/.
36. TensorFlow (2022, June 15). Available online: https://www.tensorflow.org/versions.
37. Keras (2022, June 15). Available online: https://pypi.org/project/keras/.
38. Andriluka, M., Pishchulin, L., Gehler, P., and Schiele, B. (2014, January 23–28). 2D Human Pose Estimation: New Benchmark and State of the Art Analysis. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.471 ↗
39. Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments10.1109/TPAMI.2013.248 ↗
40. HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion10.1007/s11263-009-0273-6 ↗
41. Zheng, J., Liu, X., Liu, W., He, L., Yan, C., and Mei, T. (2022). Gait Recognition in the Wild with Dense 3D Representations and A Benchmark. arXiv, Available online: http://arxiv.org/abs/2204.02569.10.1109/CVPR52688.2022.01959 ↗
42. (2024, May 10). Institute of Automation, Chinese Academy of Sciences, CASIA Gait Database. Available online: http://www.cbsr.ia.ac.cn/english/Gait%20Databases.asp.
43. A review of 3D human pose estimation algorithms for markerless motion capture10.1016/j.cviu.2021.103275 ↗
44. Recent Advances of Monocular 2D and 3D Human Pose Estimation: A Deep Learning Perspective
45. (2024, May 10). Vicon, Full body modeling with Plug-in Gait. Nexus Documentation. Available online: https://help.vicon.com/space/Nexus212/11247555/Plug-in+Gait+Reference+Guide.
46. Mathworks 2023 (2024, May 10). interp1.MATLAB Function. Available online: https://www.mathworks.com/help/matlab/interpolation.html?category=interpolation&s_tid=CRUX_topnav.
47. (2024, May 10). Lain. Open Broadcaster Software (OBS). OBS Studio. Available online: https://obsproject.com/.
48. Kwon, J. (2022, June 15). OBS-Record. GitHub Repository. Available online: https://github.com/jrkwon/obs-record.
49. A flexible new technique for camera calibration10.1109/34.888718 ↗
50. Calib.io (2022, May 10). Calib-Camera Calibrator. Calb.io. Available online: https://calib.io/.
51. Ayoola, O. (2022, June 10). Simplifying Object Segmentation with PixelLib Library. Available online: https://pixellib.readthedocs.io/en/latest/.
52. MediaPipe (2022, July 20). Pose: A Multi-Platform Library for Pose Estimation. Available online: https://github.com/google/mediapipe/blob/master/docs/solutions/pose.md.