journal article·2023-06-27Open access

High Speed and Accuracy of Animation 3D Pose Recognition Based on an Improved Deep Convolution Neural Network

Wei Ding, Wenfa Li

Applied Sciences· Volume 13· Issue 13

Abstract

Pose recognition in character animations is an important avenue of research in computer graphics. However, the current use of traditional artificial intelligence algorithms to recognize animation gestures faces hurdles such as low accuracy and speed. Therefore, to overcome the above problems, this paper proposes a real-time 3D pose recognition system, which includes both facial and body poses, based on deep convolutional neural networks and further designs a single-purpose 3D pose estimation system. First, we transformed the human pose extracted from the input image to an abstract pose data structure. Subsequently, we generated the required character animation at runtime based on the transformed dataset. This challenges the conventional concept of monocular 3D pose estimation, which is extremely difficult to achieve. It can also achieve real-time running speed at a resolution of 384 fps. The proposed method was used to identify multiple-character animation using multiple datasets (Microsoft COCO 2014, CMU Panoptic, Human3.6M, and JTA). The results indicated that the improved algorithm improved the recognition accuracy and performance by approximately 3.5% and 8–10 times, respectively, which is significantly superior to other classic algorithms. Furthermore, we tested the proposed system on multiple pose-recognition datasets. The 3D attitude estimation system speed can reach 24 fps with an error of 100 mm, which is considerably less than that of the 2D attitude estimation system with a speed of 60 fps. The pose recognition based on deep learning proposed in this study yielded surprisingly superior performance, proving that the use of deep-learning technology for image recognition has great potential.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Human Pose and Action Recognition · Human Motion and Animation · Advanced Vision and Imaging

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Journalism & Communication, Anhui Normal University, Wuhu 241002, China

The Institute of Artificial Intelligence, University of Science and Technology Beijing, Beijing 100083, China

References (48)

Show source-provided references
1. A catheter posture recognition method in three dimensions by using RF signal computation10.1016/j.sna.2023.114292 ↗
2. Isomorphic model-based initialization for convolutional neural networks10.1016/j.jvcir.2022.103677 ↗
3. Convolutional neural network initialization approaches for image manipulation detection10.1016/j.dsp.2021.103376 ↗
4. Study on Volleyball-Movement Pose Recognition Based on Joint Point Sequence10.1155/2023/2198495 ↗
5. Sleep posture recognition based on machine learning: A systematic review10.1016/j.pmcj.2023.101752 ↗
6. Machine learning and uLBP histograms for posture recognition of dependent people via Big Data Hadoop and Spark platform10.15837/ijccc.2023.1.4981 ↗
7. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
8. WSN-Driven Posture Recognition and Correction towards Basketball Exercise10.4018/IJISMD.300777 ↗
9. Ji, H., Yu, J., Lao, F., Zhuang, Y., Wen, Y., and Teng, G. (2022). Automatic Position Detection and Posture Recognition of Grouped Pigs Based on Deep Learning. Agriculture, 12.10.3390/agriculture12091314 ↗
10. Deep Learning-Based Posture Recognition for Motion-Assisted Evaluation
11. Comparative study of orthogonal moments for human postures recognition10.1016/j.engappai.2023.105855 ↗
12. Leone, A., Rescio, G., Caroppo, A., Siciliano, P., and Manni, A. (2023). Human Postures Recognition by Accelerometer Sensor and ML Architecture Integrated in Embedded Platforms: Benchmarking and Performance Evaluation. Sensors, 23.10.3390/s23021039 ↗
13. Human Skeleton Coordinate Pose Recognition10.26855/ea.2022.12.010 ↗
14. Multimodal data-based deep learning model for sitting posture recognition toward office workers’ health promotion10.1016/j.sna.2022.114150 ↗
15. A boosting framework for human posture recognition using spatio-temporal features along with radon transform10.1007/s11042-022-13536-1 ↗
16. Bourahmoune, K., Ishac, K., and Amagasa, T. (2022). Intelligent Posture Training: Machine-Learning-Powered Human Sitting Posture Recognition Based on a Pressure-Sensing IoT Cushion. Sensors, 22.10.3390/s22145337 ↗
17. Intelligent Active Correction Seat Based on Neural Network Sitting Posture Recognition
18. Touchless Head-Control (THC): Head Gesture Recognition for Cursor and Orientation Control10.1109/TNSRE.2022.3187472 ↗
19. TensorPack: A Maple-based software package for the manipulation of algebraic expressions of tensors in general relativity10.1088/1742-6596/633/1/012021 ↗
20. Towards pen-holding hand pose recognition: A new benchmark and a coarse-to-fine PHHP recognition network10.1049/bme2.12079 ↗
21. Basketball Motion Posture Recognition Based on Recurrent Deep Learning Model
22. Analysis of the Impact of Yoga on Health Care Applications and Human Pose Recognition10.1149/10701.7889ecst ↗
23. Detection, identification and posture recognition of cattle with satellites, aerial photography and UAVs using deep learning techniques10.1080/01431161.2022.2051634 ↗
24. Mu, J., Xian, S., Yu, J., Zhao, J., Song, J., Li, Z., Hou, X., Chou, X., and He, J. (2022). Synergistic Enhancement Properties of a Flexible Integrated PAN/PVDF Piezoelectric Sensor for Human Posture Recognition. Nanomaterials, 12.10.3390/nano12071155 ↗
25. Parra-Dominguez, G.S., Sanchez-Yanez, R.E., and Garcia-Capulin, C.H. (2022). Towards Facial Gesture Recognition in Photographs of Patients with Facial Palsy. Healthcare, 10.10.3390/healthcare10040659 ↗
26. Guerra, B.M.V., Schmid, M., Beltrami, G., and Ramat, S. (2022). Neural Networks for Automatic Posture Recognition in Ambient-Assisted Living. Sensors, 22.10.3390/s22072609 ↗
27. Lee, J.-H., Lee, J., and Park, S.-Y. (2022). 3D Pose Recognition System of Dump Truck for Autonomous Excavator. Appl. Sci., 12.10.3390/app12073471 ↗
28. Lin, T.Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Zitnick, C.L., and Dollár, P. (2014). Microsoft Coco: Common Objects in Context. European Conference on Computer Vision, Springer.10.1007/978-3-319-10602-1_48 ↗
29. Joo, H., Liu, H., Tan, L., Gui, L., Nabbe, B., Matthews, I., Kanade, T., Nobuhara, S., and Sheikh, Y. (2015, January 7–13). Panoptic studio: A massively multiview system for social motion capture. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.381 ↗
30. Fabbri, M., Lanzi, F., Calderara, S., Palazzi, A., Vezzani, R., and Cucchiara, R. (2018, January 8–14). Learning to detect and track visible and occluded body joints in a virtual world. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01225-0_27 ↗
31. Automated brain tumor detection and segmentation using modified UNet and ResNet model10.1007/s00500-023-08420-5 ↗
32. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
33. A Dance Somersault Pose Recognition Model Using Multifeature Fusion Algorithm
34. Graph neural networks: A review of methods and applications10.1016/j.aiopen.2021.01.001 ↗
35. Martinez, J., Hossain, R., Romero, J., and Little, J.J. (2017, January 22–29). A simple yet effective baseline for 3d human pose estimation. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.288 ↗
36. LCR-Net++: Multi-Person 2D and 3D Pose Detection in Natural Images
37. Lee, K., Lee, I., and Lee, S. (2018, January 8–14). Propagating LSTM: 3D Pose Estimation Based on Joint Interdependency. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01234-2_8 ↗
38. VNect: Real-Time 3D Human Pose Estimation with a Single RGB Camera10.1145/3072959.3073596 ↗
39. Liu, R., Shen, J., Wang, H., Chen, C., Cheung, S.C., and Asari, V. (2020, January 13–19). Attention Mechanism Exploits Temporal Contexts: Real-Time 3D Human Pose Reconstruction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00511 ↗
40. Lam, S.K., Pitrou, A., and Seibert, S. (2015, January 15). Numba: A llvm-based python JIT compiler. Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC, Austin, TX, USA.10.1145/2833157.2833162 ↗
41. Acute effects of instrument assisted soft tissue mobilization technique on pain and joint position error in individuals with chronic neck pain: A double-blind, randomized controlled trial10.1080/08990220.2022.2157388 ↗
42. Cao, Z., Simon, T., Wei, S.E., and Sheikh, Y. (2017, January 21–26). Realtime multi-person 2d pose estimation using part affinity fields. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.143 ↗
43. Wang, C., Li, J., Liu, W., Qian, C., and Lu, C. (2020, January 23–28). Hmor: Hierarchical multi-person ordinal relations for monocular multi-person 3d pose estimation. Proceedings of the Computer Vision—ECCV 2020: 16th European Conference, Glasgow, UK.10.1007/978-3-030-58580-8_15 ↗
44. Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., and Sun, J. (2018, January 18–23). Cascaded Pyramid Network for Multi-Person Pose Estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00742 ↗
45. 3D human pose and shape estimation via de-occlusion multi-task learning10.1016/j.neucom.2023.126284 ↗
46. Continuous Transfer of Neural Network Representational Similarity for Incremental Learning10.1016/j.neucom.2023.126300 ↗
47. Hyper-sausage coverage function neuron model and learning algorithm for image classi cation10.1016/j.patcog.2022.109216 ↗
48. Face editing based on facial recognition features10.1109/TCDS.2022.3182650 ↗