Human Pose Estimation Based on a Spatial Temporal Graph Convolutional Network
Source abstract
Abstract
To address the problem of poor detection and under-utilization of the spatial relationship between nodes in human pose estimation, a method based on an improved spatial temporal graph convolutional network (ST-GCN) model is proposed. Firstly, upsampling and segmented random sampling strategies are used to effectively solve the problems of class imbalance and the large sequence length of the dataset. Secondly, an improved detection transformer (DETR) structure is added to effectively suppress the generation of non-maximal suppression (NMS) and anchor points, a multi-head attention (M-ATT) module is introduced into each ST-GCN cell to capture richer feature information, and a residual module is introduced into the 9th ST-GCN cell to avoid possible network degradation in deep networks. In addition, strategies such as warmup, regularization, loss functions, and optimizers are configured to improve the model’s performance. The experimental results show that the average percentage of correct keypoints (PCK) of this method are 93.2% and 92.7% for the FSD and MPII datasets, respectively, which is 1.9% and 1.7% higher than the average PCK of the original ST-GCN method. Moreover, the confusion matrix corresponding to this method also indicated that the model has high recognition accuracy. In addition, comparison experiments with ST-GCN and other methods show that the computation of the model corresponding to this method is about 1.7 GFLOPs and the corresponding MACs are about 6.4 GMACs, which is a good performance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human Pose and Action Recognition · Gait Recognition and Analysis · Video Surveillance and Tracking Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information and Control Engineering, Xi’an University of Architecture and Technology, Xi’an 710055, China
Institute for Interdisciplinary and Innovate Research, Xi’an University of Architecture and Technology, Xi’an 710055, China
Funding
National Natural Science Foundation of China · the Interdisciplinary Foundation of Xi’an University of Architecture and Technology · the Natural Science Foundation of Xi’an University of Architecture and Technology
References (35)
Show source-provided references
1. Intelligent Visual Surveillance: A Review
2. Zhang, B.R. (2021). Research on Human Posture Estimation Model and Method Based on Machine Version. [Master’s Thesis, North China University of Water Resources and Electric Power].
3. Overview of Human Behavior Recognition Based on Deep Learning
4. Skeleton Based Action Recognition Algorithm on Multi-modal Lightweight Graph Convolutional Network
5. Newell, A., Huang, Z.A., and Deng, J. (2017, January 4–9). Associative embedding: End-to-end learning for joint detection and grouping. Proceedings of the 30th International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
6. Fang, H.S., Xie, S., Tai, Y.W., and Lu, C. (2017, January 22–29). RMPE: Regional multi-person pose estimation. Proceedings of the IEEE Iternational Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.256 ↗
7. Park, S., Hwang, J., and Kwak, N. (2016, January 8–16). 3D human pose estimation us-ing convolutional neural networks with 2D pose information. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-49409-8_15 ↗
8. Development of an algorithm for abnormal human behavior detection in intelligent video surveillance system10.1088/1757-899X/1069/1/012046 ↗
9. A novel detection framework for detecting abnormal human behavior
10. Toshev, A., and Szegedy, C. (2014, January 23–28). Deeppose: Human pose estimation via deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.214 ↗
11. Yan, S., Xiong, Y., and Lin, D. (2018, January 2–7). Spatial temporal graph convolutional networks for skeleton-based action recognition. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.12328 ↗
12. Kipf, T.N., and Welling, M. (2016). Semi-supervised classification with graph convolutional networks. arXiv.
13. Temporal convolutional neural (TCN) network for an effective weather forecasting using time-series data from the local weather station10.1007/s00500-020-04954-0 ↗
14. Sofianos, T., Sampieri, A., Franco, L., and Galasso, F. (2021, January 11–17). Space-time-separable graph convolutional network for pose forecasting. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.01102 ↗
15. Cai, Y., Ge, L., Liu, J., Cai, J., Cham, T.J., Yuan, J., and Thalmann, N.M. (November, January 27). Exploiting Spatial-Temporal Relationships for 3D Pose Estimation via Graph Convolutional Networks. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00236 ↗
16. Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., and Sun, J. (2018, January 18–23). Cascaded pyramid network for multi-person pose estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00742 ↗
17. Andriluka, M., Pishchulin, L., Gehler, P., and Schiele, B. (2014, January 23–28). 2d human pose estimation: New benchmark and state of the art analysis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.471 ↗
18. Liu, S., Liu, X., Huang, G., Feng, L., Hu, L., Jiang, D., Zhang, A., Liu, Y., and Qiao, H. (2020). FSD-10: A dataset for competitive sports content analysis. arXiv.
19. Qin, Z., Liu, Y., Ji, P., Kim, D., Wang, L., McKay, B., Anwar, S., and Gedeon, T. (2021). Fusing higher-order features in graph neural networks for skeleton-based action recognition. arXiv.10.1109/TNNLS.2022.3201518 ↗
20. Human Activity Recogni-tion Technology Based on Sliding Window and Convo-lutional Neural Network
21. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020, January 23–28). End-to-end object detection with transformers. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58452-8_13 ↗
22. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. Adv. Neural Inf. Process. Syst., 30.
23. Fang, N., Fang, X.W., and Lu, K. (2022). Anomalous Behavior Detection Based on the Isolation Forest Model with Multiple Per-spective Business Processes. Electronics, 11.10.3390/electronics11213640 ↗
24. Systematic Review of Fitbit Charge 2 Validation Studies for Exercise Tracking
25. Research and application of nuclear fuel embedded intelligent security monitoring technology10.1016/j.egyr.2022.10.045 ↗
26. OpenVINO Deep Learning Workbench: Towards Analytical Platform for Neural Networks Inference Optimization10.1088/1742-6596/1828/1/012012 ↗
27. Serdar, Y. (2017, June 27). Nvidia’s New TensorRT Speeds Machine Learning Predictions. Available online: https://www.infoworld.com/article/3203938/nvidias-new-tensorrt-speeds-machine-learning-predictions.html.
28. Samaan, G.H., Wadie, A.R., Attia, A.K., Asaad, A.M., Kamel, A.E., Slim, S.O., Abdallah, M.S., and Cho, Y.-I. (2022). MediaPipe’s Landmarks with RNN for Dynamic Sign Language Recognition. Electronics, 11.10.3390/electronics11193228 ↗
29. Ioffe, S., and Szegedy, C. (2015, January 7–9). Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the International Conference on Machine Learning, Lille, France.
30. Glorot, X., Bordes, A., and Bengio, Y. (2011, January 11–13). Deep sparse rectifier neural networks. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings, Fort Lauderdale, FL, USA.
31. Xiong, R., Yang, Y., He, D., Zheng, K., Zheng, S., Xing, C., Zhang, H., Lan, Y., Wang, L., and Liu, T. (2020, January 13–18). On layer normalization in the transformer architecture. Proceedings of the International Conference on Machine Learning, Virtual Event.
32. Zhang, H., Cisse, M., Dauphin, Y.N., and Lopez-Paz, D. (2017). mixup: Beyond empirical risk minimization. arXiv.10.1007/978-1-4899-7687-1_79 ↗
33. Ruder, S. (2016). An overview of gradient descent optimization algorithms. arXiv.
34. Li, R., Wang, S., Zhu, F., and Huang, J. (2018, January 2–7). Adaptive graph convolutional neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.11691 ↗
35. Chen, Y., Zhang, Z., Yuan, C., Li, B., Deng, Y., and Hu, W. (2021, January 11–17). Channel-wise topology refinement graph convolution for skeleton-based action recognition. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.01311 ↗