Human Pose Estimation Based on Lightweight Multi-Scale Coordinate Attention
Source abstract
Abstract
Heatmap-based traditional approaches for estimating human pose usually suffer from drawbacks such as high network complexity or suboptimal accuracy. Focusing on the issue of multi-person pose estimation without heatmaps, this paper proposes an end-to-end, lightweight human pose estimation network using a multi-scale coordinate attention mechanism based on the Yolo-Pose network to improve the overall network performance while ensuring the network is lightweight. Specifically, the lightweight network GhostNet was first integrated into the backbone to alleviate the problem of model redundancy and produce a significant number of effective feature maps. Then, by combining the coordinate attention mechanism, the sensitivity of our proposed network to direction and location perception was enhanced. Finally, the BiFPN module was fused to balance the feature information of different scales and further improve the expression ability of convolutional features. Experiments on the COCO 2017 dataset showed that, compared with the baseline method YOLO-Pose, the average accuracy of the proposed network on the COCO 2017 validation dataset was improved by 4.8% while minimizing the amount of network parameters and calculations. The experimental results demonstrated that our proposed method can improve the detection accuracy of human pose estimation while ensuring that the model is lightweight.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human Pose and Action Recognition · Anomaly Detection Techniques and Applications · Hand Gesture Recognition Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Beijing Key Laboratory of Information Service Engineering, Beijing Union University, Beijing 100101, China
Funding
Beijing Natural Science Foundation · National Natural Science Foundation of China · Key Project of Science and Technology Plan of Beijing Municipal Education Commission · Academic Research Projects of Beijing Union University · Premium Funding Project for Academic Human Resources Development in Beijing Union University
References (50)
Show source-provided references
1. An Improved Helmet Detection Algorithm Based on YOLO V410.1142/S0129054122420205 ↗
2. Gong, W., Zhang, X., Gonzàlez, J., Sobral, A., Bouwmans, T., Tu, C., and Zahzah, E.-h. (2016). Human pose estimation from monocular images: A comprehensive survey. Sensors, 16.10.3390/s16121966 ↗
3. Gadhiya, R., and Kalani, N. (2021, January 23–24). Analysis of deep learning based pose estimation techniques for locating landmarks on human body parts. Proceedings of the 2021 International Conference on Circuits, Controls and Communications (CCUBE), Bangalore, India.10.1109/CCUBE53681.2021.9702726 ↗
4. Recent advances of monocular 2d and 3d human pose estimation: A deep learning perspective
5. Nie, X., Feng, J., Zhang, J., and Yan, S. (2019–2, January 27). Single-stage multi-person pose machines. Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), Seoul, Republic of Korea.10.1109/ICCV.2019.00705 ↗
6. Tang, W., and Wu, Y. (2019–2, January 27). Does learning specific features for related parts help human pose estimation?. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seoul, Republic of Korea.10.1109/CVPR.2019.00120 ↗
7. Zhang, F., Zhu, X., and Ye, M. (2019–2, January 27). Fast human pose estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seoul, Republic of Korea.10.1109/CVPR.2019.00363 ↗
8. Monocular human pose estimation: A survey of deep learning-based methods10.1016/j.cviu.2019.102897 ↗
9. Huang, J., Zhu, Z., Huang, G., and Du, D. (2020, January 14–19). AID: Pushing the Performance Boundary of Human Pose Estimation with Information Dropping Augmentation. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.
10. Li, Y., Zhang, S., Wang, Z., Yang, S., Yang, W., Xia, S.T., and Zhou, E. (2021, January 10–17). TokenPose: Learning Keypoint Tokens for Human Pose Estimation. Proceedings of the 2021 IEEE International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.01112 ↗
11. UULPN: An ultra-lightweight network for human pose estimation based on unbiased data processing10.1016/j.neucom.2021.12.083 ↗
12. Future vehicles: Interactive wheeled robots10.1007/s11432-020-3171-4 ↗
13. Traffic Sign Detection via Improved Sparse R-CNN for Autonomous Vehicles
14. Maji, D., Nagori, S., Mathew, M., and Poddar, D. (2022, January 19–20). YOLO-Pose: Enhancing YOLO for Multi Person Pose Estimation Using Object Keypoint Similarity Loss. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPRW56347.2022.00297 ↗
15. Papandreou, G., Zhu, T., Kanazawa, N., Toshev, A., Tompson, J., Bregler, C., and Murphy, K. (2017, January 21–26). Towards accurate multi-person pose estimation in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.395 ↗
16. Fang, H.S., Xie, S., Tai, Y.W., and Lu, C. (2017, January 21–26). Rmpe: Regional multi-person pose estimation. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Honolulu, HI, USA.10.1109/ICCV.2017.256 ↗
17. Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., and Sun, J. (2018, January 18–23). Cascaded pyramid network for multi-person pose estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00742 ↗
18. Sun, K., Xiao, B., Liu, D., and Wang, J. (2019–02, January 27). Deep high-resolution representation learning for human pose estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seoul, Republic of Korea.10.1109/CVPR.2019.00584 ↗
19. Cao, Z., Simon, T., Wei, S.E., and Sheikh, Y. (2017, January 21–26). Realtime multi-person 2d pose estimation using part affinity fields. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.143 ↗
20. Papandreou, G., Zhu, T., Chen, L.C., Gidaris, S., Tompson, J., and Murphy, K. (2018, January 8–14). Personlab: Person pose estimation and instance segmentation with a bottom-up, part-based, geometric embedding model. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01264-9_17 ↗
21. Kreiss, S., Bertoni, L., and Alahi, A. (2019–2, January 27). Pifpaf: Composite fields for human pose estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seoul, Republic of Korea.10.1109/CVPR.2019.01225 ↗
22. Cheng, B., Xiao, B., Wang, J., Shi, H., Huang, T.S., and Zhang, L. (2020, January 14–19). Higherhrnet: Scale-aware representation learning for bottom-up human pose estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00543 ↗
23. Luo, Z., Wang, Z., Huang, Y., Wang, L., Tan, T., and Zhou, E. (2021, January 20–25). Rethinking the heatmap regression for bottom-up human pose estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.01306 ↗
24. Su, K., Yu, D., Xu, Z., Geng, X., and Wang, C. (2019–2, January 27). Multi-person pose estimation with enhanced channel-wise and spatial information. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seoul, Republic of Korea.10.1109/CVPR.2019.00582 ↗
25. Mao, W., Tian, Z., Wang, X., and Shen, C. (2021, January 20–25). Fcpose: Fully convolutional multi-person pose estimation with dynamic instance-aware convolutions. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.00892 ↗
26. Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., and Xu, C. (2020, January 14–19). Ghostnet: More features from cheap operations. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00165 ↗
27. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Wenjun, W., Tobias, W., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv.
28. Zhang, X., Zhou, X., Lin, M., and Sun, J. (2018, January 18–22). Shufflenet: An extremely efficient convolutional neural network for mobile devices. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00716 ↗
29. Kong, L., Wang, J., and Zhao, P. (2022). YOLO-G: A Lightweight Network Model for Improving the Performance of Military Targets Detection, IEEE Access.10.1109/ACCESS.2022.3177628 ↗
30. Aircraft Targets Detection in Remote Sensing Images with Feature Optimization
31. Hu, J., Shen, L., and Sun, G. (2018, January 18–22). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
32. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European conference on computer vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
33. An improved DenseNet model to classify the damage caused by cotton aphid10.1016/j.compag.2022.107485 ↗
34. Ma, Z., Zeng, Y., Zhang, L., and Li, J. (2022, January 7–10). The Workpiece Sorting Method Based on Improved YOLOv5 For Vision Robotic Arm. Proceedings of the 2022 IEEE International Conference on Mechatronics and Automation (ICMA), Guilin, China.10.1109/ICMA54519.2022.9856190 ↗
35. Tu, G., Qin, J., and Xiong, N.N. (2022). Algorithm of Computer Mainboard Quality Detection for Real-Time Based on QD-YOLO. Electronics, 11.10.3390/electronics11152424 ↗
36. Multi-scale spatial-spectral fusion based on multi-input fusion calculation and coordinate attention for hyperspectral image classification10.1016/j.patcog.2021.108348 ↗
37. Mei, Y., Fan, Y., Zhang, Y., Jiahui, Y., Yuqian, Z., Ding, L., Yun, F., Thomas, S.H., and Humphrey, S. (2020). Pyramid attention networks for image restoration. arXiv.
38. Ghiasi, G., Lin, T.Y., and Le, Q.V. (2019–2, January 27). Nas-fpn: Learning scalable feature pyramid architecture for object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seoul, Republic of Korea.10.1109/CVPR.2019.00720 ↗
39. Tan, M., Pang, R., and Le, Q.V. (2020, January 14–19). EfficientDet: Scalable and Efficient Object Detection. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01079 ↗
40. Guo, H., Zhang, R., Li, Y., Cheng, Y., and Xia, P. (2022, January 3–5). Research on human-vehicle gesture interaction technology based on computer visionbility. Proceedings of the 2022 IEEE 6th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Beijing, China.10.1109/IAEAC54830.2022.9929908 ↗
41. Pang, L., Sun, J., Chi, Y., Yang, Y., Zhang, F., and Zhang, L. (2022). CD-TransUNet: A Hybrid Transformer Network for the Change Detection of Urban Buildings Using L-Band SAR Images. Sustainability, 14.10.3390/su14169847 ↗
42. Yi, K., Luo, K., Chen, T., and Hu, R. (2022). An Improved YOLOX Model and Domain Transfer Strategy for Nighttime Pedestrian and Vehicle Detection. Appl. Sci., 12.10.3390/app122312476 ↗
43. Distance-IoU loss: Faster and better learning for bounding box regression
44. Zhang, Y.-F., Ren, W., Zhang, Z., Jia, Z., Wang, L., and Tan, T. (2022, January 19–20). Focal and Efficient IOU Loss for Accurate Bounding Box Regression. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1016/j.neucom.2022.07.042 ↗
45. Pixel-in-Pixel Net: Towards Efficient Facial Landmark Detection in the Wild10.1007/s11263-021-01521-4 ↗
46. Qiang, B., Zhang, S., Zhan, Y., Xie, W., and Zhao, T. (2019). Improved convolutional pose machines for human pose estimation using image sensor data. Sensors, 19.10.3390/s19030718 ↗
47. Newell, A., Yang, K., and Deng, J. (2016, January 11–14). Stacked hourglass networks for human pose estimation. Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands.10.1007/978-3-319-46484-8_29 ↗
48. Geng, Z., Sun, K., Xiao, B., Zhang, Z., and Wang, J. (2021, January 20–25). Bottom-up human pose estimation via disentangled keypoint regression. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.01444 ↗
49. Neff, C., Sheth, A., Furgurson, S., and Tabkhi, H. (2020). Efficienthrnet: Efficient scaling for lightweight high-resolution multi-person pose estimation. arXiv.10.1007/s11554-021-01132-9 ↗
50. Osokin, D. (2018). Real-time 2d multi-person pose estimation on cpu: Lightweight openpose. arXiv.10.5220/0007555407440748 ↗