HP-YOLO: A Lightweight Real-Time Human Pose Estimation Method
Source abstract
Abstract
Human Pose Estimation (HPE) plays a critical role in medical applications, particularly within nursing robotics for patient monitoring. Despite its importance, HPE faces several challenges, including high rates of false positives and negatives, stringent real-time requirements, and limited computational resources, especially in complex backgrounds. In response, we introduce the HP-YOLO model, developed using the YOLOv8 framework, to effectively address these issues. We designed an Enhanced Large Separated Kernel Attention (ELSKA) mechanism and integrated it into the backbone network, thereby improving the model’s effective receptive field and feature separation capabilities, which enhances keypoint detection accuracy in challenging environments. Additionally, the Reparameterized Network with Cross-Stage Partial Connections and Efficient Layer Aggregation Network (RepNCSPELAN4) module was incorporated into the detection head, boosting accuracy in detecting small-sized targets through multi-scale convolution and reparameterization techniques while accelerating inference speed. On the COCO dataset, our HP-YOLO model outperformed existing lightweight methods by increasing average precision (AP) by 4.9%, while using 18% fewer parameters and achieving 1.4× higher inference speed. Our method significantly enhances the real-time performance and efficiency of human pose estimation while maintaining high accuracy, offering an optimal solution for applications in complex environments.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human Pose and Action Recognition · Gait Recognition and Analysis · Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Automation, College of Electrical Engineering, Sichuan University, Chengdu 610065, China
Key Laboratory of Information and Automation Technology of Sichuan Province, Sichuan University, Chengdu 610065, China
Funding
Sichuan Science and Technology Program
References (37)
Show source-provided references
1. Vision-based human pose estimation via deep learning: A survey10.1109/THMS.2022.3219242 ↗
2. Deep learning-based human pose estimation: A survey
3. Recent advances of monocular 2d and 3d human pose estimation: A deep learning perspective
4. Ahmad, N., Strand, R., Sparresäter, B., Tarai, S., Lundström, E., Bergström, G., Ahlström, H., and Kullberg, J. (2023). Automatic segmentation of large-scale CT image datasets for detailed body composition analysis. BMC Bioinform., 24.10.1186/s12859-023-05462-2 ↗
5. Hayat, M., Aramvith, S., and Achakulvisut, T. (2024, January 15–19). SEGSRNet for stereo-endoscopic image super-resolution and surgical instrument segmentation. Proceedings of the 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA.10.1109/EMBC53108.2024.10782794 ↗
6. Squeeze & Excitation joint with Combined Channel and Spatial Attention for Pathology Image Super-Resolution10.1016/j.fraope.2024.100170 ↗
7. National Pressure Ulcer Advisory Panel’s updated pressure ulcer staging system10.1097/01.ASW.0000269314.23015.e9 ↗
8. Human pose estimation based on feature enhancement and multi-scale feature fusion10.1007/s11760-022-02271-7 ↗
9. A lightweight pose estimation network with multi-scale receptive field10.1007/s00371-023-02953-4 ↗
10. Tang, Y., Wang, Y., Xu, Y., Deng, Y., Xu, C., Tao, D., and Xu, C. (2021, January 20–25). Manifold regularized dynamic network pruning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00498 ↗
11. Cao, Z., Simon, T., Wei, S.E., and Sheikh, Y. (2017, January 21–26). Realtime multi-person 2d pose estimation using part affinity fields. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.143 ↗
12. Liu, J., Lei, Q., Qiao, Y., Gui, G., Li, X., Jin, J., and Wang, W. (2020, January 11–14). A visual based robot trajectory teaching method for traditional chinese medical moxibustion therapy. Proceedings of the 2020 IEEE 6th International Conference on Computer and Communications (ICCC), Chengdu, China.10.1109/ICCC51575.2020.9345310 ↗
13. Patient-specific pose estimation in clinical environments10.1109/JTEHM.2018.2875464 ↗
14. Cao, T., Armin, M.A., Denman, S., Petersson, L., and Ahmedt-Aristizabal, D. (2022, January 28–31). In-bed human pose estimation from unseen and privacy-preserving image domains. Proceedings of the 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), Kolkata, India.10.1109/ISBI52829.2022.9761598 ↗
15. In-bed pose estimation: Deep learning with shallow dataset10.1109/JTEHM.2019.2892970 ↗
16. Girshick, R. (2015, January 7–13). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.169 ↗
17. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
18. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.322 ↗
19. Vitpose: Simple vision transformer baselines for human pose estimation
20. Pishchulin, L., Insafutdinov, E., Tang, S., Andres, B., Andriluka, M., Gehler, P.V., and Schiele, B. (2016, January 27–30). Deepcut: Joint subset partition and labeling for multi person pose estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.533 ↗
21. Cheng, B., Xiao, B., Wang, J., Shi, H., Huang, T.S., and Zhang, L. (2020, January 13–19). Higherhrnet: Scale-aware representation learning for bottom-up human pose estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00543 ↗
22. Hou, Q., Zhou, D., and Feng, J. (2021, January 20–25). Coordinate attention for efficient mobile network design. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01350 ↗
23. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00745 ↗
24. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
25. Large separable kernel attention: Rethinking the large kernel attention design in cnn10.1016/j.eswa.2023.121352 ↗
26. Osokin, D. (2018). Real-time 2d multi-person pose estimation on cpu: Lightweight openpose. arXiv.10.5220/0007555407440748 ↗
27. Wang, Y., Li, M., Cai, H., Chen, W.M., and Han, S. (2022, January 18–24). Lite pose: Efficient architecture design for 2d human pose estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01278 ↗
28. Wang, H., Liu, J., Tang, J., and Wu, G. (November, January 29). Lightweight Super-Resolution Head for Human Pose Estimation. Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada.
29. Wang, T., Wang, K., Cai, H., Lin, J., Liu, Z., Wang, H., Lin, Y., and Han, S. (2020, January 13–19). Apq: Joint search for network architecture, pruning and quantization policy. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00215 ↗
30. Wang, C.Y., Bochkovskiy, A., and Liao, H.Y.M. (2021, January 20–25). Scaled-yolov4: Scaling cross stage partial network. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01283 ↗
31. Wang, C.Y., Yeh, I.H., and Liao, H.Y.M. (2024). YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information. arXiv.10.1007/978-3-031-72751-1_1 ↗
32. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft coco: Common objects in context. Proceedings of the Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland. Proceedings, Part V 13.10.1007/978-3-319-10602-1_48 ↗
33. Chollet, F. (2017, January 21–26). Xception: Deep learning with depthwise separable convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.195 ↗
34. Yu, F. (2015). Multi-scale context aggregation by dilated convolutions. arXiv.
35. Sparsity in deep learning: Pruning and growth for efficient inference and training in neural networks
36. Liu, Z., Sun, M., Zhou, T., Huang, G., and Darrell, T. (2018). Rethinking the value of network pruning. arXiv.
37. Ding, X., Zhang, X., Han, J., and Ding, G. (2022, January 18–24). Scaling up your kernels to 31x31: Revisiting large kernel design in cnns. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01166 ↗