A New Multi-Person Pose Estimation Method Using the Partitioned CenterPose Network
Source abstract
Abstract
In bottom-up multi-person pose estimation, grouping joint candidates into the appropriately structured corresponding instance of a person is challenging. In this paper, a new bottom-up method, the Partitioned CenterPose (PCP) Network, is proposed to better cluster the detected joints. To achieve this goal, we propose a novel approach called Partition Pose Representation (PPR) which integrates the instance of a person and its body joints based on joint offset. PPR leverages information about the center of the human body and the offsets between that center point and the positions of the body’s joints to encode human poses accurately. To enhance the relationships between body joints, we divide the human body into five parts, and then, we generate a sub-PPR for each part. Based on this PPR, the PCP Network can detect people and their body joints simultaneously, then group all body joints according to joint offset. Moreover, an improved l1 loss is designed to more accurately measure joint offset. Using the COCO keypoints and CrowdPose datasets for testing, it was found that the performance of the proposed method is on par with that of existing state-of-the-art bottom-up methods in terms of accuracy and speed.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Human Pose and Action Recognition · Hand Gesture Recognition Systems · Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Division of Computer Science and Engineering, Jeonbuk National University, Jeonju 54896, Korea
Funding
National Research Foundation of Korea
References (38)
Show source-provided references
1. Dong, J., Gao, Y., Lee, H.J., Zhou, H., Yao, Y., Fang, Z., and Huang, B. (2020). Action Recognition Based on the Fusion of Graph Convolutional Networks with High Order Features. Appl. Sci., 10.10.3390/app10041482 ↗
2. Insafutdinov, E., Andriluka, M., Pishchulin, L., Tang, S., Levinkov, E., Andres, B., and Schiele, B. (2017, January 21–26). Arttrack: Articulated mul-ti-person tracking in the wild. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.142 ↗
3. Monocular human pose estimation: A survey of deep learning-based methods10.1016/j.cviu.2019.102897 ↗
4. Newell, A., Yang, K., and Deng, J. (2016, January 11–14). Stacked hourglass networks for human pose estimation. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46484-8_29 ↗
5. He, K., Gkioxari, G., Dollar, P., and Girshick, R. (2017, January 22–29). Mask R-CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.322 ↗
6. Papandreou, G., Zhu, T., Kanazawa, N., Toshev, A., Tompson, J., Bregler, C., and Murphy, K. (2017, January 21–26). Towards Accurate Multi-Person Pose Estimation in the Wild. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.395 ↗
7. Xiao, B., Wu, H., and Wei, Y. (2018, January 10–13). Simple Baselines for Human Pose Estimation and Tracking. Proceedings of the European Conference on Computer Vision, GASTEIG Cultural Center, Munich, Germany.10.1007/978-3-030-01231-1_29 ↗
8. Sun, K., Xiao, B., Liu, D., and Wang, J. (2019, January 16–20). Deep high-resolution representation learning for human pose estimation. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00584 ↗
9. Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., and Sun, J. (2018, January 18–22). Cascaded Pyramid Network for Multi-person Pose Estimation. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.10.1109/CVPR.2018.00742 ↗
10. Sun, X., Xiao, B., Wei, F., Liang, S., and Wei, Y. (2018, January 10–13). Integral Human Pose Regression. Proceedings of the European Conference on Computer Vision, GASTEIG Cultural Center, Munich, Germany.10.1007/978-3-030-01231-1_33 ↗
11. Fang, H.-S., Xie, S., Tai, Y.-W., and Lu, C. (2017, January 22–29). RMPE: Regional Multi-person Pose Estimation. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.256 ↗
12. Cao, Z., Šimon, T., Wei, S.-E., and Sheikh, Y. (2017, January 21–26). Realtime multi-person 2d pose estimation using part affinity fields. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.143 ↗
13. Newell, A., Huang, Z., and Deng, J. (2017, January 4–9). Associative embedding: End-to-end learning for joint detection and grouping. Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17), Red Hook, NY, USA.
14. Papandreou, G., Zhu, T., Chen, L.-C., Gidaris, S., Tompson, J., and Murphy, K. (2018, January 10–13). PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Model. Proceedings of the European Conference on Computer Vision, GASTEIG Cultural Center, Munich, Germany.10.1007/978-3-030-01264-9_17 ↗
15. Kreiss, S., Bertoni, L., and Alahi, A. (2019, January 15–20). PifPaf: Composite Fields for Human Pose Estimation. Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.10.1109/CVPR.2019.01225 ↗
16. Cheng, B., Xiao, B., Wang, J., Shi, H., Huang, T.S., and Zhang, L. (2020, January 16–28). Higherhrnet: Scale-aware representation learning for bot-tom-up human pose estimation. Proceedings of the International Conference on Computer Vision and Pattern Recogni-tion (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.00543 ↗
17. Zhou, X., Wang, D., and Krähenbühl, P. (2019). Objects as points. arXiv.
18. Tompson, J., Goroshin, R., Jain, A., LeCun, Y., and Bregler, C. (2015, January 7–12). Efficient object localization using Convolutional Networks. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7298664 ↗
19. Wei, S.-E., Ramakrishna, V., Kanade, T., and Sheikh, Y. (2016, January 27–30). Convolutional Pose Machines. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.511 ↗
20. Human Pose Estimation via Convolutional Part Heatmap Regression
21. Chu, X., Yang, W., Ouyang, W., Ma, C., Yuille, A.L., and Wang, X. (2017, January 21–26). Multi-context Attention for Human Pose Estimation. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.601 ↗
22. Human Pose Estimation Using Deep Consensus Voting
23. Carreira, J., Agrawal, P., Fragkiadaki, K., and Malik, J. (2016, January 27–30). Human Pose Estimation with Iterative Error Feedback. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.512 ↗
24. Hu, P., and Ramanan, D. (2016, January 27–30). Bottom-Up and Top-Down Reasoning with Hierarchical Rectified Gaussians. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.604 ↗
25. Yu, F., Wang, D., Shelhamer, E., and Darrell, T. (2018, January 18–22). Deep Layer Aggregation. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00255 ↗
26. He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv.10.1109/CVPR.2016.90 ↗
27. Kim, S.-T., and Lee, H.J. (2020). Lightweight Stacked Hourglass Network for Human Pose Estimation. Appl. Sci., 10.10.3390/app10186497 ↗
28. Law, H., and Deng, J. (2018, January 10–13). Cornernet: Detecting objects as paired keypoints. Proceedings of the European Conference on Computer Vision, GASTEIG Cultural Center, Munich, Germany.10.1007/978-3-030-01264-9_45 ↗
29. Focal Loss for Dense Object Detection10.1109/TPAMI.2018.2858826 ↗
30. Girshick, R. (2015, January 13–16). Fast R-CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.169 ↗
31. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014). Microsoft COCO: Common objects in context. Computer Vision—ECCV ECCV Lecture Notes in Computer Science, Springer.10.1007/978-3-319-10602-1_48 ↗
32. Pose Anchor: A Single-stage Hand Keypoint Detection Network10.1109/TCSVT.2019.2923444 ↗
33. Xia, H., and Zhang, T. (2021). Self-Attention Network for Human Pose Estimation. Appl. Sci., 11.10.3390/app11041826 ↗
34. Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., and Tian, Q. (November, January 27). CenterNet: Keypoint Triplets for Object Detection. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.10.1109/ICCV.2019.00667 ↗
35. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. (2017, January 4–9). Automatic differentiation in pytorch. Proceedings of the NeurIPS Workshop, Long Beach, CA, USA.
36. Tompson, J.J., Jain, A., LeCun, Y., and Bregler, C. (2014). Joint training of a convolutional network and a graphical model for human pose estimation. Advances in Neural Information Processing Systems, MIT Press.
37. Kingma, D.P., and Ba, J. (2015, January 5–8). Adam: A method for stochastic optimization. Proceedings of the International Conference Learn, Represent, (ICLR), San Diego, CA, USA.
38. Li, J., Wang, C., Zhu, H., Mao, Y., Fang, H.-S., and Lu, C. (, January 16–20). CrowdPose: Efficient Crowded Scenes Pose Estimation and a New Benchmark. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019.10.1109/CVPR.2019.01112 ↗