A Pedestrian Detection Network Based on an Attention Mechanism and Pose Information
Source abstract
Abstract
Pedestrian detection has recently attracted widespread attention as a challenging problem in computer vision. The accuracy of pedestrian detection is affected by differences in gestures, background clutter, local occlusion, differences in scales, pixel blur, and other factors occurring in real scenes. These problems lead to false and missed detections. In view of these visual description deficiencies, we leveraged pedestrian pose information as a supplementary resource to address the occlusion challenges that arise in pedestrian detection. An attention mechanism was integrated into the visual information as a supplement to the pose information, because the acquisition of pose information was limited by the pose estimation algorithm. We developed a pedestrian detection method that integrated an attention mechanism with visual and pose information, including pedestrian region generation and pedestrian recognition networks, effectively addressing occlusion and false detection issues. The pedestrian region proposal network was used to generate a series of candidate regions with possible pedestrian targets from the original image. Then, the pedestrian recognition network was used to judge whether each candidate region contained pedestrian targets. The pedestrian recognition network was composed of four parts: visual features, pedestrian poses, pedestrian attention, and classification modules. The visual feature module was responsible for extracting the visual feature descriptions of candidate regions. The pedestrian pose module was used to extract pose feature descriptions. The pedestrian attention module was used to extract attention information, and the classification module was responsible for fusing visual features and pedestrian pose descriptions with the attention mechanism. The experimental results on the Caltech and CityPersons datasets demonstrated that the proposed method could substantially more accurately identify pedestrians than current state-of-the-art methods.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Advanced Neural Network Applications · Autonomous Vehicle Technology and Safety
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Information Engineering, Yangzhou Polytechnic College, Yangzhou 225009, China
School of Computer, Jiangsu University of Science and Technology, Zhenjiang 212003, China
Jingjiang College, Jiangsu University, Zhenjiang 212013, China
References (59)
Show source-provided references
1. Towards reaching human performance in pedestrian detection10.1109/TPAMI.2017.2700460 ↗
2. Hosang, J., Omran, M., Benenson, R., and Schiele, B. (2015, January 7–12). Taking a deeper look at pedestrians. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7299034 ↗
3. Tian, Y., Luo, P., Wang, X., and Tang, X. (2015, January 7–12). Pedestrian detection aided by deep learning semantic tasks. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.10.1109/CVPR.2015.7299143 ↗
4. Zhang, S., Benenson, R., Omran, M., Hosang, J., and Schiele, B. (2016, January 27–30). How far are we from solving pedestrian detection?. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.10.1109/CVPR.2016.141 ↗
5. Zhou, C., and Yuan, J. (2017, January 22–29). Multi-label learning of part detectors for heavily occluded pedestrian detection. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.377 ↗
6. Partial occlusion handling in pedestrian detection with a deep model10.1109/TCSVT.2015.2501940 ↗
7. Pushing the limits of deep CNNs for pedestrian detection10.1109/TCSVT.2017.2648850 ↗
8. Cai, Z., and Vasconcelos, N. (2018, January 18–23). Cascade R-CNN: Delving into high quality object detection. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00644 ↗
9. PCL: Proposal cluster learning for weakly supervised object detection10.1109/TPAMI.2018.2876304 ↗
10. Faster R-CNN: Towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
11. Zhang, L., Lin, L., Liang, X., and He, K. (2016, January 8–16). Is faster R-CNN doing well for pedestrian detection?. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46475-6_28 ↗
12. Zhang, S., Benenson, R., and Schiele, B. (2017, January 21–26). Citypersons: A diverse dataset for pedestrian detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.474 ↗
13. Song, T., Sun, L., Xie, D., Sun, H., and Pu, S. (2018, January 8–14). Small-scale pedestrian detection based on topological line localization and temporal feature aggregation. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01234-2_33 ↗
14. Tian, Y., Luo, P., Wang, X., and Tang, X. (2015, January 7–13). Deep learning strong parts for pedestrian detection. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.221 ↗
15. Zhou, C., and Yuan, J. (2018, January 8–14). Bi-box regression for pedestrian detection and occlusion estimation. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01246-5_9 ↗
16. Cao, Z., Simon, T., Wei, S.-E., and Sheikh, Y. (2017, January 21–26). Realtime multi-person 2D pose estimation using part affinity fields. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.143 ↗
17. Pedestrian detection: An evaluation of the state of the art10.1109/TPAMI.2011.155 ↗
18. Redmon, J., Divvala, S.K., Girshick, R.B., and Farhadi, A. (July, January 26). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
19. Redmon, J., and Farhad, A.I. (2017, January 21–26). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.690 ↗
20. Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv.
21. Bochkovskiy, A., Wang, C., and Liao, H.M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv.
22. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 8–16). SSD: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
23. Li, Z., Chen, Y., and Gang, Y. (2018, January 2–7). R-FCN++: Towards Accurate Region-Based Fully Convolutional Networks for Object Detection. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.10.1609/aaai.v32i1.12265 ↗
24. Shi, S., Guo, C., and Li, J. (November, January 27). PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
25. Law, H., and Deng, J. (2018, January 8–14). CornerNet: Detecting objects as paired keypoints. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01264-9_45 ↗
26. Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., and Tian, Q. (November, January 27). CenterNet: Keypoint Triplets for Object Detection. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.10.1109/ICCV.2019.00667 ↗
27. Scale-aware fast R-CNN for pedestrian detection
28. Cai, Z., Fan, Q., Feris, R.S., and Vasconcelos, N. (2016, January 8–16). A unified multi-scale deep convolutional neural network for fast object detection. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46493-0_22 ↗
29. Liu, W., Liao, S., Hu, W., Liang, X., and Chen, X. (2018, January 8–14). Learning efficient single-stage pedestrian detectors by symptotic localization fitting. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01264-9_38 ↗
30. Liu, W., Liao, S., Ren, W., Hu, W., and Yu, Y. (2019, January 16–20). High-level semantic feature detection: A new perspective for pedestrian detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00533 ↗
31. Hasan, I., Liao, S., Li, J., Akram, S.U., and Shao, L. (2021, January 20–25). Generalizable pedestrian detection: The elephant in the room. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01117 ↗
32. Khan, A.H., Munir, M., van Elst, L., and Dengel, A. (2022, January 21–25). F2dnet: Fast focal detection network for pedestrian detection. Proceedings of the 2022 26th International Conference on Pattern Recognition (ICPR), Montreal, QC, Canada.10.1109/ICPR56361.2022.9956732 ↗
33. Hagn, K., and Grau, O. (2022, January 8). Increasing pedestrian detection performance through weighting of detection impairing factors. Proceedings of the 6th ACM Computer Science in Cars Symposium, Ingolstadt, Germany.10.1145/3568160.3570225 ↗
34. Khan, A.H., Nawaz, M.S., and Dengel, A. (2023, January 18–22). Localized semantic feature mixers for efficient pedestrian detection in autonomous driving. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00530 ↗
35. Luo, Z., Fang, Z., Zheng, S., Wang, Y., and Fu, Y. (2021, January 16–19). NMS-loss: Learning with non-maximum suppression for crowded pedestrian detection. Proceedings of the 2021 International Conference on Multimedia Retrieval, Taipei, Taiwan.10.1145/3460426.3463588 ↗
36. Liu, M., Jiang, J., Zhu, C., and Yin, X.C. (2023, January 17–24). Vlpd: Context-aware pedestrian detection via vision-language semantic self-supervision. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.00644 ↗
37. Semantic head enhanced pedestrian detection in a crowd10.1016/j.neucom.2020.03.037 ↗
38. Enzweiler, M., Eigenstetter, A., Schiele, B., and Gavrila, D.M. (2010, January 13–18). Multi-cue Pedestrian Classification with Partial Occlusion Handling. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5540111 ↗
39. Noh, J., Lee, S., Kim, B., and Kim, G. (2018, January 18–22). Improving occlusion and hard negative handling for single-stage pedestrian detectors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00107 ↗
40. Zhang, S., Wen, L., Bian, X., Lei, Z., and Li, S.Z. (2018, January 8–14). Occlusion-aware R-CNN: Detecting pedestrians in a crowd. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01219-9_39 ↗
41. Wang, X., Xiao, T., Jiang, Y., Shao, S., Sun, J., and Shen, C. (2018, January 18–22). Repulsion loss: Detecting pedestrians in crowd. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00811 ↗
42. Larsson, V., Kukelova, Z., and Zheng, Y. (2018, January 18–22). Camera pose estimation with unknown principal point. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00315 ↗
43. Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., and Sun, J. (2018, January 18–22). Cascaded pyramid network for multi-person pose estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00742 ↗
44. Guler, R.A., Neverova, N., and Kokkinos, I. (2018, January 18–22). DensePose: Dense human pose estimation in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00762 ↗
45. ADORE: An adaptive holons representation framework for human pose estimation10.1109/TCSVT.2017.2707477 ↗
46. Body parts synthesis for cross-quality pose estimation10.1109/TCSVT.2017.2789224 ↗
47. Human pose estimation in video via structured space learning and halfway temporal evaluation10.1109/TCSVT.2018.2858828 ↗
48. Liu, Z., Feng, R., Chen, H., Wu, S., Gao, Y., Gao, Y., and Wang, X. (2022, January 19–24). Temporal Feature Alignment and Mutual Information Maximization for Video-Based Human Pose Estimation. Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.10.1109/CVPR52688.2022.01073 ↗
49. Tian, Z., Shen, C., Chen, H., and He, T. (2019, January 16–20). FCOS: Fully convolutional one-stage object detection. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/ICCV.2019.00972 ↗
50. Woo, S., Park, J., Lee, J.-Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV).10.1007/978-3-030-01234-2_1 ↗
51. He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7–12). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).10.1109/CVPR.2016.90 ↗
52. PEN: Pose-Embedding Network for Pedestrian Detection10.1109/TCSVT.2020.3000223 ↗
53. Kingma, D.P., and Ba, J. (2015, January 7–9). Adam: A Method for Stochastic Optimization. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA.
54. Brazil, G., and Liu, X. (2019, January 16–20). Pedestrian detection with autoregressive network phases. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00740 ↗
55. Du, X., El-Khamy, M., Lee, J., and Davis, L. (2017, January 24–31). Fused DNN: A deep neural network fusion approach to fast and robust pedestrian detection. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Santa Rosa, CA, USA.10.1109/WACV.2017.111 ↗
56. Brazil, G., Yin, X., and Liu, X. (2017, January 22–29). Illuminating pedestrians via simultaneous detection and segmentation. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.530 ↗
57. Fu, C.-Y., Liu, W., Ranga, A., Tyagi, A., and Berg, A.C. (2017). DSSD: Deconvolutional single shot detector [EB/OL]. arXiv.
58. Zhou, P., Zhou, C., Peng, P., Du, J., Sun, X., Guo, X., and Huang, F. (2020, January 12–16). NOH-NMS: Improving Pedestrian Detection by Nearby Objects Hallucination. Proceedings of the ACM International Conference on Multimedia, Seattle, WA, USA.10.1145/3394171.3413617 ↗
59. Xu, Z., Li, B., Yuan, Y., and Dang, A. (2022). Beta R-CNN: Looking into Pedestrian Detection from Another Perspective. arXiv.