A Framework for Pedestrian Attribute Recognition Using Deep Learning
Source abstract
Abstract
The pedestrian attribute recognition task is becoming more popular daily because of its significant role in surveillance scenarios. As the technological advances are significantly more than before, deep learning came to the surface of computer vision. Previous works applied deep learning in different ways to recognize pedestrian attributes. The results are satisfactory, but still, there is some scope for improvement. The transfer learning technique is becoming more popular for its extraordinary performance in reducing computation cost and scarcity of data in any task. This paper proposes a framework that can work in surveillance scenarios to recognize pedestrian attributes. The mask R-CNN object detector extracts the pedestrians. Additionally, we applied transfer learning techniques on different CNN architectures, i.e., Inception ResNet v2, Xception, ResNet 101 v2, ResNet 152 v2. The main contribution of this paper is fine-tuning the ResNet 152 v2 architecture, which is performed by freezing layers, last 4, 8, 12, 14, 20, none, and all. Moreover, data balancing techniques are applied, i.e., oversampling, to resolve the class imbalance problem of the dataset and analysis of the usefulness of this technique is discussed in this paper. Our proposed framework outperforms state-of-the-art methods, and it provides 93.41% mA and 89.24% mA on the RAP v2 and PARSE100K datasets, respectively.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Advanced Neural Network Applications · Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science and Engineering, Chittagong University of Engineering & Technology, Chattogram 4349, Bangladesh
Department of Electrical Engineering, Sejong University, 209 Neungdong-ro, Gwangjin-gu, Seoul 05006, Korea
References (50)
Show source-provided references
1. Li, D., Chen, X., and Huang, K. (2015, January 3–6). Multi-attribute learning for pedestrian attribute recognition in surveillance scenarios. Proceedings of the 2015 IEEE 3rd IAPR Asian Conference on Pattern Recognition (ACPR), Kuala Lumpur, Malaysia.10.1109/ACPR.2015.7486476 ↗
2. A survey of the recent architectures of deep convolutional neural networks10.1007/s10462-020-09825-6 ↗
3. Zhang, J., Ren, P., and Li, J. (2020). Deep Template Matching for Pedestrian Attribute Recognition with the Auxiliary Supervision of Attribute-wise Keypoints. arXiv.
4. Chen, Y., Duffner, S., Stoian, A., Dufour, J.Y., and Baskurt, A. (2018). Pedestrian attribute recognition with part-based CNN and combined feature representations. VISAPP2018, HAL Open Science.10.5220/0006622901140122 ↗
5. Wang, J., Zhu, X., Gong, S., and Li, W. (2017, January 22–29). Attribute recognition by joint recurrent learning of context and correlation. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.65 ↗
6. Pedestrian attributes recognition in surveillance scenarios with hierarchical multi-task CNN models
7. The impact of color space and intensity normalization to face detection performance
8. Han, H., and Tong, M. (2013, January 26–28). Human detection based on optical flow and spare geometric flow. Proceedings of the 2013 IEEE Seventh International Conference on Image and Graphics, Qingdao, China.10.1109/ICIG.2013.96 ↗
9. Hoang, V.D., Hernandez, D.C., and Jo, K.H. (2014, January 3–6). Partially obscured human detection based on component detectors using multiple feature descriptors. Proceedings of the International Conference on Intelligent Computing, Taiyuan, China.10.1007/978-3-319-09333-8_37 ↗
10. Real-time human detection for aerial captured video sequences via deep models10.1155/2018/1639561 ↗
11. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
12. Girshick, R. (2015, January 7–13). Fast r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.169 ↗
13. Faster r-cnn: Towards real-time object detection with region proposal networks
14. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask r-cnn. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.322 ↗
15. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11–14). Ssd: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
16. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
17. Human detection techniques for real time surveillance: A comprehensive survey10.1007/s11042-020-10103-4 ↗
18. Gray, D., and Tao, H. (2008, January 12–18). Viewpoint invariant pedestrian recognition with an ensemble of localized features. Proceedings of the European Conference on Computer Vision, Marseille, France.10.1007/978-3-540-88682-2_21 ↗
19. Person re-identification by support vector ranking
20. Person re-identification by attributes
21. Layne, R., Hospedales, T.M., and Gong, S. (2014). Attributes-based re-identification. Person Re-Identification, Springer.10.1007/978-1-4471-6296-4_5 ↗
22. Deng, Y., Luo, P., Loy, C.C., and Tang, X. (2014, January 3–7). Pedestrian attribute recognition at far distance. Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, FL, USA.10.1145/2647868.2654966 ↗
23. Li, D., Zhang, Z., Chen, X., Ling, H., and Huang, K. (2016). A richly annotated dataset for pedestrian attribute recognition. arXiv.
24. Vaquero, D.A., Feris, R.S., Tran, D., Brown, L., Hampapur, A., and Turk, M. (2009, January 7–8). Attribute-based people search in surveillance environments. Proceedings of the 2009 IEEE Workshop on Applications of Computer Vision (WACV), Snowbird, UT, USA.10.1109/WACV.2009.5403131 ↗
25. Sudowe, P., Spitzer, H., and Leibe, B. (2015, January 7–13). Person attribute recognition with a jointly-trained holistic cnn model. Proceedings of the IEEE International Conference on Computer Vision Workshops, Santiago, Chile.10.1109/ICCVW.2015.51 ↗
26. Zhu, J., Liao, S., Yi, D., Lei, Z., and Li, S.Z. (2015, January 19–22). Multi-label cnn based pedestrian attribute learning for soft biometrics. Proceedings of the 2015 IEEE International Conference on Biometrics (ICB), Phuket, Thailand.10.1109/ICB.2015.7139070 ↗
27. Matsukawa, T., and Suzuki, E. (2016, January 4–8). Person re-identification using CNN features learned from combination of attributes. Proceedings of the 2016 IEEE 23rd International Conference on Pattern Recognition (ICPR), Cancun, Mexico.10.1109/ICPR.2016.7900000 ↗
28. Kurnianggoro, L., and Jo, K.H. (November, January 29). Identification of pedestrian attributes using deep network. Proceedings of the IECON 2017-43rd Annual Conference of the IEEE Industrial Electronics Society, Beijing, China.10.1109/IECON.2017.8217493 ↗
29. Liu, P., Liu, X., Yan, J., and Shao, J. (2018). Localization guided learning for pedestrian attribute recognition. arXiv.
30. Han, K., Wang, Y., Shu, H., Liu, C., Xu, C., and Xu, C. (2019). Attribute aware pooling for pedestrian attribute recognition. arXiv.10.24963/ijcai.2019/341 ↗
31. Attention-based pedestrian attribute analysis10.1109/TIP.2019.2919199 ↗
32. Li, Y., Xu, H., Bian, M., and Xiao, J. (2020). Attention based CNN-ConvLSTM for pedestrian attribute recognition. Sensors, 20.10.3390/s20030811 ↗
33. Zeng, H., Ai, H., Zhuang, Z., and Chen, L. (2020, January 6–10). Multi-Task Learning Via Co-Attentive Sharing For Pedestrian Attribute Recognition. Proceedings of the 2020 IEEE International Conference on Multimedia and Expo (ICME), London, UK.10.1109/ICME46284.2020.9102757 ↗
34. Image-attribute reciprocally guided attention network for pedestrian attribute recognition10.1016/j.patrec.2019.01.010 ↗
35. Yu, K., Leng, B., Zhang, Z., Li, D., and Huang, K. (2016). Weakly-supervised learning of mid-level features for pedestrian attribute recognition and localization. arXiv.
36. He, K., Wang, Z., Fu, Y., Feng, R., Jiang, Y.G., and Xue, X. (2017, January 23–27). Adaptively weighted multi-task deep network for person attribute classification. Proceedings of the 25th ACM International Conference on Multimedia, Mountain View, CA, USA.10.1145/3123266.3123424 ↗
37. Task-aware attention model for clothing attribute prediction10.1109/TCSVT.2019.2902268 ↗
38. Bharati, P., and Pramanik, A. (2020). Deep learning techniques—R-CNN to mask R-CNN: A survey. Computational Intelligence in Pattern Recognition, Springer.10.1007/978-981-13-9042-5_56 ↗
39. Clothing attribute recognition based on RCNN framework using L-Softmax loss10.1109/ACCESS.2020.2979164 ↗
40. Yu, W., Kim, S., Chen, F., and Choi, J. (2020). Pedestrian Detection Based on Improved Mask R-CNN Algorithm. International Conference on Intelligent and Fuzzy Systems, Springer.10.1007/978-3-030-51156-2_176 ↗
41. (2021, April 13). GitHub—Facebookresearch/Detectron: FAIR’s Research Platform for Object Detection Research, Implementing Popular Algorithms Like Mask R-CNN and RetinaNet. Available online: https://github.com/facebookresearch/Detectron.
42. (2021, July 01). Keras Applications. Available online: https://keras.io/api/applications/.
43. A richly annotated pedestrian dataset for person retrieval in real surveillance scenarios10.1109/TIP.2018.2878349 ↗
44. C4. 5, class imbalance, and cost sensitivity: Why under-sampling beats over-sampling
45. Zhu, J., Liao, S., Lei, Z., Yi, D., and Li, S. (2013, January 2–8). Pedestrian attribute classification in surveillance: Database and evaluation. Proceedings of the IEEE International Conference on Computer Vision Workshops, Sydney, Australia.10.1109/ICCVW.2013.51 ↗
46. Evaluating appearance models for recognition, reacquisition, and tracking
47. Liu, X., Zhao, H., Tian, M., Sheng, L., Shao, J., Yi, S., Yan, J., and Wang, X. (2017, January 22–29). Hydraplus-net: Attentive deep features for pedestrian analysis. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.46 ↗
48. Sarafianos, N., Xu, X., and Kakadiaris, I.A. (2018, January 8–14). Deep imbalanced attribute classification using visual attention aggregation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01252-6_42 ↗
49. Guo, H., Zheng, K., Fan, X., Yu, H., and Wang, S. (2019, January 15–20). Visual attention consistency under image transforms for multi-label image classification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00082 ↗
50. Nature News & Comment (2021, November 06). Can We Open the Black Box of AI?. Available online: https://www.nature.com/news/can-we-open-the-black-box-of-ai-1.20731.