Parts Semantic Segmentation Aware Representation Learning for Person Re-Identification
Source abstract
Abstract
Person re-identification is a typical computer vision problem which aims at matching pedestrians across disjoint camera views. It is challenging due to the misalignment of body parts caused by pose variations, background clutter, detection errors, camera point of view variation, different accessories and occlusion. In this paper, we propose a person re-identification network which fuses global and local features, to deal with part misalignment problem. The network is a four-branch convolutional neural network (CNN) which learns global person appearance and local features of three human body parts respectively. Local patches, including the head, torso and lower body, are segmented by using a U_Net semantic segmentation CNN architecture. All four feature maps are then concatenated and fused to represent a person image. We propose a DropParts method to solve the parts missing problem, with which the local features are weighed according to the number of parts found by semantic segmentation. Since three body parts are well aligned, the approach significantly improves person re-identification. Experiments on the standard benchmark datasets, such as Market1501, CUHK03 and DukeMTMC-reID datasets, show the effectiveness of our proposed pipeline.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Advanced Neural Network Applications · Human Pose and Action Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer Science, Zhejiang University of Technology, Hangzhou 310014, China
College of Computer Science and Engineering, Tianjin University of Technology, Tianjin 300384, China
Zhejiang Jieshang Vision Technology Co., Ltd., Hangzhou 311121, China
Collaborative Innovation Center for Economics Crime Investigation and Prevention Technology, Nanchang 330000, China
References (48)
Show source-provided references
1. Review of person re-identification techniques10.1049/iet-cvi.2013.0180 ↗
2. A survey of approaches and trends in person re-identification10.1016/j.imavis.2014.02.001 ↗
3. Tong, X., Li, H., Ouyang, W., and Wang, X. (July, January 26). Learning deep feature representations with domain guided dropout for person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
4. A Siamese inception architecture network for person re-identification10.1007/s00138-017-0843-5 ↗
5. Su, C., Li, J., Zhang, S., Xing, J., Gao, W., and Tian, Q. (2017, January 22–29). Pose-driven deep convolutional model for person re-identification. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.427 ↗
6. Wang, G., Yuan, Y., Chen, X., Li, J., and Zhou, X. (2018, January 22–26). Learning discriminative features with multiple granularities for person re-identification. Proceedings of the 26th ACM International Conference on Multimedia, Seoul, Korea.10.1145/3240508.3240552 ↗
7. Zhang, X., Luo, H., Fan, X., Xiang, W., Sun, Y., Xiao, Q., Jiang, W., Zhang, C., and Sun, J. (arXiv, 2017). AlignedReID: Surpassing human-level performance in person re-identification, arXiv.
8. Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., and Tian, Q. (2015, January 13–16). Scalable person re-identification: A benchmark. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.133 ↗
9. Zheng, Z., Zheng, L., and Yang, Y. (2017, January 22–29). Unlabeled samples generated by gan improve the person re-identification baseline in vitro. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.405 ↗
10. Yi, D., Lei, Z., Liao, S., and Li, S.Z. (2014, January 24–28). Deep metric learning for person re-identification. Proceedings of the 22nd International Conference on Pattern Recognition, Stockholm, Sweden.10.1109/ICPR.2014.16 ↗
11. Sarfraz, M.S., Schumann, A., Eberle, A., and Stiefelhagen, R. (2018, January 19–21). A pose-sensitive embedding for person re-identification with expanded cross neighborhood re-ranking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00051 ↗
12. Navab, N., Hornegger, J., Wells, W.M., and Frangi, A.F. (2015). U-Net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention 2015, Springer International Publishing.10.1007/978-3-319-24553-9 ↗
13. Miri, K., Jaehoon, J., Hyuncheol, K., and Joonki, P. (2017, January 9–11). Person re-identification using color name descriptor-based sparse representation. Proceedings of the IEEE 7th Annual Computing and Communication Workshop and Conference, Las Vegas, NV, USA.10.1109/CCWC.2017.7868394 ↗
14. Zhao, R., Ouyang, W., and Wang, X. (2013, January 23–28). Unsupervised salience learning for person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.460 ↗
15. Li, W., and Wang, X. (2013, January 23–28). Locally aligned feature transforms across views. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.461 ↗
16. Liao, S., Hu, Y., Zhu, X., and Li, S.Z. (2015, January 8–10). Person re-identification by local maximal occurrence representation and metric learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298832 ↗
17. Li, W., Zhao, R., Xiao, T., and Wang, X. (2014, January 24–27). DeepReID: Deep filter pairing neural network for person re-Identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.27 ↗
18. Wu, L., Wang, Y., Gao, J., and Li, X. (2018). Where-and-when to look: Deep siamese attention networks for video-based person re-identification. IEEE Trans. Multimed.10.1109/TMM.2018.2877886 ↗
19. Alex, D., Sami, Z., Banerjee, S., and Panda, S. (arXiv, 2018). Cluster loss for person re-identification, arXiv.10.1145/3293353.3293396 ↗
20. Multilevel triplet deep learning model for person re-identification10.1016/j.patrec.2018.04.029 ↗
21. A novel deep model with multi-loss and efficient training for person re-identification10.1016/j.neucom.2018.03.073 ↗
22. Zhao, H., Tian, M., Sun, S., Shao, J., Yan, J., Yi, S., Wang, X., and Tang, X. (2017, January 21–26). Spindle Net: Person re-identification with human body region guided feature decomposition and fusion. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.103 ↗
23. Ahmed, E., Jones, M., and Marks, T.K. (2015, January 8–10). An improved deep learning architecture for person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7299016 ↗
24. Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A.C., Salakhutdinov, R., Zemel, R.S., and Bengio, Y. (arXiv, 2015). Show, Attend and tell: Neural image caption generation with visual attention, arXiv.
25. Zhao, L., Li, X., Zhuang, Y., and Wang, J. (2017, January 22–29). Deeply-learned part-aligned representations for person re-identification. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.349 ↗
26. Wu, L., Shen, C., and van den Hengel, A. (arXiv, 2016). PersonNet: Person re-identification with deep convolutional neural networks, arXiv.
27. Liu, X., Zhao, H., Tian, M., Sheng, L., Shao, J., Yi, S., Yan, J., and Wang, X. (2017, January 22–29). HydraPlus-Net: Attentive deep features for pedestrian analysis. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.46 ↗
28. Li, W., Zhu, X., and Gong, S. (2018, January 19–21). Harmonious attention network for person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00243 ↗
29. Xu, J., Zhao, R., Zhu, F., Wang, H., and Ouyang, W. (2018, January 19–21). Attention-aware compositional network for person re-identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00226 ↗
30. Metric learning: A survey10.1561/2200000019 ↗
31. Dropout: A simple way to prevent neural networks from overfitting
32. Sergey, I., and Christian, S. (arXiv, 2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift, arXiv.
33. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
34. Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Science, T.D.J.C. (arXiv, 2014). Caffe: Convolutional architecture for fast feature embedding, arXiv.10.1145/2647868.2654889 ↗
35. Neural networks: Tricks of the trade
36. Gong, K., Liang, X., Zhang, D., Shen, X., and Lin, L. (2017, January 21–26). Look into Person: Self-Supervised Structure-Sensitive Learning and a New Benchmark for Human Parsing. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.715 ↗
37. Segnet: A deep convolutional encoder-decoder architecture for image segmentation10.1109/TPAMI.2016.2644615 ↗
38. Fully convolutional networks for semantic segmentation10.1109/TPAMI.2016.2572683 ↗
39. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs10.1109/TPAMI.2017.2699184 ↗
40. Chen, L.-C., Yang, Y., Wang, J., Xu, W., and Yuille, A.L. (2016, January 27–30). Attention to scale: Scale-aware semantic image segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.396 ↗
41. Felzenszwalb, P., McAllester, D., and Ramanan, D. (2008, January 24–26). A discriminatively trained, multiscale, deformable part model. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA.10.1109/CVPR.2008.4587597 ↗
42. Zhang, Y., Li, X., Zhao, L., and Zhang, Z. (2016, January 9–15). Semantics-aware deep correspondence structure learning for robust person re-identification. Proceedings of the International Joint Conference on Artificial Intelligence, New York, NY, USA.
43. Sun, Y., Zheng, L., Deng, W., and Wang, S. (2017, January 22–29). SVDNet for pedestrian retrieval. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.410 ↗
44. Zheng, Z., Zheng, L., and Yang, Y. (2018). Pedestrian alignment network for large-scale person re-identification. IEEE Trans. Circuits Syst. Video Technol.10.1109/TCSVT.2018.2873599 ↗
45. Ristani, E., and Tomasi, C. (2018, January 19–21). Features for multi-target multi-camera tracking and re-Identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00632 ↗
46. Simonyan, K., and Zisserman, A. (arXiv, 2014). Very deep convolutional networks for large-scale image recognition, arXiv.
47. Huang, G., Liu, Z., Maaten, L.v.d., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
48. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27–30). Rethinking the Inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.308 ↗