CNN Attention Enhanced ViT Network for Occluded Person Re-Identification
Source abstract
Abstract
Person re-identification (ReID) is often affected by occlusion, which makes most of the features extracted by ReID models contain a lot of identity-independent noise. Recently, the use of Vision Transformer (ViT) has enabled significant progress in various visual artificial intelligence tasks. However, ViT suffers from insufficient local information extraction capability, which should be of concern to researchers in the field of occluded ReID. This paper conducts a study to exploit the potential of attention mechanisms to enhance ViT in ReID tasks. In this study, an Attention Enhanced ViT Network (AET-Net) is proposed for occluded ReID. We use ViT as the backbone network to extract image features. Even so, occlusion and outlier problems still exist in ReID. Then, we combine the spatial attention mechanism into the ViT architecture, by which we enhance the attention of ViT patch embedding vectors to important regions. In addition, we design a MultiFeature Training Module to optimize the network by the construction of multiple classification features and calculation of the multi-feature loss to enhance the performance of the model. Finally, the effectiveness and superiority of the proposed method are demonstrated by broad experiments on both occluded and non-occluded datasets.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Advanced Neural Network Applications · Human Pose and Action Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Shanghai Marine Intelligent Information and Navigation Remote Sensing Engineering Technology Research Center, Shanghai Ocean University, Shanghai 201306, China
Funding
National Natural Science Foundation of China · National Key R&D Program of China
References (52)
Show source-provided references
1. Deep learning for person re-identification: A survey and outlook10.1109/TPAMI.2021.3054775 ↗
2. Wang, G.A., Yang, S., and Liu, H. (2020, January 14–19). High-order information matters: Learning relation and topology for occluded person re-identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00648 ↗
3. Wang, P., Ding, C., Shao, Z., Hong, Z., Zhang, S., and Tao, D. (2022). Quality-aware part models for occluded person re-identification. IEEE Trans. Multimed.10.1109/TMM.2022.3156282 ↗
4. Huang, H., Li, D., and Zhang, Z. (2018, January 18–22). Adversarial occluded samples for person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00535 ↗
5. Jia, M., Cheng, X., and Lu, S. (2022). Learning disentangled representation implicitly via transformer for occluded person re-identification. IEEE Trans. Multimed.10.1109/TMM.2022.3141267 ↗
6. Matching on sets: Conquer occluded person re-identification without alignment
7. Miao, J., Wu, Y., and Liu, P. (November, January 27). Pose-guided feature alignment for occluded person re-identification. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
8. Li, Y., He, J., and Zhang, T. (2021, January 20–25). Diverse part discovery: Occluded person re-identification with part-aware transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00292 ↗
9. He, L., Wang, Y., and Liu, W. (November, January 27). Foreground-aware pyramid reconstruction for alignment-free occluded person re-identification. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
10. Bi-directional Style Adaptation Network for Person Re-Identification10.1109/JSEN.2022.3173534 ↗
11. Deep Batch Active Learning and Knowledge Distillation for Person Re-Identification10.1109/JSEN.2022.3181238 ↗
12. Wu, Y., Lin, Y., and Dong, X. (2018, January 18–22). Exploit the unknown gradually: One-shot video-based person re-identification by stepwise learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00543 ↗
13. Progressive learning for person re-identification with one example10.1109/TIP.2019.2891895 ↗
14. Unsupervised person re-identification via cross-camera similarity exploration10.1109/TIP.2020.2982826 ↗
15. Cai, H., Wang, Z., and Cheng, J. (2019, January 16–17). Multi-scale body-part mask guided attention for person re-identification. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA.10.1109/CVPRW.2019.00197 ↗
16. Song, C., Huang, Y., and Ouyang, W. (2018, January 18–22). Mask-guided contrastive attention model for person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00129 ↗
17. Segmentation mask-guided person image generation10.1007/s10489-020-01907-w ↗
18. Zhao, H., Tian, M., and Sun, S. (2017, January 21–26). Spindle net: Person re-identification with human body region guided feature decomposition and fusion. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.103 ↗
19. Su, C., Li, J., and Zhang, S. (2017, January 22–29). Pose-driven deep convolutional model for person re-identification. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.427 ↗
20. Kalayeh, M.M., Basaran, E., and Gökmen, M. (2018, January 18–22). Human semantic parsing for person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00117 ↗
21. Zhu, K., Guo, H., and Liu, Z. (2020, January 23–28). Identity-guided human semantic parsing for person re-identification. Proceedings of the European Conference on Computer Vision, Glasgow, UK.10.1007/978-3-030-58580-8_21 ↗
22. Sun, Y., Zheng, L., and Yang, Y. (2018, January 8–14). Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01225-0_30 ↗
23. Wang, G., Yuan, Y., and Chen, X. (2018, January 22–26). Learning discriminative features with multiple granularities for person re-identification. Proceedings of the 26th ACM International Conference on Multimedia, Seoul, Republic of Korea.10.1145/3240508.3240552 ↗
24. He, S., Luo, H., and Wang, P. (2021, January 11–17). Transreid: Transformer-based object re-identification. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.01474 ↗
25. Zhang, Z., Lan, C., and Zeng, W. (2020, January 13–19). Relation-aware global attention for person re-identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00325 ↗
26. Chen, B., Deng, W., and Hu, J. (November, January 27). Mixed high-order attention network for person re-identification. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
27. Vaswani, A., Shazeer, N., and Parmar, N. (2017, January 4–9). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA.
28. Attention in natural language processing10.1109/TNNLS.2020.3019893 ↗
29. Zhao, J., Wang, H., and Zhou, Y. (2022). Spatial-Channel Enhanced Transformer for Visible-Infrared Person Re-Identification. IEEE Trans. Multimed.10.1109/TMM.2022.3163847 ↗
30. Transformers in vision: A survey10.1145/3505244 ↗
31. Liu, Y., Zhang, Y., and Wang, Y. (2021). A survey of visual transformers. arXiv.
32. Person re-identification in aerial imagery10.1109/TMM.2020.2977528 ↗
33. Yang, W., Huang, H., and Zhang, Z. (2019, January 15–20). Towards rich feature discovery with class activation maps augmentation for person re-identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00148 ↗
34. Zhao, L., Li, X., and Zhuang, Y. (2017, January 22–29). Deeply-learned part-aligned representations for person re-identification. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.349 ↗
35. He, L., Liang, J., and Li, H. (2018, January 18–22). Deep spatial feature reconstruction for partial person re-identification: Alignment-free approach. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00739 ↗
36. Zhang, X., Luo, H., and Fan, X. (2017). Alignedreid: Surpassing human-level performance in person re-identification. arXiv.
37. Attention mechanisms in computer vision: A survey10.1007/s41095-022-0271-y ↗
38. Li, W., Zhu, X., and Gong, S. (2018, January 18–22). Harmonious attention network for person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00243 ↗
39. Dosovitskiy, A., Beyer, L., and Kolesnikov, A. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
40. Wu, B., Xu, C., and Dai, X. (2020). Visual transformers: Token-based image representation and processing for computer vision. arXiv.
41. Srinivas, A., Lin, T.-Y., and Parmar, N. (2021, January 20–25). Bottleneck transformers for visual recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01625 ↗
42. Liu, Z., Lin, Y., and Cao, Y. (2021, January 11–17). Swin transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCV48922.2021.00986 ↗
43. Touvron, H., Cord, M., and Douze, M. (2021, January 18–24). Training data-efficient image transformers & distillation through attention. Proceedings of the International Conference on Machine Learning, Virtual.
44. D’Ascoli, S., Touvron, H., and Leavitt, M.L. (2021, January 18–24). Convit: Improving vision transformers with soft convolutional inductive biases. Proceedings of the International Conference on Machine Learning, Virtual.
45. ResT: An efficient transformer for visual recognition
46. Woo, S., Park, J., and Lee, J.Y. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
47. Multi-view feature fusion for person re-identification10.1016/j.knosys.2021.107344 ↗
48. Zheng, L., Shen, L., and Tian, L. (2015, January 7–13). Scalable person re-identification: A benchmark. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.133 ↗
49. Zheng, Z., Zheng, L., and Yang, Y. (2017, January 22–29). Unlabeled samples generated by gan improve the person re-identification baseline in vitro. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.405 ↗
50. Wang, X., Doretto, G., and Sebastian, T. (2007, January 14–21). Shape and appearance context modeling. Proceedings of the 2007 IEEE 11th International conference on Computer Vision, Rio De Janeiro, Brazil.10.1109/ICCV.2007.4409019 ↗
51. Identifying visible parts via pose estimation for occluded person re-identification10.1109/TNNLS.2021.3059515 ↗
52. Luo, H., Gu, Y., and Liao, X. (2019, January 16–17). Bag of tricks and a strong baseline for deep person re-identification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA.10.1109/CVPRW.2019.00190 ↗