Transformers in Pedestrian Image Retrieval and Person Re-Identification in a Multi-Camera Surveillance System
Source abstract
Abstract
Person Re-Identification is an essential task in computer vision, particularly in surveillance applications. The aim is to identify a person based on an input image from surveillance photographs in various scenarios. Most Person re-ID techniques utilize Convolutional Neural Networks (CNNs); however, Vision Transformers are replacing pure CNNs for various computer vision tasks such as object recognition, classification, etc. The vision transformers contain information about local regions of the image. The current techniques take this advantage to improve the accuracy of the tasks underhand. We propose to use the vision transformers in conjunction with vanilla CNN models to investigate the true strength of transformers in person re-identification. We employ three backbones with different combinations of vision transformers on two benchmark datasets. The overall performance of the backbones increased, showing the importance of vision transformers. We provide ablation studies and show the importance of various components of the vision transformers in re-identification tasks.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Advanced Neural Network Applications · Human Pose and Action Recognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computing and Informatics, Saudi Electronic University, Riyadh 11673, Saudi Arabia
Data61-Commonwealth Scientific and Industrial Research Organization(CSIRO), Clayton South, VIC 3169, Australia
College of Engineering and Computer Science, Australian National University, Canberra, ACT 2601, Australia
School of Computer Science, University of Technology Sydney, Sydney, NSW 2007, Australia
Funding
Deanship of Scientific Research at Saudi Electronic University, Riyadh, Saudi Arabia
References (46)
Show source-provided references
1. Zheng, L., Yang, Y., and Hauptmann, A.G. (2016). Person re-identification: Past, present and future. arXiv.
2. Bai, X., Yang, M., Huang, T., Dou, Z., Yu, R., and Xu, Y. (2017). Deep-person: Learning discriminative deep features for person re-identification. arXiv.
3. Yan, Y., Zhang, Q., Ni, B., Zhang, W., Xu, M., and Yang, X. (2019, January 15–20). Learning Context Graph for Person Search. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00226 ↗
4. Protecting Water Infrastructure From Cyber and Physical Threats: Using Multimodal Data Fusion and Adaptive Deep Learning to Monitor Critical Systems10.1109/MSP.2018.2885359 ↗
5. Xu, Y., Ma, B., Huang, R., and Lin, L. (2014, January 3–7). Person search in a scene by jointly modeling people commonness and person uniqueness. Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, FL, USA.10.1145/2647868.2654965 ↗
6. Dai, Z., Chen, M., Zhu, S., and Tan, P. (2018). Batch feature erasing for person re-identification and beyond. arXiv.10.1109/ICCV.2019.00379 ↗
7. Huang, H., Yang, W., Chen, X., Zhao, X., Huang, K., Lin, J., Huang, G., and Du, D. (2018). EANet: Enhancing Alignment for Cross-Domain Person Re-identification. arXiv.
8. Zheng, Z., Yang, X., Yu, Z., Zheng, L., Yang, Y., and Kautz, J. (2019, January 15–20). Joint discriminative and generative learning for person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00224 ↗
9. Wang, G., Lai, J., Huang, P., and Xie, X. (February, January 27). Spatial-temporal person re-identification. Proceedings of the AAAI Conference on Artificial Intelligence, Hilton Hawaiian Village, Honolulu, HI, USA.
10. Attention driven person re-identification10.1016/j.patcog.2018.08.015 ↗
11. Adaimi, G., Kreiss, S., and Alahi, A. (2019). Rethinking Person Re-Identification with Confidence. arXiv.
12. Wieczorek, M., Rychalska, B., and Dabrowski, J. (2021). On the Unreasonable Effectiveness of Centroids in Image Retrieval. arXiv.10.1007/978-3-030-92273-3_18 ↗
13. Ye, M., Shen, J., Lin, G., Xiang, T., Shao, L., and Hoi, S.C. (2021). Deep learning for person re-identification: A survey and outlook. IEEE Trans. Pattern Anal. Mach. Intell.10.1109/TPAMI.2021.3054775 ↗
14. Wang, H., Fan, Y., Wang, Z., Jiao, L., and Schiele, B. (2018). Parameter-Free Spatial Attention Network for Person Re-Identification. arXiv.
15. Wojke, N., and Bewley, A. (2018, January 12–15). Deep cosine metric learning for person re-identification. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.10.1109/WACV.2018.00087 ↗
16. Zhong, Z., Zheng, L., Zheng, Z., Li, S., and Yang, Y. (2018, January 18–23). Camera style adaptation for person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00541 ↗
17. Zheng, F., Deng, C., Sun, X., Jiang, X., Guo, X., Yu, Z., Huang, F., and Ji, R. (2019, January 15–20). Pyramidal Person Re-IDentification via Multi-Loss Dynamic Training. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00871 ↗
18. Luo, H., Gu, Y., Liao, X., Lai, S., and Jiang, W. (2019, January 16–17). Bag of tricks and a strong baseline for deep person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA.10.1109/CVPRW.2019.00190 ↗
19. Quan, R., Dong, X., Wu, Y., Zhu, L., and Yang, Y. (2019). Auto-ReID: Searching for a Part-aware ConvNet for Person Re-Identification. arXiv.10.1109/ICCV.2019.00385 ↗
20. Ro, Y., Choi, J., Jo, D.U., Heo, B., Lim, J., and Choi, J.Y. (2019). Backbone Can Not be Trained at Once: Rolling Back to Pre-trained Network for Person Re-identification. arXiv.10.1609/aaai.v33i01.33018859 ↗
21. Zeng, Z., Wang, Z., Wang, Z., Chuang, Y.Y., and Satoh, S. (2019). Illumination-Adaptive Person Re-identification. arXiv.
22. Zhang, S., Yin, Z., Wu, X., Wang, K., Zhou, Q., and Kang, B. (2021). FPB: Feature Pyramid Branch for Person Re-Identification. arXiv.
23. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
24. Sharma, C., Kapil, S.R., and Chapman, D. (2021). Person Re-Identification with a Locally Aware Transformer. arXiv.
25. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv.
26. Yunpeng, G. (2021). A general multi-modal data learning method for Person Re-identification. arXiv.
27. Wang, D., and Zhang, S. (2020, January 13–19). Unsupervised person re-identification via multi-label classification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01099 ↗
28. Shu, X., Wang, X., Zhang, S., Zhang, X., Chen, Y., Li, G., and Tian, Q. (2021). Large-Scale Spatio-Temporal Person Re-identification: Algorithm and Benchmark. arXiv.
29. Jin, H., Wang, X., Liao, S., and Li, S.Z. (2017, January 1–4). Deep person re-identification with improved embedding and efficient training. Proceedings of the 2017 IEEE International Joint Conference on Biometrics (IJCB), Denver, CO, USA.10.1109/BTAS.2017.8272706 ↗
30. Xiao, T., Li, S., Wang, B., Lin, L., and Wang, X. (2017, January 21–26). Joint detection and identification feature learning for person search. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.360 ↗
31. Signature Verification using a “Siamese” Time Delay Neural Network10.1142/S0218001493000339 ↗
32. Ge, Y., Li, Z., Zhao, H., Yin, G., Yi, S., Wang, X., and li, H. (2018, January 3–8). FD-GAN: Pose-guided feature distilling GAN for robust person re-identification. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada.
33. Wang, G., Yuan, Y., Chen, X., Li, J., and Zhou, X. (2018, January 22–26). Learning discriminative features with multiple granularities for person re-identification. Proceedings of the 2018 ACM Multimedia Conference on Multimedia Conference, Seoul, Korea.10.1145/3240508.3240552 ↗
34. Zhong, Z., Zheng, L., Luo, Z., Li, S., and Yang, Y. (2019, January 15–20). Invariance matters: Exemplar memory for domain adaptive person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00069 ↗
35. Zhou, K., Yang, Y., Cavallaro, A., and Xiang, T. (2019). Omni-Scale Feature Learning for Person Re-Identification. arXiv.10.1109/ICCV.2019.00380 ↗
36. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
37. Sun, Y., Zheng, L., Yang, Y., Tian, Q., and Wang, S. (2018, January 8–14). Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01225-0_30 ↗
38. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA.
39. Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., and Tian, Q. (2015, January 7–13). Scalable person re-identification: A benchmark. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.133 ↗
40. Ristani, E., Solera, F., Zou, R., Cucchiara, R., and Tomasi, C. (2016, January 8–16). Performance measures and a data set for multi-target, multi-camera tracking. Proceedings of the European Conference on Computer Vision, Workshops, Amsterdam, The Netherlands.10.1007/978-3-319-48881-3_2 ↗
41. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Li, F.-F. (2009, January 20–25). Imagenet: A large-scale hierarchical image database. Proceedings of the Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
42. Tahir, M., Anwar, S., and Mian, A. (2018). Deep localization of protein structures in fluorescence microscopy images. arXiv.
43. Deep ancient Roman Republican coin classification via feature fusion and attention10.1016/j.patcog.2021.107871 ↗
44. Chattopadhay, A., Sarkar, A., Howlader, P., and Balasubramanian, V.N. (2018, January 12–15). Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. Proceedings of the Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.10.1109/WACV.2018.00097 ↗
45. Wang, H., Wang, Z., Du, M., Yang, F., Zhang, Z., Ding, S., Mardziel, P., and Hu, X. (2020, January 14–19). Score-CAM: Score-weighted visual explanations for convolutional neural networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.1109/CVPRW50498.2020.00020 ↗
46. Muhammad, M.B., and Yeasin, M. (2020, January 19–24). Eigen-CAM: Class Activation Map using Principal Components. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Glasgow, UK.10.1109/IJCNN48605.2020.9206626 ↗