SiamCAM: A Real-Time Siamese Network for Object Tracking with Compensating Attention Mechanism
Source abstract
Abstract
The Siamese-based object tracking algorithm regards tracking as a similarity matching problem. It determines the object location according to the response value of the object template to the search template. When there is similar object interference in complex scenes, it is easy to cause tracking drift. We propose a real-time Siamese network object tracking algorithm combined with a compensating attention mechanism to solve this problem. Firstly, the attention mechanism is introduced in the feature extraction module of the template branch and search branch of the Siamese network to improve the feature representation of the network to the object. The attention mechanism of the search branch enhances the feature representation of both the target and the similar backgrounds simultaneously. Therefore, based on the above two-branch attention, we propose a compensated attention model, which introduces the attention selected by the template branch into the search branch, and improves the discriminative ability of the search branch to the object by using the feature attention weighting of the template branch to the object. Experimental results on three popular benchmarks, including OTB2015, VOT2018, and LaSOT, show that the accuracy and robustness of the algorithm in this paper are adequate. It improved occlusion cases, similar object interference, and high-speed motion. The processing speed on GPU reaches 47 fps, which can achieve real-time object tracking.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Visual Attention and Saliency Detection · Advanced Neural Network Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Jiangxi Key Laboratory of Image Processing and Pattern Recognition, Nanchang Hangkong University, Nanchang 330063, China
College of Softerware, Nanchang Hangkong University, Nanchang 330063, China
References (51)
Show source-provided references
1. Xing, J., Ai, H., and Lao, S. (2010, January 23–26). Multiple human tracking based on multi-view upper-body detection and discriminative learning. Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey.10.1109/ICPR.2010.420 ↗
2. Yuan, D., Li, Q., Yang, X., Zhang, M., and Sun, Z. (2022). Object-Aware Adaptive Convolution Kernel Attention Mechanism in Siamese Network for Visual Tracking. Appl. Sci., 12.10.3390/app12020716 ↗
3. Luo, S., Li, B., Yuan, X., and Liu, H. (2021). Robust Long-Term Visual Object Tracking via Low-Rank Sparse Learning for Re-Detection. Appl. Sci., 11.10.3390/app11041963 ↗
4. Perez-Cham, O.E., Puente, C., Soubervielle-Montalvo, C., Olague, G., Aguirre-Salado, C.A., and Nuñez-Varela, A.S. (2020). Parallelization of the honeybee search algorithm for object tracking. Appl. Sci., 10.10.3390/app10062122 ↗
5. Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Kämäräinen, J.K., Danelljan, M., Zajc, L.Č., Lukežič, A., and Drbohlav, O. (2020, January 23–28). The eighth visual object tracking VOT2020 challenge results. Proceedings of the European Conference on Computer Vision, Virtual.
6. Cheng, S., Zhong, B., Li, G., Liu, X., Tang, Z., Li, X., and Wang, J. (2021, January 15–19). Learning to Filter: Siamese Relation Network for Robust Tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual.10.1109/CVPR46437.2021.00440 ↗
7. Zhang, Z., Liu, Y., Wang, X., Li, B., and Hu, W. (2021, January 11–17). Learn to match: Automatic matching network design for visual tracking. Proceedings of the IEEE/CVF International Conference on Computer Vision, Virtual.10.1109/ICCV48922.2021.01309 ↗
8. Danelljan, M., Bhat, G., Khan, F.S., and Felsberg, M. (, January 16–20). Atom: Accurate tracking by overlap maximization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.
9. Jiang, B., Luo, R., Mao, J., Xiao, T., and Jiang, Y. (2018, January 8–14). Acquisition of localization confidence for accurate object detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01264-9_48 ↗
10. Bhat, G., Danelljan, M., Gool, L.V., and Timofte, R. (November, January 27). Learning discriminative model prediction for tracking. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.
11. Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., and Torr, P.H. (2016, January 8–16). Fully-convolutional siamese networks for object tracking. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-48881-3_56 ↗
12. Tao, R., Gavves, E., and Smeulders, A.W. (2016, January 17–22). Siamese instance search for tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, NY, USA.10.1109/CVPR.2016.158 ↗
13. Li, B., Yan, J., Wu, W., Zhu, Z., and Hu, X. (2018, January 18–23). High performance visual tracking with siamese region proposal network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00935 ↗
14. Faster r-cnn: Towards real-time object detection with region proposal networks
15. Zhu, Z., Wang, Q., Li, B., Wu, W., Yan, J., and Hu, W. (2018, January 8–14). Distractor-aware siamese networks for visual object tracking. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01240-3_7 ↗
16. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Fontainebleau Resort, Miami Beach, Florida, USA.10.1109/CVPR.2009.5206848 ↗
17. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 6–12). Microsoft coco: Common objects in context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
18. Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., and Yan, J. (2019, January 16–20). Siamrpn++: Evolution of siamese visual tracking with very deep networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00441 ↗
19. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 17–22). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, NY, USA.10.1109/CVPR.2016.90 ↗
20. Imagenet classification with deep convolutional neural networks
21. Wang, Q., Zhang, L., Bertinetto, L., Hu, W., and Torr, P.H. (2019, January 16–20). Fast online object tracking and segmentation: A unifying approach. Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00142 ↗
22. Guo, Q., Feng, W., Zhou, C., Huang, R., Wan, L., and Wang, S. (2017, January 22–29). Learning dynamic siamese network for visual object tracking. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.196 ↗
23. Yang, T., and Chan, A.B. (2018, January 8–14). Learning dynamic memory networks for object tracking. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01240-3_10 ↗
24. Zhang, L., Gonzalez-Garcia, A., Weijer, J.V.D., Danelljan, M., and Khan, F.S. (November, January 27). Learning the model update for siamese trackers. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.
25. Chen, X., Yan, B., Zhu, J., Wang, D., Yang, X., and Lu, H. (2021, January 19–15). Transformer tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual.10.1109/CVPR46437.2021.00803 ↗
26. Xie, F., Wang, C., Wang, G., Yang, W., and Zeng, W. (2021, January 11–17). Learning Tracking Representations via Dual-Branch Fully Transformer Networks. Proceedings of the IEEE/CVF International Conference on Computer Vision, Virtual.10.1109/ICCVW54120.2021.00303 ↗
27. Yan, B., Peng, H., Fu, J., Wang, D., and Lu, H. (2021, January 11–17). Learning spatio-temporal transformer for visual tracking. Proceedings of the IEEE/CVF International Conference on Computer Vision, Virtual.10.1109/ICCV48922.2021.01028 ↗
28. Chen, Z., Zhong, B., Li, G., Zhang, S., and Ji, R. (2020, January 14–19). Siamese box adaptive network for visual tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual.10.1109/CVPR42600.2020.00670 ↗
29. Guo, D., Wang, J., Cui, Y., Wang, Z., and Chen, S. (2020, January 14–19). SiamCAR: Siamese fully convolutional classification and regression for visual tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual.10.1109/CVPR42600.2020.00630 ↗
30. Bolme, D.S., Beveridge, J.R., Draper, B.A., and Lui, Y.M. (2010, January 13–18). Visual object tracking using adaptive correlation filters. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5539960 ↗
31. Danelljan, M., Bhat, G., Shahbaz Khan, F., and Felsberg, M. (2017, January 21–26). Eco: Efficient convolution operators for tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.733 ↗
32. Voigtlaender, P., Luiten, J., Torr, P.H., and Leibe, B. (2020, January 14–19). Siam r-cnn: Visual tracking by re-detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual.10.1109/CVPR42600.2020.00661 ↗
33. Wang, Q., Teng, Z., Xing, J., Gao, J., Hu, W., and Maybank, S. (2018, January 18–23). Learning attentions: Residual attentional siamese network for high performance online visual tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00510 ↗
34. Zeng, Y., Wang, H., and Lu, T. (2019, January 11–13). Learning spatial-channel attention for visual tracking. Proceedings of the 2019 IEEE/CIC International Conference on Communications in China (ICCC), Changchun, China.10.1109/ICCChina.2019.8855908 ↗
35. Nam, H., and Han, B. (2016, January 17–22). Learning multi-domain convolutional neural networks for visual tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, New York, NY, USA.10.1109/CVPR.2016.465 ↗
36. He, A., Luo, C., Tian, X., and Zeng, W. (2018, January 18–23). A twofold siamese network for real-time object tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00508 ↗
37. Initial matting-guided visual tracking with siamese network10.1109/ACCESS.2019.2907282 ↗
38. Rahman, M., Ahmed, M.R., Laishram, L., Kim, S.H., and Jung, S.K. (2020). Siamese high-level feature refine network for visual object tracking. Electronics, 9.10.3390/electronics9111918 ↗
39. Efficient visual tracking with stacked channel-spatial attention learning10.1109/ACCESS.2020.2997917 ↗
40. Siamese attentional keypoint network for high performance visual tracking10.1016/j.knosys.2019.105448 ↗
41. Zhou, X., Wang, D., and Krähenbühl, P. (2019). Objects as points. arXiv.
42. Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8–14). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_1 ↗
43. Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Cehovin Zajc, L., Vojir, T., Bhat, G., Lukezic, A., and Eldesokey, A. (2018, January 8–14). The sixth visual object tracking vot2018 challenge results. Proceedings of the European Conference on Computer Vision (ECCV) Workshops, Munich, Germany.
44. Wu, Y., Lim, J., and Yang, M.H. (2013, January 23–28). Online object tracking: A benchmark. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.312 ↗
45. Fan, H., Lin, L., Yang, F., Chu, P., Deng, G., Yu, S., Bai, H., Xu, Y., Liao, C., and Ling, H. (2019, January 16–20). Lasot: A high-quality benchmark for large-scale single object tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00552 ↗
46. Zhang, F., Wang, Q., and Chen, Z. (2021, December 30). PySOT: SenseTime Research Platform for Single Object Tracking. 2019. GitHub. Available online: https://github.com/STVIR/pysot.
47. Pytorch: An imperative style, high-performance deep learning library
48. Real, E., Shlens, J., Mazzocchi, S., Pan, X., and Vanhoucke, V. (2017, January 21–26). Youtube-boundingboxes: A large high-precision human-annotated data set for object detection in video. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.789 ↗
49. Got-10k: A large high-diversity benchmark for generic object tracking in the wild10.1109/TPAMI.2019.2957464 ↗
50. Imagenet large scale visual recognition challenge10.1007/s11263-015-0816-y ↗
51. Danelljan, M., Robinson, A., Khan, F.S., and Felsberg, M. (2016, January 8–16). Beyond correlation filters: Learning continuous convolution operators for visual tracking. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46454-1_29 ↗