Object Tracking Algorithm Based on Multi-Layer Feature Fusion and Semantic Enhancement
Source abstract
Abstract
The TransT object tracking algorithm, built on Transformer architecture, effectively integrates deep feature extraction with attention mechanisms, thereby enhancing the stability and accuracy of the algorithm. However, this algorithm exhibits insufficient tracking accuracy and boundary box drift when dealing with similar background clutter, which directly affects the subsequent tracking process. To overcome this problem, this paper constructs a semantic enhancement model, which utilizes multi-layer feature representations extracted from deep networks, and correlates and fuses shallow features with deep features by using cross-attention. At the same time, in order to adapt to the changes in the surrounding environment of the object and establish good discrimination with similar objects, this paper proposes a dynamic mask strategy to optimize the attention allocation mechanism and finally employs an object template update mechanism to improve the adaptability of the model by comparing the spatio-temporal information of successive frames to update the object template in time, further enhancing its tracking performance in complex scenes. Experimental comparison results demonstrate that the algorithm proposed in this paper can effectively handle similar background clutter, leading to a significant improvement in the overall performance of the tracking model.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Fire Detection and Safety Systems · Remote Sensing and Land Use
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Printing, Packaging and Digital Media, Xi’an University of Technology, Xi’an 710048, China
School of Automation and Information Engineering, Xi’an University of Technology, Xi’an 710048, China
Funding
National Natural Science Foundation (NNSF) of China · National science basic research program of Shaanxi
References (36)
Show source-provided references
1. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4). Attention Is All You Need. Proceedings of the Conference on Neural Information Processing Systems, Long Beach, CA, USA.
2. Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., and Torr, P.H. (15–16, January 8–10). Fully-Convolutional Siamese Networks For Object Tracking. Proceedings of the Computer Vision–ECCV 2016 Workshops, Amsterdam, The Netherlands.10.1007/978-3-319-48881-3_56 ↗
3. Li, B., Yan, J., Wu, W., Zhu, Z., and Hu, X. (2018, January 18–23). High Performance Visual Tracking With Siamese Region Proposal Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00935 ↗
4. Zhu, Z., Wang, Q., Li, B., Wu, W., Yan, J., and Hu, W. (2018, January 8–14). Distractor-Aware Siamese Networks For Visual Object Tracking. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01240-3_7 ↗
5. Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., and Yan, J. (2019, January 15–20). Siamrpn++: Evolution Of Siamese Visual Tracking With Very Deep Networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00441 ↗
6. Wang, Q., Zhang, L., Bertinetto, L., Hu, W., and Torr, P.H. (2019, January 15–20). Fast Online Object Tracking And Segmentation: A Unifying Approach. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00142 ↗
7. SiamFCA: A new fish single object tracking method based on siamese network with coordinate attention in aquaculture10.1016/j.compag.2023.108542 ↗
8. Siamese graph attention networks for robust visual object tracking10.1016/j.cviu.2023.103634 ↗
9. Yan, B., Peng, H., Fu, J., Wang, D., and Lu, H. (2021, January 11–17). Learning Spatio-Temporal Transformer For Visual Tracking. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.01028 ↗
10. Lan, J.P., Cheng, Z.Q., He, J.Y., Li, C., Luo, B., Bao, X., Xiang, W., Geng, Y., and Xie, X. (2023, January 4–10). Procontext: Exploring Progressive Context Transformer For Tracking. Proceedings of the ICASSP 2023—2023 IEEE International Conference on Acoustics, Speech and Signal Processing, Rhodes Island, Greece.10.1109/ICASSP49357.2023.10094971 ↗
11. Xie, F., Chu, L., Li, J., Lu, Y., and Ma, C. (2023, January 17–24). VideoTrack: Learning To Track Objects via Video Transformer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.02186 ↗
12. Swintrack: A simple and strong baseline for transformer tracking
13. Xing, D., Evangeliou, N., Tsoukalas, A., and Tzes, A. (2022, January 3–8). Siamese Transformer Pyramid Networks For Real-Time UAV Tracking. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV51458.2022.00196 ↗
14. Cao, Z., Fu, C., Ye, J., Li, B., and Li, Y. (2021, January 11–17). Hift: Hierarchical Feature Transformer For Aerial Tracking. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.01517 ↗
15. Cui, Y., Jiang, C., Wang, L., and Wu, G. (2022, January 18–24). Mixformer: End-to-end tracking with iterative mixed attention. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01324 ↗
16. Wang, N., Zhou, W., Wang, J., and Li, H. (2021, January 19–25). Transformer Meets Tracker: Exploiting Temporal Context For Robust Visual Tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00162 ↗
17. Target-Aware Transformer Tracking10.1109/TCSVT.2023.3276061 ↗
18. Wang, Z., and Kamata, S.-I. (2022, January 25–27). Multiple Mask Enhanced Transformer For Robust Visual Tracking. Proceedings of the IEEE 2022 4th International Conference on Robotics and Computer Vision, Wuhan, China.10.1109/ICRCV55858.2022.9953264 ↗
19. Wu, Q., Yang, T., Liu, Z., Wu, B., Shan, Y., and Chan, A.B. (2023, January 17–24). Dropmae: Masked Autoencoders With Spatial-Attention Dropout For Tracking Tasks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.10.1109/CVPR52729.2023.01399 ↗
20. TabCtNet: Target-aware bilateral CNN-transformer network for single object tracking in satellite videos
21. Chen, X., Yan, B., Zhu, J., Wang, D., Yang, X., and Lu, H. (2021, January 19–25). Transformer Tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.00803 ↗
22. Wang, J., Wang, Y., Que, Y., Huang, W., and Wei, Y. (2024). Object Tracking Algorithm Based on Integrated Multi-Scale Templates Guided by Judgment Mechanism. Electronics, 13.10.3390/electronics13214309 ↗
23. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep Residual Learning For Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
24. Loshchilov, I. (2017). Decoupled weight decay regularization. arXiv.
25. Fan, H., Lin, L., Yang, F., Chu, P., Deng, G., Yu, S., Bai, H., Xu, Y., Liao, C., and Ling, H. (2019, January 15–20). Lasot: A High-Quality Benchmark For Large-Scale Single Object Tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00552 ↗
26. Wu, Y., Lim, J., and Yang, M.-H. (2013, January 23–28). Online Object Tracking: A Benchmark. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.312 ↗
27. Mueller, M., Smith, N., and Ghanem, B. (2016, January 11–14). A benchmark and simulator for uav tracking. Proceedings of the Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_27 ↗
28. Tracking Vision Transformer With Class And Regression Tokens10.1016/j.ins.2022.11.055 ↗
29. Yu, Y., Xiong, Y., Huang, W., and Scott, M.R. (2020, January 14–19). Deformable Siamese Attention Networks For Visual Object Tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.00676 ↗
30. Chen, X., Kang, B., Wang, D., Li, D., and Lu, H. (2022, January 23–27). Efficient Visual Tracking Via Hierarchical Cross-Attention Transformer. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-031-25085-9_26 ↗
31. Bhat, G., Danelljan, M., Gool, L.V., and Timofte, R. (November, January 27). Learning Discriminative Model Prediction For Tracking. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
32. Blatter, P., Kanakis, M., Danelljan, M., and Van Gool, L. (2023, January 2–7). Efficient Visual Tracking With Exemplar Transformers. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV56688.2023.00162 ↗
33. Target signature network for small object tracking10.1016/j.engappai.2024.109445 ↗
34. Learning a novel ensemble tracker for robust visual tracking10.1109/TMM.2023.3307939 ↗
35. Adaptively bypassing vision transformer blocks for efficient visual tracking10.1016/j.patcog.2024.111278 ↗
36. Wei, Z., He, Y., and Cai, Z. (2024). Enhanced Object Tracking by Self-Supervised Auxiliary Depth Estimation Learning. arXiv.