Reinforcement Learning-Based Sequence Training for Robust Vehicle Tracking in Dynamic Traffic Scenes
Source abstract
Abstract
Vehicle tracking is essential for autonomous driving, traffic surveillance, and intelligent transportation, yet most existing trackers rely on frame-level training that neglects temporal dependencies. This mismatch between training and testing leads to error propagation, mislocalization in challenging frames, and failure to re-identify vehicles after occlusion. We present a reinforcement learning (RL)-based sequence-level training framework that formulates tracking as a sequential decision process and directly incorporates evaluation metrics consistent with testing. Our approach enhances robustness in difficult frames and occlusion scenarios by leveraging temporal decision dependencies, and introduces a temporal data augmentation strategy based on sliding-window sampling to improve generalization across diverse motion patterns. Experiments on challenging benchmarks indicate that our method provides improved robustness and temporal continuity over frame-level training approaches, suggesting the benefits of incorporating sequence-level learning in vehicle tracking.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Autonomous Vehicle Technology and Safety · Target Tracking and Data Fusion in Sensor Networks
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Artificial Intelligence, Shanxi Institute of Mechanical & Electrical Engineering, Changzhi 046011, China
School of Future Transportation, Chang’an University, Xi’an 710064, China
Funding
Natural Science Basic Research Plan in Shaanxi Province of China
References (37)
Show source-provided references
1. Multi-object detection and tracking, based on DNN, for autonomous vehicles: A review10.1109/JSEN.2020.3041615 ↗
2. Hung, W.C.W., Zakaria, M.A., Ishak, M.I., and Heerwan, P.M. (2022). Object tracking for autonomous vehicle using yolo v3. Enabling Industry 4.0 Through Advances in Mechatronics: Selected Articles from iM3F 2021, Malaysia, Springer.10.1007/978-981-19-2095-0_23 ↗
3. AOH: Online multiple object tracking with adaptive occlusion handling10.1109/LSP.2022.3191549 ↗
4. Lui, A., Chan, Y., and Leung, M. (2021, January 15–18). Modelling of Destinations for Data-driven Pedestrian Trajectory Prediction in Public Buildings. Proceedings of the 2021 IEEE International Conference on Big Data (Big Data), Orlando, FL, USA.10.1109/BigData52589.2021.9671813 ↗
5. A pov-based highway vehicle trajectory dataset and prediction architecture10.1109/TITS.2024.3435775 ↗
6. Design and analysis of visual vehicle tracking system for traffic surveillance using deep learning
7. Sumitra, V., Tharani, D., Veni, N., Bhargavi, R., Heasreetha Reddy, V., and Amrutha, B. (2025, January 16–17). Collision Detection and Alert System for Road Safety. Proceedings of the 2025 International Conference on Computing and Communication Technologies, Indore, India.
8. Vehicle localization in an explainable dynamic Bayesian network framework for self-aware agents10.1016/j.inffus.2025.103136 ↗
9. Joint active and passive beamforming for vehicle localization with reconfigurable intelligent surfaces10.1109/TITS.2024.3408315 ↗
10. Combining reinforcement learning and conventional control to improve automatic guided vehicles tracking of complex trajectories10.1111/exsy.13076 ↗
11. Achieving accurate trajectory predicting and tracking for autonomous vehicles via reinforcement learning-assisted control approaches10.1016/j.engappai.2024.108773 ↗
12. Human-level control through deep reinforcement learning10.1038/nature14236 ↗
13. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv.
14. Mastering the game of Go with deep neural networks and tree search10.1038/nature16961 ↗
15. Huang, C., Lucey, S., and Ramanan, D. (2017, January 22–29). Learning policies for adaptive tracking with deep feature cascades. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.21 ↗
16. Yun, S., Choi, J., Yoo, Y., Yun, K., and Young Choi, J. (2017, January 21–26). Action-decision networks for visual tracking with deep reinforcement learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.148 ↗
17. Chen, B., Wang, D., Li, P., Wang, S., and Lu, H. (2018, January 8–14). Real-time ‘actor-critic’ tracking. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01234-2_20 ↗
18. Ren, L., Yuan, X., Lu, J., Yang, M., and Zhou, J. (2018, January 8–14). Deep reinforcement learning with iterative shift for visual tracking. Proceedings of the European Conference on Computer Vision, Munich, Germany.10.1007/978-3-030-01240-3_42 ↗
19. Beyond greedy search: Tracking by multi-agent reinforcement learning-based beam search10.1109/TIP.2022.3208437 ↗
20. Liu, X., Tan, J., Ren, X., Ren, W., and Dai, H. (2024, January 5–8). KURL: A knowledge-guided reinforcement learning model for active object tracking. Proceedings of the Asian Conference on Machine Learning, Hanoi, Vietnam.
21. Dong, X., Shen, J., Wang, W., Liu, Y., Shao, L., and Porikli, F. (2018, January 18–22). Hyperparameter optimization for tracking with continuous deep q-learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00061 ↗
22. Dunnhofer, M., Martinel, N., Luca Foresti, G., and Micheloni, C. (November, January 27). Visual tracking by means of deep reinforcement learning and an expert demonstrator. Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, Seoul, Republic of Korea.
23. Kim, M., Lee, S., Ok, J., Han, B., and Cho, M. (2022, January 23–27). Towards sequence-level training for visual tracking. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.10.1007/978-3-031-20047-2_31 ↗
24. Felici, R., Saveriano, M., Roveda, L., and Paolillo, A. (2023, January 20–21). Imitation learning-based visual servoing for tracking moving objects. Proceedings of the International Workshop on Human-Friendly Robotics, Munich, Germany.10.1007/978-3-031-55000-3_8 ↗
25. The use of reinforcement learning algorithms in object tracking: A systematic literature review10.1016/j.neucom.2024.127954 ↗
26. Wiering, M.A., and Van Otterlo, M. (2012). Reinforcement Learning, Springer. Adaptation, Learning, and Optimization.10.1007/978-3-642-27645-3 ↗
27. Rennie, S.J., Marcheret, E., Mroueh, Y., Ross, J., and Goel, V. (2017, January 21–26). Self-critical sequence training for image captioning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.131 ↗
28. The art of data augmentation10.1198/10618600152418584 ↗
29. Wu, Y., Lim, J., and Yang, M.H. (2013, January 23–28). Online object tracking: A benchmark. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.312 ↗
30. Fan, H., Lin, L., Yang, F., Chu, P., Deng, G., Yu, S., Bai, H., Xu, Y., Liao, C., and Ling, H. (2019, January 15–20). Lasot: A high-quality benchmark for large-scale single object tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00552 ↗
31. Zhuang, Z., Liu, M., Cutkosky, A., and Orabona, F. (2022). Understanding adamw through proximal methods and scale-freeness. arXiv.
32. Yu, Q., Fan, K., Wang, Y., and Zheng, Y. (2022). Faster MDNet for visual object tracking. Appl. Sci., 12.10.3390/app12052336 ↗
33. ACSiam: Asymmetric convolution structures for visual tracking with Siamese network10.1016/j.jvcir.2022.103465 ↗
34. Lightweight full-convolutional siamese tracker10.1016/j.knosys.2024.111439 ↗
35. Chen, X., Yan, B., Zhu, J., Wang, D., Yang, X., and Lu, H. (2021, January 19–25). Transformer tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Virtual.10.1109/CVPR46437.2021.00803 ↗
36. Cui, Y., Jiang, C., Wang, L., and Wu, G. (2022, January 21–24). Mixformer: End-to-end tracking with iterative mixed attention. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01324 ↗
37. Hermosillo-Reynoso, F., López-Pimentel, J.-C., Ruiz-Ibarra, E., García-Berumen, A., Del-Puerto-Flores, J.A., Gilardi-Velazquez, H.E., Kumaravelu, V.B., and Luna-Rodriguez, L.A. (2025). A Transformer-Based Multi-Task Learning Model for Vehicle Traffic Surveillance. Mathematics, 13.10.3390/math13233832 ↗