STC-SORT: A Dynamic Spatio-Temporal Consistency Framework for Multi-Object Tracking in UAV Videos
Source abstract
Abstract
Multi-object tracking (MOT) in videos captured by Unmanned Aerial Vehicles (UAVs) is critically challenged by significant camera ego-motion, frequent occlusions, and complex object interactions. To address the limitations of conventional trackers that depend on static, rule-based association strategies, this paper introduces STC-SORT, a novel tracking framework whose core is a two-level reasoning architecture for data association. First, a Spatio-Temporal Consistency Graph Network (STC-GN) models inter-object relationships via graph attention to learn adaptive weights for fusing motion, appearance, and geometric cues. Second, these dynamic weights are integrated into a 4D association cost volume, enabling globally optimal matching across a temporal window. When integrated with an enhanced AEE-YOLO detector, STC-SORT achieves significant and statistically robust improvements on major UAV tracking benchmarks. It elevates MOTA by 13.0% on UAVDT and 6.5% on VisDrone, while boosting IDF1 by 9.7% and 9.9%, respectively. The framework also maintains real-time inference speed (75.5 FPS) and demonstrates substantial reductions in identity switches. These results validate STC-SORT as having strong potential for robust multi-object tracking in challenging UAV scenarios.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · UAV Applications and Optimization · Advanced Technologies in Various Fields
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
National Key Laboratory of Aerospace Mechanism, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China
Joint Laboratory of Spatial Intelligent Perception and Large Model Application, Huantian Smart Technology Co., Ltd., Meishan 620564, China
Funding
Joint Laboratory of Spatial intelligentPerception and Large Model Application, Huantian Smart Technology Co., Ltd · the National Natural Science Foundation of China
References (43)
Show source-provided references
1. Bi, J., Li, K., Zheng, X., Zhang, G., and Lei, T. (2025). SPDC-YOLO: An Efficient Small Target Detection Network Based on Improved YOLOv8 for Drone Aerial Image. Remote Sens., 17.10.3390/rs17040685 ↗
2. Liu, Z., Chen, C., Wang, Z., Cong, L., Lu, H., Pei, Q., and Wan, S. (2024). Automated Vehicle Platooning: A Two-Stage Approach Based on Vehicle-Road Cooperation. IEEE Trans. Intell. Veh., 1–15.10.1109/TIV.2024.3448501 ↗
3. Li, Y., Yuan, D., Sun, M., Wang, H., Liu, X., and Liu, J. (2023, January 19–20). A Global-Local Tracking Framework Driven by Both Motion and Appearance for Infrared Anti-UAV. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancouver, BC, Canada.10.1109/CVPRW59228.2023.00304 ↗
4. TSRes-YOLO: An Accurate and Fast Cascaded Detector for Waste Collection and Transportation Supervision10.1016/j.engappai.2023.106997 ↗
5. Zheng, Z., Yuan, J., Yao, W., Kwan, P., Yao, H., Liu, Q., and Guo, L. (2024). Fusion of UAV-Acquired Visible Images and Multispectral Data by Applying Machine-Learning Methods in Crop Classification. Agronomy, 14.10.3390/agronomy14112670 ↗
6. A Detection Algorithm for Cherry Fruits Based on the Improved YOLO-v4 Model10.1007/s00521-021-06029-z ↗
7. Multi-Object Tracking in UAVs with Feature Fusion Distribution and Occlusion Awareness10.1007/s11760-024-03715-y ↗
8. ArbiTrack: A Novel Multi-Object Tracking Framework for a Moving AAV to Detect and Track Arbitrarily Oriented Targets10.1109/TMM.2025.3543018 ↗
9. DFA-MOT: A Dynamic Field-Aware Multi-Object Tracking Framework for Uncrewed Aerial Vehicles10.1109/TCSVT.2025.3576487 ↗
10. Real-Time Multi-Object Detection and Tracking in UAV Systems: Improved YOLOv11-EFAC and Optimized Tracking Algorithms10.1007/s11554-025-01758-z ↗
11. Bewley, A., Ge, Z., Ott, L., Ramos, F., and Upcroft, B. (2016, January 25–28). Simple Online and Realtime Tracking. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.10.1109/ICIP.2016.7533003 ↗
12. Improved foreign object tracking algorithm in coal for belt conveyor gangue selection robot with YOLOv7 and DeepSORT10.1016/j.measurement.2024.114180 ↗
13. Joshi, A., and Sanyal, B. (2025, January 22–23). CBAM-YOLOv8n and ViT-DeepSORT for UAV Vehicle Tracking. Proceedings of the International Conference on Information Systems and Management Science, Msida, Malta.10.1007/978-3-031-95017-9_16 ↗
14. Ma, J., Luo, H., Chen, Q., Qi, Y., Sun, Y., Beheshti, A., Zhang, J., and Yang, M.-H. (2025). Tracking the Unstable: Appearance-Guided Motion Modeling for Robust Multi-Object Tracking in UAV-Captured Videos. arXiv.
15. Silva Cotta, J.L., Gutierrez, H., Bertaska, I.R., Inness, J.P., and Rakoczy, J. (2024). High-Altitude Precision Landing by Smartphone Video Guidance Sensor and Sensor Fusion. Drones, 8.10.3390/drones8020037 ↗
16. Yu, C., Feng, Z., Wu, Z., Wei, R., Song, B., and Cao, C. (2023). HB-YOLO: An Improved YOLOv7 Algorithm for Dim-Object Tracking in Satellite Remote Sensing Videos. Remote Sens., 15.10.3390/rs15143551 ↗
17. Vehicular/Non-Vehicular Multi-Class Multi-Object Tracking in Drone-Based Aerial Scenes10.1109/TVT.2023.3332132 ↗
18. FLSTrack: Focused Linear Attention Swin-Transformer Network with Dual-Branch Decoder for End-to-End Multi-Object Tracking10.1007/s11760-024-03676-2 ↗
19. Yu, Q., Ma, Y., He, J., Yang, D., and Zhang, T. (2023, January 17–24). A Unified Transformer Based Tracker for Anti-UAV Tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023, Vancouver, BC, Canada.10.1109/CVPRW59228.2023.00305 ↗
20. Liu, W., Lu, H., Fu, H., and Cao, Z. (2023, January 2–3). Learning to Upsample by Learning to Sample. Proceedings of the IEEE/CVF International Conference on Computer Vision 2023, Paris, France.10.1109/ICCV51070.2023.00554 ↗
21. Xiong, Y., Li, Z., Chen, Y., Wang, F., Zhu, X., Luo, J., Wang, W., Lu, T., Li, H., and Qiao, Y. (2024, January 16). Efficient Deformable ConvNets: Rethinking Dynamic and Sparse Operator for Vision Applications. Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR52733.2024.00540 ↗
22. Du, D., Zhu, P., Wen, L., Bian, X., Lin, H., Hu, Q., Peng, T., Zheng, J., Wang, X., and Zhang, Y. (November, January 27). VisDrone-DET2019: The Vision Meets Drone Object Detection in Image Challenge Results. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Republic of Korea.
23. Du, D., Qi, Y., Yu, H., Yang, Y., Duan, K., Li, G., Zhang, W., Huang, Q., and Tian, Q. (2018, January 8–14). The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking. Proceedings of the Computer Vision—ECCV 2018, Munich, Germany.10.1007/978-3-030-01249-6_23 ↗
24. Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y.M. (2023, January 17–24). YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.00721 ↗
25. Zhou, S., and Zhou, H. (2024). Detection Based on Semantics and a Detail Infusion Feature Pyramid Network and a Coordinate Adaptive Spatial Feature Fusion Mechanism Remote Sensing Small Object Detector. Remote Sens., 16.10.3390/rs16132416 ↗
26. SS-YOLOv8: Small-Size Object Detection Algorithm Based on Improved YOLOv8 for UAV Imagery10.1007/s00530-024-01622-3 ↗
27. Wei, C., and Wang, W. (2025). RFAG-YOLO: A Receptive Field Attention-Guided YOLO Network for Small-Object Detection in UAV Images. Sensors, 25.10.3390/s25072193 ↗
28. Zhang, G., Peng, Y., and Li, J. (2025). YOLO-MARS: An Enhanced YOLOv8n for Small Object Detection in UAV Aerial Imagery. Sensors, 25.10.3390/s25082534 ↗
29. YOLO-RSFM: An Efficient Road Small Object Detection Method10.1049/ipr2.13247 ↗
30. ARFP: A Novel Adaptive Recursive Feature Pyramid for Object Detection in Aerial Images10.1007/s10489-021-03147-y ↗
31. DFE-Net: Detail Feature Extraction Network for Small Object Detection10.1007/s00371-024-03277-7 ↗
32. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11–14). SSD: Single Shot Multibox Detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
33. Wojke, N., Bewley, A., and Paulus, D. (2017, January 17–20). Simple Online and Realtime Tracking with a Deep Association Metric. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.10.1109/ICIP.2017.8296962 ↗
34. Zhang, Y., Sun, P., Jiang, Y., Yu, D., Weng, F., Yuan, Z., Luo, P., Liu, W., and Wang, X. (2022, January 23–27). ByteTrack: Multi-Object Tracking by Associating Every Detection Box. Proceedings of the European Conference on Computer Vision 2022, Tel Aviv, Israel.10.1007/978-3-031-20047-2_1 ↗
35. Aharon, N., Orfaig, R., and Bobrovsky, B.-Z. (2022). BoT-SORT: Robust Associations Multi-Pedestrian Tracking. arXiv.
36. StrongSORT: Make DeepSORT Great Again10.1109/TMM.2023.3240881 ↗
37. Li, J., Ding, Y., Wei, H.-L., Zhang, Y., and Lin, W. (2022). SimpleTrack: Rethinking and Improving the JDE Approach for Multi-Object Tracking. Sensors, 22.10.3390/s22155863 ↗
38. Liu, S., Li, X., Lu, H., and He, Y. (2022, January 18–24). Multi-Object Tracking Meets Moving UAV. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022, New Orleans, LA, USA.10.1109/CVPR52688.2022.00867 ↗
39. Multiple Object Detection and Tracking from Drone Videos Based on GM-YOLO and Multi-Tracker10.1016/j.imavis.2024.104951 ↗
40. Cao, J., Pang, J., Weng, X., Khirodkar, R., and Kitani, K. (2023, January 17–24). Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.00934 ↗
41. Qin, Z., Zhou, S., Wang, L., Duan, J., Hua, G., and Tang, W. (2023, January 17–24). MotionTrack: Learning Robust Short-Term and Long-Term Motions for Multi-Object Tracking. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.10.1109/CVPR52729.2023.01720 ↗
42. Sun, P., Cao, J., Jiang, Y., Zhang, R., Xie, E., Yuan, Z., Wang, C., and Luo, P. (2021, January 20–25). TransTrack: Multiple Object Tracking with Transformer 2021. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2021, Nashville, TN, USA.
43. Pang, J., Qiu, L., Li, X., Chen, H., Li, Q., Darrell, T., and Yu, F. (2021, January 20–25). Quasi-Dense Similarity Learning for Multiple Object Tracking. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.10.1109/CVPR46437.2021.00023 ↗