Kalman Filtering and Bipartite Matching Based Super-Chained Tracker Model for Online Multi Object Tracking in Video Sequences
Source abstract
Abstract
Object tracking has gained importance in various applications especially in traffic monitoring, surveillance and security, people tracking, etc. Previous methods of multiobject tracking (MOT) carry out detections and perform object tracking. Although not optimal, these frameworks perform the detection and association of objects with feature extraction separately. In this article, we have proposed a Super Chained Tracker (SCT) model, which is convenient and online and provides better results when compared with existing MOT methods. The proposed model comprises subtasks, object detection, feature manipulation, and using representation learning into one end-to-end solution. It takes adjacent frames as input, converting each frame into bounding boxes’ pairs and chaining them up with Intersection over Union (IoU), Kalman filtering, and bipartite matching. Attention is made by object attention, which is in paired box regression branch, caused by the module of object detection, and a module of ID verification creates identity attention. The detections from these branches are linked together by IoU matching, Kalman filtering, and bipartite matching. This makes our SCT speedy, simple, and effective enough to achieve a Multiobject Tracking Accuracy (MOTA) of 68.4% and Identity F1 (IDF1) of 64.3% on the MOT16 dataset. We have studied existing tracking techniques and analyzed their performance in this work. We have achieved more qualitative and quantitative tracking results than other existing techniques with relatively improved margins.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Fire Detection and Safety Systems · IoT-based Smart Home Systems
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer and Information Sciences, Pakistan Institute of Engineering and Applied Sciences (PIEAS), Islamabad 45650, Pakistan
Department of Computer Science and Information Technology, King Abdullah Campus Chatter Kalas, University of Azad Jammu and Kashmir, Muzaffarabad 13100, Pakistan
Department of Computer Science and Information Technology, Neelum Campus, University of Azad Jammu and Kashmir, Athmuqam 13230, Pakistan
Department of Computing, Islamabad Campus, Iqra University, Islamabad 44000, Pakistan
Information and Computing Technology (ICT) Division, College of Science and Engineering (CSE), Hamad Bin Khalifa University (HBKU), Doha P.O. Box 34110, Qatar
References (70)
Show source-provided references
1. Deep learning in video multi-object tracking: A survey10.1016/j.neucom.2019.11.023 ↗
2. An Effectively Finite-Tailed Updating for Multiple Object Tracking in Crowd Scenes10.3390/app12031061 ↗
3. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
4. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
5. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
6. Faster r-cnn: Towards real-time object detection with region proposal networks
7. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A.C. (2016). Ssd: Single shot multibox detector. European Conference on Computer Vision, Springer.10.1007/978-3-319-46448-0_2 ↗
8. Redmon, J., and Farhadi, A. (2017, January 21–26). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.690 ↗
9. Tangirala, K.V., and Namuduri, K.R. (2005, January 23). Object tracking in video using particle filtering. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’05), Philadelphia, PA, USA.
10. Anomaly detection with particle filtering for online video surveillance10.1109/ACCESS.2021.3054040 ↗
11. Takala, V., and Pietikainen, M. (2007, January 17–22). Multi-object tracking using color, texture and motion. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.10.1109/CVPR.2007.383506 ↗
12. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns10.1109/TPAMI.2002.1017623 ↗
13. Shu, G. (2014). Human Detection, Tracking and Segmentation in Surveillance Video. [Ph.D. Thesis, University of Central Florida].
14. Video tracking using learned hierarchical features10.1109/TIP.2015.2403231 ↗
15. Explicitly exploiting hierarchical features in visual object tracking10.1016/j.neucom.2020.02.038 ↗
16. Zhong, Z., Gao, Y., Zheng, Y., and Zheng, B. (2020). Efficient spatio-temporal recurrent neural network for video deblurring. European Conference on Computer Vision, Springer.10.1007/978-3-030-58539-6_12 ↗
17. Kang, K., Ouyang, W., Li, H., and Wang, X. (2016, January 27–30). Object detection from video tubelets with convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.95 ↗
18. Ning, G., Zhang, Z., Huang, C., Ren, X., Wang, H., Cai, C., and He, Z. (2017, January 28–31). Spatially supervised recurrent convolutional neural networks for visual object tracking. Proceedings of the 2017 IEEE International Symposium on Circuits and Systems (ISCAS), Baltimore, MD, USA.10.1109/ISCAS.2017.8050867 ↗
19. Held, D., Thrun, S., and Savarese, S. (2016). Learning to track at 100 fps with deep regression networks. European conference on Computer Vision, Springer.10.1007/978-3-319-46448-0_45 ↗
20. Ma, C., Huang, J.-B., Yang, X., and Yang, M.-H. (2015, January 7–13). Hierarchical convolutional features for visual tracking. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.352 ↗
21. Wang, L., Ouyang, W., Wang, X., and Lu, H. (2015, January 7–13). Visual tracking with fully convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.357 ↗
22. Moving object detection in aerial video based on spatiotemporal saliency10.1016/j.cja.2013.07.038 ↗
23. Coarse-to-fine object detection in unmanned aerial vehicle imagery using lightweight convolutional neural network and deep motion saliency10.1016/j.neucom.2019.03.102 ↗
24. Hui, T.-W., Tang, X., and Loy, C.C. (2018, January 18–23). Liteflownet: A lightweight convolutional neural network for optical flow estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00936 ↗
25. Wang, R.J., Li, X., and Ling, C.X. (2018). Pelee: A real-time object detection system on mobile devices. arXiv.
26. Generic object detection using improved gentleboost classifier10.1016/j.phpro.2012.03.272 ↗
27. MapReduce based text detection in big data natural scene videos10.1016/j.procs.2015.07.297 ↗
28. Background modelling from a moving camera10.1016/j.procs.2015.08.023 ↗
29. Adaptive pedestrian detection in infrared images using background subtraction and local thresholding10.1016/j.procs.2015.08.091 ↗
30. Park, Y., Dang, L.M., Lee, S., Han, D., and Moon, H. (2021). Multiple object tracking in deep learning approaches: A survey. Electronics, 10.10.3390/electronics10192406 ↗
31. Bergmann, P., Meinhardt, T., and Leal-Taixe, L. (2019, January 2). Tracking without bells and whistles. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea.10.1109/ICCV.2019.00103 ↗
32. Zhu, J., Yang, H., Liu, N., Kim, M., Zhang, W., and Yang, M.-H. (2018, January 8–14). Online multi-object tracking with dual matching attention networks. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01228-1_23 ↗
33. Chu, Q., Ouyang, W., Li, H., Wang, X., Liu, B., and Yu, N. (2017, January 22–29). Online multi-object tracking using CNN-based single object tracker with spatial-temporal attention mechanism. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.518 ↗
34. Choi, W. (2015, January 7–13). Near-online multi-target tracking with aggregated local flow descriptor. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.347 ↗
35. Yu, F., Li, W., Li, Q., Liu, Y., Shi, X., and Yan, J. (2016). Poi: Multiple object tracking with high performance detection and appearance feature. European Conference on Computer Vision, Springer.10.1007/978-3-319-48881-3_3 ↗
36. Sanchez-Matilla, R., Poiesi, F., and Cavallaro, A. (2016). Online multi-target tracking with strong and weak detections. European Conference on Computer Vision, Springer.10.1007/978-3-319-48881-3_7 ↗
37. Wojke, N., Bewley, A., and Paulus, D. (2017, January 7). Simple online and realtime tracking with a deep association metric. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.10.1109/ICIP.2017.8296962 ↗
38. Multi-target tracking using CNN-based features: CNNMTT10.1007/s11042-018-6467-6 ↗
39. Zaech, J.-N., Liniger, A., Dai, D., Danelljan, M., and Van Gool, L. (2022, January 23–27). Learnable online graph representations for 3d multi-object tracking. Proceedings of the IEEE Robotics and Automation Letters, Philadelphia, PA, USA.10.1109/LRA.2022.3145952 ↗
40. Mat: Motion-aware multi-object tracking10.1016/j.neucom.2021.12.104 ↗
41. Online multiple object tracking based on fusing global and partial features10.1016/j.neucom.2021.10.107 ↗
42. Yin, T., Zhou, X., and Krahenbuhl, P. (2021, January 20–25). Center-based 3d object detection and tracking. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPR46437.2021.01161 ↗
43. MobileNet-JDE: A lightweight multi-object tracking model for embedded systems10.1007/s11042-022-12095-9 ↗
44. Breitenstein, M.D., Reichlin, F., Leibe, B., Koller-Meier, E., and Van Gool, L. (October, January 29). Robust tracking-by-detection using a detector confidence particle filter. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan.10.1109/ICCV.2009.5459278 ↗
45. Bochinski, E., Eiselein, V., and Sikora, T. (September, January 29). High-speed tracking-by-detection without using image information. Proceedings of the 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Lecce, Italy.10.1109/AVSS.2017.8078516 ↗
46. Peng, J., Wang, C., Wan, F., Wu, Y., Wang, Y., Tai, Y., Wang, C., Li, J., Huang, F., and Fu, Y. (2020). Chained-tracker: Chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking. European Conference on Computer Vision, Springer.10.1007/978-3-030-58548-8_9 ↗
47. Semantic loop closure detection based on graph matching in multi-objects scenes10.1016/j.jvcir.2021.103072 ↗
48. Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., and Savarese, S. (2019, January 15–20). Generalized intersection over union: A metric and a loss for bounding box regression. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00075 ↗
49. Moving object tracking using kalman filter
50. Meinhardt, T., Kirillov, A., Leal-Taixe, L., and Feichtenhofer, C. (2021). Trackformer: Multi-object tracking with transformers. arXiv.10.1109/CVPR52688.2022.00864 ↗
51. Shokoufandeh, A., and Dickinson, S. (1999, January 21). Applications of bipartite matching to problems in object recognition. Proceedings of the ICCV Workshop on Graph Algorithms and Computer Vision, Corfu, Greece.
52. Bodla, N., Singh, B., Chellappa, R., and Davis, L.S. (2017, January 22–29). Soft-NMS-improving object detection with one line of code. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.593 ↗
53. Virtual reality support for joint attention using the Floreo Joint Attention Module: Usability and feasibility pilot study10.2196/14429 ↗
54. Lin, T.-Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
55. Milan, A., Leal-Taixé, L., Reid, I., Roth, S., and Schindler, K. (2016). MOT16: A benchmark for multi-object tracking. arXiv.
56. Yu, E., Li, Z., Han, S., and Wang, H. Relationtrack: Relation-aware multiple object tracking with decoupled representation. IEEE Trans. Multimed., 2022.10.1109/TMM.2022.3150169 ↗
57. Object detection with discriminatively trained part-based models10.1109/TPAMI.2009.167 ↗
58. Chen, Y., Li, W., Sakaridis, C., Dai, D., and Van Gool, L. (2018, January 18–23). Domain adaptive faster r-cnn for object detection in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00352 ↗
59. Dai, J., He, K., and Sun, J. (2016, January 27–30). Instance-aware semantic segmentation via multi-task network cascades. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.343 ↗
60. Yang, F., Choi, W., and Lin, Y. (2016, January 27–30). Exploit all the layers: Fast and accurate cnn object detector with scale dependent pooling and cascaded rejection classifiers. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.234 ↗
61. Evaluating multiple object tracking performance: The clear mot metrics10.1155/2008/246309 ↗
62. SegDQ: Segmentation Assisted Multi-Object Tracking with Dynamic Query-based Transformers10.1016/j.neucom.2022.01.073 ↗
63. Li, Y., Huang, C., and Nevatia, R. (2009, January 20–25). Learning to associate: Hybridboosted multi-target tracker for crowded scene. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206735 ↗
64. Online multiple object tracking using rule distillated siamese random forest10.1109/ACCESS.2020.3028770 ↗
65. Tang, S., Andriluka, M., Andres, B., and Schiele, B. (2017, January 21–26). Multiple people tracking by lifted multicut and person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.394 ↗
66. Son, J., Baek, M., Cho, M., and Han, B. (2017, January 21–26). Multi-object tracking with quadruplet convolutional neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.403 ↗
67. Kim, C., Li, F., and Rehg, J.M. (2018, January 8–14). Multi-object tracking with neural gating using bilinear lstm. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01237-3_13 ↗
68. Chen, J., Sheng, H., Zhang, Y., and Xiong, Z. (2017, January 21–26). Enhancing detection model for multiple hypothesis tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.10.1109/CVPRW.2017.266 ↗
69. Confidence-based data association and discriminative deep appearance learning for robust online multi-object tracking10.1109/TPAMI.2017.2691769 ↗
70. Lee, B., Erdenee, E., Jin, S., Nam, M.Y., Jung, Y.G., and Rhee, P.K. (2016). Multi-class multi-object tracking using changing point detection. European Conference on Computer Vision, Springer.10.1007/978-3-319-48881-3_6 ↗