A Novel Dataset for Multi-View Multi-Player Tracking in Soccer Scenarios
Source abstract
Abstract
Localization and tracking in multi-player sports present significant challenges, particularly in wide and crowded scenes where severe occlusions can occur. Traditional solutions relying on a single camera are limited in their ability to accurately identify players and may result in ambiguous detection. To overcome these challenges, we proposed fusing information from multiple cameras positioned around the field to improve positioning accuracy and eliminate occlusion effects. Specifically, we focused on soccer, a popular and representative multi-player sport, and developed a multi-view recording system based on a 1+N strategy. This system enabled us to construct a new benchmark dataset and continuously collect data from several sports fields. The dataset includes 17 sets of densely annotated multi-view videos, each lasting 2 min, as well as 1100+ min multi-view videos. It encompasses a wide range of game types and nearly all scenarios that could arise during real game tracking. Finally, we conducted a thorough assessment of four multi-view multi-object tracking (MVMOT) methods and gained valuable insights into the tracking process in actual games.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Analysis and Summarization · Video Surveillance and Tracking Methods · Remote Sensing and LiDAR Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Public Physical and Art Education, Zhejiang University, Hangzhou 310058, China
National Engineering Research Center for Optical Instruments, Zhejiang University, Hangzhou 310058, China
Centre for Optical and Electromagnetic Research, Zhejiang University, Hangzhou 310058, China
Hangzhou Zhuxing Information Technology Co., Ltd., Hangzhou 311100, China
Shanghai Institute for Advanced Study, Zhejiang University, Shanghai 200135, China
Department of Applied Biology and Chemical Technology, The Hong Kong Polytechnic University, Hong Kong SAR, China
Funding
National Natural Science Foundation of China · Key Projects of Major Humanities and Social Sciences of Zhejiang’s Universities 383 2019–2020 · Zhejiang Province’s “14th Five-Year Plan” graduate teaching reform project
References (84)
Show source-provided references
1. Tactic analysis based on real-world ball trajectory in soccer video10.1016/j.patcog.2011.10.023 ↗
2. A review of vision-based systems for soccer video analysis10.1016/j.patcog.2010.03.009 ↗
3. Lu, X., Wang, W., Ma, C., Shen, J., Shao, L., and Porikli, F. (2019, January 16–20). See more, know more: Unsupervised video object segmentation with co-attention siamese networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00374 ↗
4. Semi-supervised video object segmentation with super-trajectories10.1109/TPAMI.2018.2819173 ↗
5. Soccer analytics unravelling the complexity of “the beautiful game”10.1111/j.1740-9713.2018.01146.x ↗
6. Fernandez, J., and Bornn, L. (2018, January 23–24). Wide open spaces: A statistical technique for measuring space creation in professional soccer. Proceedings of the Sloan Sports Analytics Conference, Boston, MA, USA.
7. Space evaluation in football games via field weighting based on tracking data10.1038/s41598-021-84939-7 ↗
8. Dave, A., Khurana, T., Tokmakov, P., Schmid, C., and Ramanan, D. (2020, January 23–28). Tao: A large-scale benchmark for tracking any object. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK.10.1007/978-3-030-58558-7_26 ↗
9. Huang, W., He, S., Sun, Y., Evans, J., Song, X., Geng, T., Sun, G., and Fu, X. (2022). Open dataset recorded by single cameras for multi-player tracking in soccer scenarios. Appl. Sci., 12.10.3390/app12157473 ↗
10. A public data set of spatio-temporal match events in soccer competitions10.1038/s41597-019-0247-7 ↗
11. Voigtlaender, P., Krause, M., Osep, A., Luiten, J., Sekar, B.B.G., Geiger, A., and Leibe, B. (2019, January 16–20). Mots: Multi-object tracking and segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00813 ↗
12. Geiger, A., Lenz, P., and Urtasun, R. (2012, January 16–21). Are we ready for autonomous driving? The kitti vision benchmark suite. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.10.1109/CVPR.2012.6248074 ↗
13. Sun, P., Cao, J., Jiang, Y., Zhang, R., Xie, E., Yuan, Z., Wang, C., and Luo, P. (2020). Transtrack: Multiple object tracking with transformer. arXiv.
14. Xu, Y., Ban, Y., Delorme, G., Gan, C., Rus, D., and Alameda-Pineda, X. (2021). Transcenter: Transformers with dense queries for multiple-object tracking. arXiv.10.1109/TPAMI.2022.3225078 ↗
15. Fairmot: On the fairness of detection and re-identification in multiple object tracking10.1007/s11263-021-01513-4 ↗
16. Zhou, X., Koltun, V., and Krähenbühl, P. (2020, January 23–28). Tracking objects as points. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK.10.1007/978-3-030-58548-8_28 ↗
17. Yang, Y., Zhang, R., Wu, W., Peng, Y., and Xu, M. (2021, January 10–15). Multi-camera sports players 3d localization with identification reasoning. Proceedings of the 25th International Conference on Pattern Recognition (ICPR), Milano, Italy.10.1109/ICPR48806.2021.9412935 ↗
18. Confidence-based data association and discriminative deep appearance learning for robust online multi-object tracking10.1109/TPAMI.2017.2691769 ↗
19. Bewley, A., Ge, Z., Ott, L., Ramos, F., and Upcroft, B. (2016, January 25–28). Simple online and realtime tracking. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.10.1109/ICIP.2016.7533003 ↗
20. Fang, K., Xiang, Y., Li, X., and Savarese, S. (2018, January 12–15). Recurrent autoregressive networks for online multi-object tracking. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, CA, USA.10.1109/WACV.2018.00057 ↗
21. Multicamera people tracking with a probabilistic occupancy map10.1109/TPAMI.2007.1174 ↗
22. Li, P., Li, G., Yan, Z., Li, Y., Lu, M., Xu, P., Gu, Y., Bai, B., Zhang, Y., and Chuxing, D. (2019, January 16–20). Spatio-temporal consistency and hierarchical matching for multi-target multi-camera vehicle tracking. Proceedings of the CVPR Workshops, Long Beach, CA, USA.
23. Tang, S., Andriluka, M., Andres, B., and Schiele, B. (2017, January 21–26). Multiple people tracking by lifted multicut and person re-identification. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.394 ↗
24. Wojke, N., Bewley, A., and Paulus, D. (2017, January 17–20). Simple online and realtime tracking with a deep association metric. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.10.1109/ICIP.2017.8296962 ↗
25. Multiple hypothesis tracking algorithm for multi-target multi-camera tracking with disjoint views10.1049/iet-ipr.2017.1244 ↗
26. Yu, F., Li, W., Li, Q., Liu, Y., Shi, X., and Yan, J. (October, January 8–10,). Poi: Multiple object tracking with high performance detection and appearance feature. Proceedings of the Computer Vision–ECCV 2016 Workshops, Amsterdam, The Netherlands. Proceedings, Part II 14.10.1007/978-3-319-48881-3_3 ↗
27. Zhou, Z., Xing, J., Zhang, M., and Hu, W. (2018, January 20–24). Online multi-target tracking with tensor-based high-order graph matching. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.10.1109/ICPR.2018.8545450 ↗
28. Bredereck, M., Jiang, X., Körner, M., and Denzler, J. (November, January 30). Data association for multi-object-tracking-by-detection in multi-camera networks. Proceedings of the 2012 Sixth International Conference on Distributed Smart Cameras (ICDSC), Hong Kong, China.
29. Vedaldi, A., Bischof, H., Brox, T., and Frahm, J. (2020). Multiview Detection with Feature Perspective Transformation, Springer International Publishing. Computer Vision—ECCV, 2020.10.1007/978-3-030-58592-1 ↗
30. Hsu, H.-M., Huang, T.-W., Wang, G., Cai, J., Lei, Z., and Hwang, J.-N. (2019, January 16–20). Multi-camera tracking of vehicles based on deep features re-id and trajectory-based camera link models. Proceedings of the CVPR Workshops, Long Beach, CA, USA.
31. A bayesian filter for multi-view 3d multi-object tracking with occlusion handling10.1109/TPAMI.2020.3034435 ↗
32. Xu, Y., Liu, X., Liu, Y., and Zhu, S.-C. (2016, January 27–30). Multi-view people tracking via hierarchical trajectory composition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.461 ↗
33. Salsa: A novel dataset for multimodal group behavior analysis
34. Chavdarova, T., Baqué, P., Bouquet, S., Maksai, A., Jose, C., Bagautdinov, T., Lettry, L., Fua, P., Gool, L.V., and Fleuret, F. (2018, January 18–23). Wildtrack: A multi-camera hd dataset for dense unscripted pedestrian detection. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00528 ↗
35. Chavdarova, T., and Fleuret, F. (2017, January 18–21). Deep multi-camera people detection. Proceedings of the 16th IEEE International Conference on Machine Learning and Applications (ICMLA), Cancun, Mexico.10.1109/ICMLA.2017.00-50 ↗
36. Ferryman, J., and Shahrokni, A. (2009, January 7–9). Pets2009: Dataset and challenge. Proceedings of the 2009 Twelfth IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, Snowbird, UT, USA.10.1109/PETS-WINTER.2009.5399556 ↗
37. Distributed video acquisition and annotation for sport-event summarization
38. Ristani, E., Solera, F., Zou, R., Cucchiara, R., and Tomasi, C. (2016, January 8–16). Performance measures and a data set for multi-target, multi-camera tracking. Proceedings of the Computer Vision–ECCV 2016 Workshops, Amsterdam, The Netherlands.10.1007/978-3-319-48881-3_2 ↗
39. Krumm, J., Harris, S., Meyers, B., Brumitt, B., Hale, M., and Shafer, S. (2000, January 1). Multi-camera multi-person tracking for easyliving. Proceedings of the Third IEEE International Workshop on Visual Surveillance, Dublin, Ireland.
40. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
41. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A.C. (2016, January 11–14). Ssd: Single shot multibox detector. Proceedings of the Computer Vision—ECCV 2016: 14th European Conference, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
42. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
43. A survey of modern deep learning based object detection models10.1016/j.dsp.2022.103514 ↗
44. Ouyang, W., and Wang, X. (2013, January 1–8). Joint deep learning for pedestrian detection. Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Australia.10.1109/ICCV.2013.257 ↗
45. Foveabox: Beyound anchor-based object detection10.1109/TIP.2020.3002345 ↗
46. Song, T., Sun, L., Xie, D., Sun, H., and Pu, S. (2018, January 8–14). Small-scale pedestrian detection based on topological line localization and temporal feature aggregation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01234-2_33 ↗
47. Zhang, S., Wen, L., Bian, X., Lei, Z., and Li, S.Z. (2018, January 8–14). Occlusion-aware r-cnn: Detecting pedestrians in a crowd. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01219-9_39 ↗
48. Chen, L., Ai, H., Shang, C., Zhuang, Z., and Bai, B. (2017, January 17–20). Online multi-object tracking with convolutional neural networks. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.10.1109/ICIP.2017.8296360 ↗
49. Li, D., Chen, X., Zhang, Z., and Huang, K. (2017, January 21–26). Learning deep context-aware features over body and latent parts for person re-identification. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.782 ↗
50. Li, Y., Yao, H., Duan, L., Yao, H., and Xu, C. (2019, January 21–25). Adaptive feature fusion via graph neural network for person re-identification. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.10.1145/3343031.3350982 ↗
51. Su, C., Li, J., Zhang, S., Xing, J., Gao, W., and Tian, Q. (2017, January 22–29). Pose-driven deep convolutional model for person re-identification. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.427 ↗
52. Deep affinity network for multiple object tracking
53. Deep representation learning with part loss for person re-identification10.1109/TIP.2019.2891888 ↗
54. Zhao, H., Tian, M., Sun, S., Shao, J., Yan, J., Yi, S., Wang, X., and Tang, X. (2017, January 21–26). Spindle net: Person re-identification with human body region guided feature decomposition and fusion. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.103 ↗
55. Leal-Taixé, L., Canton-Ferrer, C., and Schindler, K. (July, January 26). Learning by tracking: Siamese cnn for robust target association. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops, Las Vegas, NV, USA.10.1109/CVPRW.2016.59 ↗
56. Iqbal, U., Milan, A., and Gall, J. (2017, January 21–26). Posetrack: Joint multi-person pose estimation and tracking. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.495 ↗
57. Choi, W. (2015, January 7–13). Near-online multi-target tracking with aggregated local flow descriptor. Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.347 ↗
58. Wan, X., Wang, J., Kong, Z., Zhao, Q., and Deng, S. (2018, January 7–10). Multi-object tracking using online metric learning with long short-term memory. Proceedings of the 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece.10.1109/ICIP.2018.8451174 ↗
59. The hungarian method for the assignment problem10.1002/nav.3800020109 ↗
60. Hou, Y., Zheng, L., Wang, Z., and Wang, S. (2019). Locality aware appearance metric for multi-target multi-camera tracking. arXiv.
61. Ristani, E., and Tomasi, C. (2018, January 18–23). Features for multi-target multi-camera tracking and re-identification. Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00632 ↗
62. Tesfaye, Y.T., Zemene, E., Prati, A., Pelillo, M., and Shah, M. (2017). Multi-target tracking in multiple non-overlapping cameras using constrained dominant sets. arXiv.
63. Zhang, Z., Wu, J., Zhang, X., and Zhang, C. (2017). a Multi-target, multi-camera tracking by hierarchical clustering: Recent progress on dukemtmc project. arXiv.
64. Online scheme for multiple camera multiple target tracking based on multiple hypothesis tracking10.1109/TCSVT.2016.2593619 ↗
65. Multiple object tracking using k-shortest paths optimization10.1109/TPAMI.2011.21 ↗
66. Jiang, N., Bai, S., Xu, Y., Xing, C., Zhou, Z., and Wu, W. (2018, January 22–26). Online inter-camera trajectory association exploiting person re-identification and camera topology. Proceedings of the 26th ACM International Conference on Multimedia, Seoul, Republic of Korea.10.1145/3240508.3240663 ↗
67. He, Y., Han, J., Yu, W., Hong, X., Wei, X., and Gong, Y. (2020, January 14–19). City-scale multi-camera vehicle tracking by semantic attribute parsing and cross-camera tracklet matching. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Virtual.10.1109/CVPRW50498.2020.00296 ↗
68. Multi-target multi-camera tracking by tracklet-to-target assignment10.1109/TIP.2020.2980070 ↗
69. Hofmann, M., Wolf, D., and Rigoll, G. (2013, January 23–28). Hypergraphs for joint multi-view reconstruction and multi-object tracking. Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.468 ↗
70. Jiang, X., Rodner, E., and Denzler, J. (2012, January 1–5). Multi-person tracking-by-detection based on calibrated multi-camera systems. Proceedings of the International Conference on Computer Vision and Graphics, Moscow, Russia.10.1007/978-3-642-33564-8_89 ↗
71. Zamir, A.R., Dehghan, A., and Shah, M. (2012, January 7–13). Gmcp-tracker: Global multi-object tracking using generalized minimum clique graphs. Proceedings of the European Conference on Computer Vision, Florence, Italy.
72. An equalized global graph model-based approach for multicamera object tracking10.1109/TCSVT.2016.2589619 ↗
73. Gan, Y., Han, R., Yin, L., Feng, W., and Wang, S. (2021, January 20–24). Self-supervised multi-view multi-human association and tracking. Proceedings of the 29th ACM International Conference on Multimedia, Virtual.10.1145/3474085.3475177 ↗
74. Multi-view multi-human association with deep assignment network10.1109/TIP.2021.3139178 ↗
75. You, Q., and Jiang, H. (2020). Real-time 3d deep multi-camera tracking. arXiv.
76. Engilberge, M., Liu, W., and Fua, P. (2023, January 3–7). Multi-view tracking using weakly supervised human motion prediction. Proceedings of the 2023 IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV56688.2023.00163 ↗
77. Liu, W., Salzmann, M., and Fua, P. (2020, January 23–28). Estimating people flows to better count them in crowded scenes. Proceedings of the European Conference on Computer Vision, Virtual.10.1007/978-3-030-58555-6_43 ↗
78. Tanaka, Y., Iwata, T., Kurashima, T., Toda, H., and Ueda, N. (2023, January 19–25). Estimating latent people flow without tracking individuals. Proceedings of the 32nd International Joint Conference on Artificial Intelligence, Macao, China.
79. A flexible new technique for camera calibration10.1109/34.888718 ↗
80. Zach, C. (2014, January 5–12). Robust bundle adjustment revisited. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10602-1_50 ↗
81. Dark Programmer (2023, February 11). Dark Programmer. Darklabel. Available online: https://github.com/darkpgmr/DarkLabel.
82. Hota: A higher order metric for evaluating multi-object tracking10.1007/s11263-020-01375-2 ↗
83. Milan, A., Leal-Taixé, L., Reid, I., Roth, S., and Schindler, K. (2016). Mot16: A benchmark for multi-object tracking. arXiv.
84. Multiple object tracking: A literature review10.1016/j.artint.2020.103448 ↗