Pedestrian Flow Tracking and Statistics of Monocular Camera Based on Convolutional Neural Network and Kalman Filter
Source abstract
Abstract
Pedestrian flow statistics and analysis in public places is an important means to ensure urban safety. However, in recent years, a video-based pedestrian flow statistics algorithm mainly relies on binocular vision or a vertical downward camera, which has serious limitations on the application scene and counting area, and cannot make use of the large number of monocular cameras in the city. To solve this problem, we propose a pedestrian flow statistics algorithm based on monocular camera. Firstly, a convolution neural network is used to detect the pedestrian targets. Then, with a Kalman filter, the motion models for the targets are established. Based on these motion models, data association algorithm completes target tracking. Finally, the pedestrian flow is counted by the pedestrian counting method based on virtual blocks. The algorithm is tested on real scenes and public data sets. The experimental results show that the algorithm has high accuracy and strong real-time performance, which verifies the reliability of the algorithm.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Video Surveillance and Tracking Methods · Fire Detection and Safety Systems · Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China
Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110016, China
University of Chinese Academy of Sciences, Beijing 100049, China
Key Laboratory of Opto-Electronic Information Processing, Chinese Academy of Science, Shenyang 110016, China
The Key Lab of Image Understanding and Computer Vision, Shenyang 110016, China
References (41)
Show source-provided references
1. Counting people by RGB or depth overhead cameras10.1016/j.patrec.2016.05.033 ↗
2. Coşkun, A., Kara, A., Parlaktuna, M., Ozkan, M., and Parlaktuna, O. (2015, January 2–4). People counting system by using kinect sensor. Proceedings of the 2015 IEEE International Symposium on Innovations in Intelligent SysTems and Applications (INISTA), Taipei, Taiwan.10.1109/INISTA.2015.7276740 ↗
3. Verma, N.K., Dev, R., Maurya, S., Dhar, N.K., and Agrawal, P. (2019). People Counting with Overhead Camera Using Fuzzy-Based Detector. Computational Intelligence: Theories, Applications and Future Directions—Volume I, Springer.10.1007/978-981-13-1132-1_46 ↗
4. A method for counting people attending large public events10.1007/s11042-013-1628-0 ↗
5. Beymer, D. (2000, January 7–8). Person counting using stereo. Proceedings of the Workshop on Human Motion, Austin, TX, USA.
6. Gradient-based learning applied to document recognition10.1109/5.726791 ↗
7. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3–6). Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA.
8. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
9. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference On Computer Vision And Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7298594 ↗
10. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference On Computer Vision And Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
11. Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21–26). Densely connected convolutional networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.10.1109/CVPR.2017.243 ↗
12. Bochinski, E., Eiselein, V., and Sikora, T. (September, January 29). High-speed tracking-by-detection without using image information. Proceedings of the 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Lecce, Italy.10.1109/AVSS.2017.8078516 ↗
13. Stauffer, C., and Grimson, W.E.L. (1999, January 23–25). Adaptive background mixture models for real-time tracking. Proceedings of the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149), Fort Collins, CO, USA.
14. Barnich, O., and Van Droogenbroeck, M. (2009, January 19–24). ViBe: A powerful random technique to estimate the background in video sequences. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2009, Taipei, Taiwan.10.1109/ICASSP.2009.4959741 ↗
15. Dalal, N., and Triggs, B. (2005, January 20–25). Histograms of oriented gradients for human detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR 2005, San Diego, CA, USA.
16. Zhou, C., and Yuan, J. (2018, January 8–14). Bi-box Regression for Pedestrian Detection and Occlusion Estimation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01246-5_9 ↗
17. Faster R-CNN: Towards real-time object detection with region proposal networks10.1109/TPAMI.2016.2577031 ↗
18. Dai, J., Li, Y., He, K., and Sun, J. (2016, January 5–10). R-fcn: Object detection via region-based fully convolutional networks. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain.
19. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature pyramid networks for object detection. Proceedings of the CVPR, Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
20. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 8–16). Ssd: Single shot multibox detector. Proceedings of the European Conference On Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
21. Fu, C.Y., Liu, W., Ranga, A., Tyagi, A., and Berg, A.C. (2017). DSSD: Deconvolutional single shot detector. arXiv.
22. Li, Z., and Zhou, F. (2017). FSSD: Feature Fusion Single Shot Multibox Detector. arXiv.
23. Single and multiple object tracking using log-Euclidean Riemannian subspace and block-division appearance model10.1109/TPAMI.2012.42 ↗
24. Zhang, L., and van der Maaten, L. (2013, January 23–28). Structure preserving object tracking. Proceedings of the IEEE Conference On Computer Vision And Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.240 ↗
25. Multiple object tracking using k-shortest paths optimization10.1109/TPAMI.2011.21 ↗
26. Breitenstein, M.D., Reichlin, F., Leibe, B., Koller-Meier, E., and Van Gool, L. (October, January 29). Robust tracking-by-detection using a detector confidence particle filter. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan.10.1109/ICCV.2009.5459278 ↗
27. Robust multiperson tracking from a mobile platform10.1109/TPAMI.2009.109 ↗
28. A general framework for tracking multiple people from a moving camera10.1109/TPAMI.2012.248 ↗
29. MCMC-based particle filtering for tracking a variable number of interacting targets10.1109/TPAMI.2005.223 ↗
30. Kuo, C.H., Huang, C., and Nevatia, R. (2010, January 13–18). Multi-target tracking by on-line learned discriminative appearance models. Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA.10.1109/CVPR.2010.5540148 ↗
31. Continuous energy minimization for multitarget tracking10.1109/TPAMI.2013.103 ↗
32. Ullah, M., Cheikh, F.A., and Imran, A.S. (2016, January 23–26). Hog based real-time multi-target tracking in bayesian framework. Proceedings of the 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Colorado Springs, CO, USA.10.1109/AVSS.2016.7738080 ↗
33. Confidence-based data association and discriminative deep appearance learning for robust online multi-object tracking10.1109/TPAMI.2017.2691769 ↗
34. Kryjak, T., Komorkiewicz, M., and Gorgon, M. (2014, January 8–10). Hardware-software implementation of vehicle detection and counting using virtual detection lines. Proceedings of the 2014 Conference on IEEE Design and Architectures for Signal and Image Processing (DASIP), Madrid, Spain.10.1109/DASIP.2014.7115618 ↗
35. Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv.
36. He, Y., Zhang, X., and Sun, J. (2017, January 22–29). Channel pruning for accelerating very deep neural networks. Proceedings of the International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.155 ↗
37. Bodla, N., Singh, B., Chellappa, R., and Davis, L.S. (2017, January 22–29). Soft-nms—Improving object detection with one line of code. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.593 ↗
38. Leal-Taixé, L., Milan, A., Reid, I., Roth, S., and Schindler, K. (2015). Motchallenge 2015: Towards a benchmark for multi-target tracking. arXiv.
39. Xiang, Y., Alahi, A., and Savarese, S. (2015, January 11–18). Learning to track: Online multi-object tracking by decision making. Proceedings of the IEEE International Conference On Computer Vision, RegióN Metropolitana, Chile.10.1109/ICCV.2015.534 ↗
40. Kim, H.U., and Kim, C.S. (2016, January 8–16). CDT: Cooperative detection and tracking for tracing multiple objects in video sequences. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46466-4_51 ↗
41. Bewley, A., Ge, Z., Ott, L., Ramos, F., and Upcroft, B. (2016, January 25–28). Simple online and realtime tracking. Proceedings of the 2016 IEEE International Conference on. IEEE Image Processing (ICIP), Phoenix, AZ, USA.10.1109/ICIP.2016.7533003 ↗