Investigations of Object Detection in Images/Videos Using Various Deep Learning Techniques and Embedded Platforms—A Comprehensive Review
Source abstract
Abstract
In recent years there has been remarkable progress in one computer vision application area: object detection. One of the most challenging and fundamental problems in object detection is locating a specific object from the multiple objects present in a scene. Earlier traditional detection methods were used for detecting the objects with the introduction of convolutional neural networks. From 2012 onward, deep learning-based techniques were used for feature extraction, and that led to remarkable breakthroughs in this area. This paper shows a detailed survey on recent advancements and achievements in object detection using various deep learning techniques. Several topics have been included, such as Viola–Jones (VJ), histogram of oriented gradient (HOG), one-shot and two-shot detectors, benchmark datasets, evaluation metrics, speed-up techniques, and current state-of-art object detectors. Detailed discussions on some important applications in object detection areas, including pedestrian detection, crowd detection, and real-time object detection on Gpu-based embedded systems have been presented. At last, we conclude by identifying promising future directions.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Neural Network Applications · Video Surveillance and Tracking Methods · Anomaly Detection Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electronics and Communication Engineering, National Institute of Technology, Warangal 506004, India
Department of Engineering—Renewable Energy and Thermodynamics, Aarhus University, 8000 Aarhus, Denmark
Department of Energy IT, Gachon University, Seongnam 13120, Korea
Funding
National Research Foundation of Korea
References (100+)
Show source-provided references
1. Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, Curran Associates, Inc.
2. Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, Curran Associates, Inc.
3. Nguyen, H.T., Lee, E.H., and Lee, S. (2020). Study on the Classification Performance of Underwater Sonar Image Classification Based on Convolutional Neural Networks for Detecting a Submerged Human Body. Sensors, 20.10.3390/s20010094 ↗
4. Imagenet large scale visual recognition challenge10.1007/s11263-015-0816-y ↗
5. The pascal visual object classes (voc) challenge10.1007/s11263-009-0275-4 ↗
6. Harmony filter: A robust visual tracking system using the improved harmony search algorithm10.1016/j.imavis.2010.05.006 ↗
7. Circle detection by harmony search optimization10.1007/s10846-011-9611-3 ↗
8. Deep learning10.1038/nature14539 ↗
9. Reducing the dimensionality of data with neural networks10.1126/science.1127647 ↗
10. Background subtraction techniques
11. Viola, P., and Jones, M. (2001, January 8–14). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, USA.
12. Robust real-time face detection10.1023/B:VISI.0000013087.49260.fb ↗
13. Dalal, N., and Triggs, B. (2005, January 20–25). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Sonference on Computer Vision and Pattern Recognition, San Diego, CA, USA.
14. Felzenszwalb, P., McAllester, D., and Ramanan, D. (2008, January 23–28). A discriminatively trained, multiscale, deformable part model. Proceedings of the Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA.10.1109/CVPR.2008.4587597 ↗
15. Felzenszwalb, P.F., Girshick, R.B., and McAllester, D. (2010, January 13–18). Cascade object detection with deformable part models. Proceedings of the Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5539906 ↗
16. Object detection with discriminatively trained part-based models10.1109/TPAMI.2009.167 ↗
17. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 24–27). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
18. Zou, Z., Shi, Z., Guo, Y., and Ye, J. (2019). Object detection in 20 years: A survey. arXiv.
19. Deep learning for generic object detection: A survey10.1007/s11263-019-01247-4 ↗
20. Application of deep learning for object detection10.1016/j.procs.2018.05.144 ↗
21. Sultana, F., Sufian, A., and Dutta, P. (2019). A review of object detection models based on convolutional neural network. arXiv.10.1007/978-981-15-4288-6_1 ↗
22. Object detection with deep learning: A review10.1109/TNNLS.2018.2876865 ↗
23. Mittal, U., Srivastava, S., and Chawla, P. (2019, January 15–16). Review of different techniques for object detection using deep learning. Proceedings of the Third International Conference on Advanced Informatics for Computing Research, Shimla, India.10.1145/3339311.3339357 ↗
24. Lowe, D.G. (1999, January 20–25). Object recognition from local scale-invariant features. Proceedings of the International Conference on Computer Vision, Kerkyra, Corfu, Greece.10.1109/ICCV.1999.790410 ↗
25. Distinctive image features from scale-invariant keypoints10.1023/B:VISI.0000029664.99615.94 ↗
26. Shape matching and object recognition using shape contexts10.1109/34.993558 ↗
27. Girshick, R.B., Felzenszwalb, P.F., and Mcallester, D.A. (2011). Object detection with grammar models. Advances in Neural Information Processing Systems, Curran Associates Inc.
28. Girshick, R.B. (2012). From Rigid Templates to Grammars: Object Detection with Structured Models. [Ph.D. Thesis, The University of Chicago].
29. Li, Y.F., Kwok, J.T., Tsang, I.W., and Zhou, Z.H. (2009, January 14–18). A convex method for locating regions of interest with multi-instance learning. Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Antwerp, Belgium.10.1007/978-3-642-04174-7_2 ↗
30. Selective search for object recognition10.1007/s11263-013-0620-5 ↗
31. Girshick, R.B., Felzenszwalb, P.F., and McAllester, D. (2020, May 07). Discriminatively Trained Deformable Part Models, Release 5. Available online: http://people.cs.uchicago.edu/~rbg/latent-release5/.
32. Spatial pyramid pooling in deep convolutional networks for visual recognition10.1109/TPAMI.2015.2389824 ↗
33. Girshick, R. (2015, January 13–16). Fast R-CNN. Proceedings of the International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.169 ↗
34. Zeiler, M.D., and Fergus, R. (2014, January 14–18). Visualizing and understanding convolutional networks. Proceedings of the European Conference on Computer Vision, Nancy, France.10.1007/978-3-319-10590-1_53 ↗
35. Dai, J., Li, Y., He, K., and Sun, J. (2016). R-FCN: Object detection via region-based fully convolutional networks. Advances in Neural Information Processing Systems, Curran Associates Inc.
36. Li, Z., Peng, C., Yu, G., Zhang, X., Deng, Y., and Sun, J. (2017). Light-head R-CNN: In defense of two-stage object detector. arXiv.
37. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature pyramid networks for object detection. Proceedings of the Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
38. He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017, January 22–29). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.322 ↗
39. He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
40. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (July, January 26). You only look once: Unified, real-time object detection. Proceedings of the Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
41. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 8–16). SSD: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
42. Redmon, J., and Farhadi, A. (2017, January 21–26). YOLO9000: Better, faster, stronger. Proceedings of the Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.690 ↗
43. Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv.
44. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017, January 22–29). Focal loss for dense object detection. Proceedings of the International Cconference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.324 ↗
45. Wu, B., Iandola, F., Jin, P.H., and Keutzer, K. (2017, January 21–26). SqueezeDet: Unified, small, low power fully convolutional neural networks for real-time object detection for autonomous driving. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.10.1109/CVPRW.2017.60 ↗
46. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size. arXiv.
47. Xia, G.S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., and Zhang, L. (2018, January 18–23). DOTA: A large-scale dataset for object detection in aerial images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00418 ↗
48. Law, H., and Deng, J. (2018, January 8–14). Cornernet: Detecting objects as paired keypoints. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.10.1007/978-3-030-01264-9_45 ↗
49. Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., and Tian, Q. (2019). Centernet: Object detection with keypoint triplets. arXiv.10.1109/ICCV.2019.00667 ↗
50. Fu, C.Y., Liu, W., Ranga, A., Tyagi, A., and Berg, A.C. (2017). Dssd: Deconvolutional single shot detector. arXiv.
51. (2019, December 31). Mathematica. Available online: https://www.wolfram.com/mathematica/.
52. (2019, December 31). Dlib. Available online: Dlib.net.
53. (2019, December 31). Theano. Available online: http://deeplearning.net/software/theano/.
54. (2019, December 31). Caffe. Available online: http://caffe.berkeleyvision.org/.
55. (2019, December 31). Deeplearning4j. Available online: https://deeplearning4j.org.
56. (2019, December 31). Cahiner. Available online: https://chainer.org.
57. (2019, December 31). Keras. Available online: https://keras.io/.
58. (2019, December 31). Mathworks—Deep Learning. Available online: https://in.mathworks.com/solutions/deep-learning.html.
59. (2019, December 31). Apache. Available online: http://singa.apache.org.
60. (2019, December 31). TensorFlow. Available online: https://www.tensorflow.org/.
61. (2019, December 31). Pytorch. Available online: https://pytorch.org.
62. (2019, December 31). BigDL. Available online: https://github.com/intel-analytics/BigDL.
63. (2019, December 31). Apache. Available online: http://www.apache.org.
64. (2019, December 31). MXnet. Available online: http://mxnet.io/.
65. (2019, December 31). Microsoft Cognitive Service. Available online: https://www.microsoft.com/cognitive-services/en-us/computer-vision-api.
66. (2019, December 31). Amazon Recognition. Available online: https://aws.amazon.com/rekognition/.
67. (2019, December 31). IBM Watson Vision Recognition service. Available online: http://www.ibm.com/watson/developercloud/visual-recognition.html.
68. (2019, December 31). Google Cloud Vision API. Available online: https://cloud.google.com/vision/.
69. (2019, December 31). Cloud Sight. Available online: https://cloudsight.readme.io/v1.0/docs.
70. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20–25). Imagenet: A large-scale hierarchical image database. Proceedings of the Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206848 ↗
71. Yang, S., Luo, P., Loy, C.C., and Tang, X. (July, January 26). Wider face: A face detection benchmark. Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA.
72. Jain, V., and Learned-Miller, E. (2010). Fddb: A Benchmark for Face Detection in Unconstrained Settings, UMass Amherst Libraries. Technical Report, UMass Amherst Technical Report.
73. Zhang, S., Benenson, R., and Schiele, B. (2017, January 21–26). Citypersons: A diverse dataset for pedestrian detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.474 ↗
74. Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (July, January 26). The cityscapes dataset for semantic urban scene understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
75. Ess, A., Leibe, B., and Van Gool, L. (2007, January 14–21). Depth and appearance for mobile scene analysis. Proceedings of the 2007 IEEE International Conference on Computer Vision, Rio de Janeiro, Brazil.10.1109/ICCV.2007.4409092 ↗
76. 80 million tiny images: A large data set for nonparametric object and scene recognition10.1109/TPAMI.2008.128 ↗
77. Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C.L. (2014, January 5–12). Microsoft coco: Common objects in context. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10602-1_48 ↗
78. Krizhevsky, A. (2009). Learning Multiple Layers of Features from Tiny Images. [Master’s Thesis, University of Tront].
79. Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S., Goering, C., Berg, T., and Belhumeur, P. (2011). Caltech-UCSD Birds-200-2011, California Institute of Technology.
80. Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., and Perona, P. (2010). Caltech-UCSD birds 200, California Institute of Technology.
81. Griffin, G., Holub, A., and Perona, P. (2007). Caltech-256 Object Category Dataset, California Institute of Technology.
82. (2019, December 31). ILSVRC Detection Challenge Results. Available online: http://www.image-net.org/challenges/LSVRC/.
83. The pascal visual object classes challenge: A retrospective10.1007/s11263-014-0733-5 ↗
84. LabelMe: A database and web-based tool for image annotation10.1007/s11263-007-0090-8 ↗
85. Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., and Torralba, A. (2010, January 13–18). Sun database: Large-scale scene recognition from abbey to zoo. Proceedings of the Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5539970 ↗
86. (2019, December 31). Open Images. Available online: https://www.kaggle.com/bigquery/open-images.
87. Kragh, M.F., Christiansen, P., Laursen, M.S., Larsen, M., Steen, K.A., Green, O., Karstoft, H., and Jørgensen, R.N. (2017). FieldSAFE: Dataset for obstacle detection in agriculture. Sensors, 17.10.3390/s17112579 ↗
88. Grady, N.W., Underwood, M., Roy, A., and Chang, W.L. (2014, January 27–30). Big data: Challenges, practices and technologies: NIST big data public working group workshop at IEEE big data 2014. Proceedings of the International Conference on Big Data, Washington, DC, USA.10.1109/BigData.2014.7004470 ↗
89. Dollár, P., Tu, Z., Perona, P., and Belongie, S. (2009). Integral Channel Features, BMVC Press.10.5244/C.23.91 ↗
90. Maji, S., Berg, A.C., and Malik, J. (2008, January 23–28). Classification using intersection kernel support vector machines is efficient. Proceedings of the Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA.10.1109/CVPR.2008.4587630 ↗
91. Zhu, Q., Yeh, M.C., Cheng, K.T., and Avidan, S. (2006, January 17–22). Fast human detection using a cascade of histograms of oriented gradients. Proceedings of the Computer Society Conference on Computer Vision and Pattern Recognition, New York, NY, USA.
92. Example-based object detection in images by components10.1109/34.917571 ↗
93. Wang, X., Han, T.X., and Yan, S. (October, January 29). An HOG-LBP human detector with partial occlusion handling. Proceedings of the International Conference on Computer Vision, Kyoto, Japan.
94. Wu, B., and Nevatia, R. (2005, January 17–21). Detection of multiple, partially occluded humans in a single image by bayesian combination of edgelet part detectors. Proceedings of the International Conference on Computer Vision, Beijing, China.
95. 50 years of object recognition: Directions forward10.1016/j.cviu.2013.04.005 ↗
96. Sadeghi, M.A., and Forsyth, D. (2014, January 6–12). 30hz object detection with dpm v5. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.10.1007/978-3-319-10590-1_5 ↗
97. Hosang, J., Omran, M., Benenson, R., and Schiele, B. (2015, January 7–12). Taking a deeper look at pedestrians. Proceedings of the Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.10.1109/CVPR.2015.7299034 ↗
98. An improved tiny-yolov3 pedestrian detection algorithm10.1016/j.ijleo.2019.02.038 ↗
99. Zhang, L., Lin, L., Liang, X., and He, K. (2016, January 8–16). Is faster R-CNNN doing well for pedestrian detection?. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46475-6_28 ↗
100. Song, T., Sun, L., Xie, D., Sun, H., and Pu, S. (2018). Small-scale pedestrian detection based on somatic topology localization and temporal feature aggregation. arXiv.10.1007/978-3-030-01234-2_33 ↗