An Analysis of Artificial Intelligence Techniques in Surveillance Video Anomaly Detection: A Comprehensive Survey
Source abstract
Abstract
Surveillance cameras have recently been utilized to provide physical security services globally in diverse private and public spaces. The number of cameras has been increasing rapidly due to the need for monitoring and recording abnormal events. This process can be difficult and time-consuming when detecting anomalies using human power to monitor them for special security purposes. Abnormal events deviate from normal patterns and are considered rare. Furthermore, collecting or producing data on these rare events and modeling abnormal data are difficult. Therefore, there is a need to develop an intelligent approach to overcome this challenge. Many research studies have been conducted on detecting abnormal events using machine learning and deep learning techniques. This study focused on abnormal event detection, particularly for video surveillance applications, and included an up-to-date state-of-the-art that extends previous related works. The major objective of this survey was to examine the existing machine learning and deep learning techniques in the literature and the datasets used to detect abnormal events in surveillance videos to show their advantages and disadvantages and summarize the literature studies, highlighting the major challenges.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Network Security and Intrusion Detection · Video Surveillance and Tracking Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Engineering, Faculty of Engineering, Ankara University, 06100 Ankara, Turkey
Department of Cybersecurity, Princess Sumaya University for Technology (PSUT), Amman 11941, Jordan
Department of Networks and Communications Engineering, College of Computer Science and Information Systems, Najran University, Najran 61441, Saudi Arabia
Department of Information Technology, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia
Department of Information Systems, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia
Funding
Deanship of Scientific Research at Najran University
References (100+)
Show source-provided references
1. Kumari, P., Bedi, A.K., and Saini, M. (2021). Multimedia Datasets for Anomaly Detection: A Survey. arXiv.
2. A review of supervised and unsupervised machine learning techniques for suspicious behavior recognition in intelligent surveillance system
3. Zhao, Y. (2021). Deep Learning in Video Anomaly Detection and Its Applications. [Ph.D. Thesis, The University of Liverpool].
4. Abu Al-Haija, Q., and Zein-Sabatto, S. (2020). An Efficient Deep-Learning-Based Detection and Classification System for Cyber-Attacks in IoT Communication Networks. Electronics, 9.10.20944/preprints202011.0508.v2 ↗
5. Procedures for detecting outlying observations in samples10.1080/00401706.1969.10490657 ↗
6. Hawkins, D.M. (1980). Identification of Outliers, Springer.10.1007/978-94-015-3994-4 ↗
7. Barnett, V., and Lewis, T. (1984). Outliers in Statistical Data, Wiley.
8. Anomaly detection in video sequences: A benchmark and computational model10.1049/ipr2.12258 ↗
9. Aldayri, A., and Albattah, W. (2022). Taxonomy of Anomaly Detection Techniques in Crowd Scenes. Sensors, 22.10.3390/s22166080 ↗
10. A Comprehensive Study on Automated Anomaly Detection Techniques in Video Surveillance
11. Anomaly detection: A survey10.1145/1541880.1541882 ↗
12. Comparative Study for Anomaly Detection in Crowded Scenes
13. Wilmet, V., Verma, S., Redl, T., Sandaker, H., and Li, Z. (2021). A Comparison of Supervised and Unsupervised Deep Learning Methods for Anomaly Detection in Images. arXiv.
14. Abu Al-Haija, Q., Alohaly, M., and Odeh, A. (2023). A Lightweight Double-Stage Scheme to Identify Malicious DNS over HTTPS Traffic Using a Hybrid Learning Approach. Sensors, 23.10.3390/s23073489 ↗
15. Medel, J.R. (2016). Anomaly Detection Using Predictive Convolutional Long Short-Term Memory Units, Rochester Institute of Technology.
16. Attention-based residual autoencoder for video anomaly detection10.1007/s10489-022-03613-1 ↗
17. Real-time anomaly detection on surveillance video with two-stream spatiotemporal generative model10.1007/s00530-022-00979-7 ↗
18. Long-term multiobject tracking using alternative correlation filters10.3906/elk-1709-245 ↗
19. Video synopsis: A survey10.1016/j.cviu.2019.02.004 ↗
20. Ramachandra, B., Jones, M., and Vatsavai, R. (2020, January 1–5). Learning a distance function with a Siamese network to localize anomalies in videos. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass Village, CO, USA.10.1109/WACV45572.2020.9093417 ↗
21. Colque, R.V.H.M. (2023, January 12). Robust Approaches for Anomaly Detection Applied to Video Surveillance. Available online: https://repositorio.ufmg.br/manutencao/.
22. Chen, C.H. (2015). Handbook of Pattern Recognition and Computer Vision, World Scientific.10.1142/9503 ↗
23. A Survey of Deep Learning Solutions for Anomaly Detection in Surveillance Videos
24. Alsulami, A.A., Abu Al-Haija, Q., Tayeb, A., and Alqahtani, A. (2020). An Intrusion Detection and Classification System for IoT Traffic with Improved Data Engineering. Appl. Sci., 12.10.3390/app122312336 ↗
25. An overview of the supervised machine learning methods10.20544/HORIZONS.B.04.1.17.P05 ↗
26. Improving supervised learning classification methods using multigranular linguistic modeling and fuzzy entropy
27. A taxonomy of traffic forecasting regression problems from a supervised learning perspective10.1109/ACCESS.2019.2917228 ↗
28. Multi-frame feature-fusion-based model for violence detection10.1007/s00371-020-01878-6 ↗
29. Abnormal event detection based on analysis of movement information of video sequence10.1016/j.ijleo.2017.07.064 ↗
30. Patil, N., and Biswas, P.K. (2016, January 15–17). Global abnormal events detection in surveillance video—A hierarchical approach. Proceedings of the 2016 Sixth International Symposium on Embedded Computing and System Design (ISED), Patna, India.10.1109/ISED.2016.7977085 ↗
31. Multiple Hierarchical Dirichlet Processes for anomaly detection in traffic10.1016/j.cviu.2018.01.011 ↗
32. High-performance intrusion detection system for networked UAVs via deep learning10.1007/s00521-022-07015-9 ↗
33. Sultani, W., Chen, C., and Shah, M. (2018, January 18–22). Real-world anomaly detection in surveillance videos. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00678 ↗
34. Alsulami, A.A., Abu Al-Haija, Q., Alqahtani, A., and Alsini, R. (2022). Symmetrical Simulation Scheme for Anomaly Detection in Autonomous Vehicles Based on LSTM Model. Symmetry, 14.10.20944/preprints202207.0039.v1 ↗
35. Semi-supervised and unsupervised extreme learning machines10.1109/TCYB.2014.2307349 ↗
36. Chriki, A., Touati, H., Snoussi, H., and Kamoun, F. (2020, January 7–10). Uav-based surveillance system: An anomaly detection approach. Proceedings of the 2020 IEEE Symposium on Computers and Communications (ISCC), Rennes, France.10.1109/ISCC50000.2020.9219585 ↗
37. Deep learning and handcrafted features for one-class anomaly detection in UAV video10.1007/s11042-020-09774-w ↗
38. Al-Qudah, M., Ashi, Z., Alnabhan, M., and Abu Al-Haija, Q. (2022). Effective One-Class Classifier Model for Memory Dump Malware Detection. J. Sens. Actuator Netw., 12.10.20944/preprints202210.0426.v1 ↗
39. Automatic incident classification for large-scale traffic data by adaptive boosting SVM10.1016/j.ins.2018.07.044 ↗
40. Ravanbakhsh, M., Nabi, M., Sangineto, E., Marcenaro, L., Regazzoni, C., and Sebe, N. (2017, January 17–20). Abnormal event detection in videos using generative adversarial nets. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.10.1109/ICIP.2017.8296547 ↗
41. A survey on semi-supervised learning10.1007/s10994-019-05855-6 ↗
42. Bhakat, S., and Ramakrishnan, G. (2019, January 3–5). Anomaly detection in surveillance videos. Proceedings of the ACM India Joint International Conference on Data Science and Management of Data, Kolkata, India.10.1145/3297001.3297034 ↗
43. Anomaly detection in road traffic using visual surveillance: A survey10.1145/3417989 ↗
44. Anomaly detection using local kernel density estimation and context-based regression10.1109/TKDE.2018.2882404 ↗
45. Local anomaly detection in hybrid rocket combustion tests10.1007/s00348-021-03236-1 ↗
46. Crowd anomaly detection and localization using histogram of magnitude and momentum10.1007/s00371-019-01647-0 ↗
47. Combining motion and appearance cues for anomaly detection10.1016/j.patcog.2015.09.005 ↗
48. Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes10.1016/j.cviu.2018.02.006 ↗
49. Coherence pursuit: Fast, simple, and robust principal component analysis10.1109/TSP.2017.2749215 ↗
50. Machine learning: Algorithms, real-world applications and research directions10.1007/s42979-021-00592-x ↗
51. Detection of abnormal visual events via global optical flow orientation histogram10.1109/TIFS.2014.2315971 ↗
52. Online anomaly detection in surveillance videos with asymptotic bound on false alarm rate10.1016/j.patcog.2021.107865 ↗
53. Aboah, A. (2021, January 20–25). A vision-based system for traffic anomaly detection using deep learning and decision trees. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPRW53098.2021.00475 ↗
54. Saeedi, J., and Giusti, A. (2022, January 6–8). Anomaly Detection for Industrial Inspection using Convolutional Autoencoder and Deep Feature-based One-class Classification. Proceedings of the VISIGRAPP (5: VISAPP), Online.10.5220/0010780200003124 ↗
55. Chen, Y., Tian, Y., Pang, G., and Carneiro, G. (2022, January 7–14). Deep one-class classification via interpolated gaussian descriptor. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.10.1609/aaai.v36i1.19915 ↗
56. Lee, K., Lee, H., Lee, K., and Shin, J. (2017). Training confidence-calibrated classifiers for detecting out-of-distribution samples. arXiv.
57. Liu, W., Luo, W., Lian, D., and Gao, S. (2018, January 18–22). Future frame prediction for anomaly detection–a new baseline. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00684 ↗
58. Discriminative reconstruction constrained generative adversarial network for hyperspectral anomaly detection10.1109/TGRS.2020.2965961 ↗
59. SecureAD: A secure video anomaly detection framework on convolutional neural network in edge computing environment10.1109/TCC.2020.2990946 ↗
60. Zhao, Y., Deng, B., Shen, C., Liu, Y., Lu, H., and Hua, X.S. (2017, January 23–27). spatiotemporal autoencoder for video anomaly detection. Proceedings of the 25th ACM International Conference on Multimedia, Mountain View, CA, USA.10.1145/3123266.3123451 ↗
61. Xu, D., Ricci, E., Yan, Y., Song, J., and Sebe, N. (2015). Learning deep representations of appearance and motion for anomalous event detection. arXiv.10.5244/C.29.8 ↗
62. Video anomaly detection and localization via gaussian mixture fully convolutional variational autoencoder10.1016/j.cviu.2020.102920 ↗
63. Anomaly detection in videos using optical flow and convolutional autoencoder10.1109/ACCESS.2019.2960654 ↗
64. Madan, N., Farkhondeh, A., Nasrollahi, K., Escalera, S., and Moeslund, T.B. (2021, January 11–17). Temporal cues from socially unacceptable trajectories for anomaly detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada.10.1109/ICCVW54120.2021.00244 ↗
65. Learning normal patterns via adversarial attention-based autoencoder for abnormal event detection in videos10.1109/TMM.2019.2950530 ↗
66. Adversarial 3d convolutional auto-encoder for abnormal event detection in videos10.1109/TMM.2020.3023303 ↗
67. Nguyen, T.N., and Meunier, J. (November, January 27). Anomaly detection in video sequence with appearance-motion correspondence. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
68. Feng, X., Song, D., Chen, Y., Chen, Z., Ni, J., and Chen, H. (2021, January 21–25). Convolutional transformer based dual discriminator generative adversarial networks for video anomaly detection. Proceedings of the 29th ACM International Conference on Multimedia, Nice, France.10.1145/3474085.3475693 ↗
69. Lee, J., Nam, W.J., and Lee, S.W. (2022, January 21–25). Multi-Contextual Predictions with Vision Transformer for Video Anomaly Detection. Proceedings of the 2022 26th International Conference on Pattern Recognition (ICPR), Montreal, QC, Canada.10.1109/ICPR56361.2022.9956507 ↗
70. Transanomaly: Video anomaly detection using video vision transformer10.1109/ACCESS.2021.3109102 ↗
71. Action recognition in video sequences using deep bi-directional LSTM with CNN features10.1109/ACCESS.2017.2778011 ↗
72. Unsupervised anomaly detection with LSTM neural networks10.1109/TNNLS.2019.2935975 ↗
73. Ristea, N.C., Madan, N., Ionescu, R.T., Nasrollahi, K., Khan, F.S., Moeslund, T.B., and Shah, M. (2022, January 18–24). Self-supervised predictive convolutional attentive block for anomaly detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01321 ↗
74. Anomalynet: An anomaly detection network for video surveillance10.1109/TIFS.2019.2900907 ↗
75. Spatiotemporal anomaly detection using deep learning for real-time video surveillance10.1109/TII.2019.2938527 ↗
76. Ullah, W., Ullah, A., Hussain, T., Khan, Z.A., and Baik, S.W. (2021). An efficient anomaly recognition framework using an attention residual LSTM in surveillance videos. Sensors, 21.10.3390/s21082811 ↗
77. Ranjith, R., Athanesious, J.J., and Vaidehi, V. (2015, January 15–17). Anomaly detection using DBSCAN clustering technique for traffic video surveillance. Proceedings of the 2015 Seventh International Conference on Advanced Computing (ICoAC), Chennai, India.10.1109/ICoAC.2015.7562795 ↗
78. Road traffic anomaly detection based on fuzzy theory10.1109/ACCESS.2018.2851747 ↗
79. Chang, M.C., Wei, Y., Song, N., and Lyu, S. (2018, January 18–22). Video analytics in smart transportation for the AIC’18 challenge. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA.10.1109/CVPRW.2018.00016 ↗
80. Lu, C., Shi, J., and Jia, J. (2013, January 1–8). Abnormal event detection at 150 fps in matlab. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia.10.1109/ICCV.2013.338 ↗
81. Robust real-time unusual event detection using multiple fixed-location monitors10.1109/TPAMI.2007.70825 ↗
82. Mahadevan, V., Li, W., Bhalodia, V., and Vasconcelos, N. (2010, January 13–18). Anomaly detection in crowded scenes. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5539872 ↗
83. Lee, S., Kim, H.G., and Ro, Y.M. (2018, January 15–20). STAN: Spatiotemporal adversarial networks for abnormal event detection. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462388 ↗
84. Mehran, R., Oyama, A., and Shah, M. (2009, January 20–25). Abnormal crowd behavior detection using social force model. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPR.2009.5206641 ↗
85. Boekhoudt, K., Matei, A., Aghaei, M., and Talavera, E. (2021, January 28–30). HR-Crime: Human-Related Anomaly Detection in Surveillance Videos. Proceedings of the International Conference on Computer Analysis of Images and Patterns, Virtual.10.1007/978-3-030-89131-2_15 ↗
86. Generative neural networks for anomaly detection in crowded scenes10.1109/TIFS.2018.2878538 ↗
87. Del Giorno, A., Bagnell, J.A., and Hebert, M. (2016, January 11–14). A discriminative framework for anomaly detection in large videos. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.10.1007/978-3-319-46454-1_21 ↗
88. Tudor Ionescu, R., Smeureanu, S., Alexe, B., and Popescu, M. (2017, January 22–29). Unmasking the abnormal events in video. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.10.1109/ICCV.2017.315 ↗
89. Wang, S., Zhu, E., Yin, J., and Porikli, F. (2016, January 4–8). Anomaly detection in crowded scenes by SL-HOF descriptor and foreground classification. Proceedings of the 2016 23rd International Conference on Pattern Recognition (ICPR), Cancun, Mexico.
90. Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A.K., and Davis, L.S. (2016, January 27–30). Learning temporal regularity in video sequences. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NA, USA.10.1109/CVPR.2016.86 ↗
91. Luo, W., Liu, W., and Gao, S. (2017, January 10–14). Remembering history with convolutional lstm for anomaly detection. Proceedings of the 2017 IEEE International Conference on Multimedia and Expo (ICME), San Diego, CA, USA.10.1109/ICME.2017.8019325 ↗
92. Vu, H., Nguyen, T.D., Travers, A., Venkatesh, S., and Phung, D. (2017, January 22–25). Energy-based localized anomaly detection in video surveillance. Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining, Nanjing, China.10.1007/978-3-319-57454-7_50 ↗
93. Chong, Y.S., and Tay, Y.H. Abnormal event detection in videos using spatiotemporal autoencoder. Advances in Neural Networks—ISNN 2017.10.1007/978-3-319-59081-3_23 ↗
94. Anomaly detection in moving crowds through spatiotemporal autoencoding and additional attention10.1155/2018/2087574 ↗
95. Tudor Ionescu, R., Smeureanu, S., Popescu, M., and Alexe, B. (2018). Detecting abnormal events in video using Narrowed Normality Clusters. arXiv, arXiv1801.10.1109/WACV.2019.00212 ↗
96. Zhong, J.X., Li, N., Kong, W., Liu, S., Li, T.H., and Li, G. (2019, January 15–20). Graph convolutional label noise cleaner: Train a plug-and-play action classifier for anomaly detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.10.1109/CVPR.2019.00133 ↗
97. Gong, D., Liu, L., Le, V., Saha, B., Mansour, M.R., Venkatesh, S., and Hengel, A.V.D. (November, January 27). Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.
98. Video anomaly detection with sparse coding inspired deep neural networks10.1109/TPAMI.2019.2944377 ↗
99. Attention-driven loss for anomaly detection in video surveillance10.1109/TCSVT.2019.2962229 ↗
100. BMAN: Bidirectional multi-scale aggregation networks for abnormal event detection10.1109/TIP.2019.2948286 ↗