A Review of Abnormal Crowd Behavior Recognition Technology Based on Computer Vision
Source abstract
Abstract
Abnormal crowd behavior recognition is one of the research hotspots in computer vision. Its goal is to use computer vision technology and abnormal behavior detection models to accurately perceive, predict, and intervene in potential abnormal behaviors of the crowd and monitor the status of the crowd system in public places in real time, to effectively prevent and deal with public security risks and ensure public life safety and social order. To this end, focusing on the abnormal crowd behavior recognition technology in the computer vision system, a systematic review study of its theory and cutting-edge technology is conducted. First, the crowd level and abnormal behaviors in public places are defined, and the challenges faced by abnormal crowd behavior recognition are expounded. Then, from the dimensions based on traditional methods and based on deep learning, the mainstream technologies of abnormal behavior recognition are discussed, and the design ideas, advantages, and limitations of various methods are analyzed. Next, the mainstream software tools are introduced to provide a comprehensive reference for the technical framework. Secondly, typical abnormal behavior datasets at home and abroad are sorted out, and the characteristics of these datasets are compared in detail from multiple perspectives such as scale, characteristics, and uses, and the performance indicators of different algorithms on the datasets are compared and analyzed. Finally, the full text is summarized and the future development direction of abnormal crowd behavior recognition technology is prospected.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Video Surveillance and Tracking Methods · Digital Media Forensic Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electronic and Information Engineering, Tongji University, Shanghai 201804, China
Hongkong Institute of Water and Sainitation Safety, Hong Kong 999077, China
Shanghai Karon Eco-Valve Manufacturing Co., Ltd., Shanghai 201804, China
Funding
National Natural Science Foundation of China
References (91)
Show source-provided references
1. Data-based tools can prevent crowd crushes10.1126/science.adf5949 ↗
2. Detecting and Quantifying Crowd-level Abnormal Behaviors in Crowd Events10.1109/TIFS.2024.3423388 ↗
3. Comprehensive systematic review of information fusion methods in smart cities and urban environments10.1016/j.inffus.2024.102317 ↗
4. Research on behavior recognition method based on multi-order information fusion
5. A crowd abnormal behavior detection algorithm based on pulse line flow convolutional neural network
6. Abnormal behavior detection in surveillance videos by fusing bag attention mechanism
7. Alam, E., Sufian, A., Dutta, P., Dutta, P., and Leo, M. (2022). Vision-based human fall detection systems using deep learning: A review. Comput. Biol. Med., 146.10.1016/j.compbiomed.2022.105626 ↗
8. Anomaly Detection and Modeling of Surveillance Video
9. A Weakly-Supervised Crowd Density Estimation Method Based on Two-Stage Linear Feature Calibration10.1109/JAS.2023.123960 ↗
10. Image-based crowd stability analysis using improved multi-column convolutional neural network10.1109/TITS.2021.3054376 ↗
11. Review of pedestrian abnormal behavior recognition methods in public places
12. Small sample learning algorithm based on feature transformation and metric network
13. The violent turn in non-violent collective action: What happens?10.1111/ajsp.12596 ↗
14. Research on passenger flow monitoring and early warning in crowded places based on video analysis
15. Wang, X., and Li, Y. (2024). Edge Detection and Simulation Analysis of Multimedia Images Based on Intelligent Monitoring Robot. Informatica, 48.10.31449/inf.v48i5.5366 ↗
16. High-performance transformer tracking
17. Motion estimation of high-density crowd using fluid dynamics10.1080/13682199.2020.1767843 ↗
18. Zhou, Z., Sun, R., Guo, C., Zhu, Y., and Hierarchical Motion Estimation Method for Spatially Unstable Targets under Gaussian Mixture Model (2024, May 26). Acta Optica Sinica:1-19. Available online: http://kns.cnki.net/kcms/detail/31.1252.O4.20240517.1540.029.html.
19. Crowd descriptors and interpretable gathering understanding10.1109/TMM.2024.3381040 ↗
20. Anomalous event detection and localization in dense crowd scenes10.1007/s11042-022-13967-w ↗
21. Detecting anomalous crowd behavior using correlation analysis of optical flow10.1007/s11760-019-01474-9 ↗
22. Video anomaly detection and localization based on appearance and motion models10.1007/s11042-021-10921-0 ↗
23. A new approach to dominant motion pattern recognition at the macroscopic crowd level10.1016/j.engappai.2022.105387 ↗
24. Object detection and crowd analysis using deep learning techniques: Comprehensive review and future directions10.1016/j.neucom.2024.127932 ↗
25. Self-supervised masked convolutional transformer block for anomaly detection10.1109/TPAMI.2023.3322604 ↗
26. CrowdGAN: Identity-free interactive crowd video generation and beyond10.1109/TPAMI.2020.3043372 ↗
27. Pedestrian trajectory prediction based on deep convolutional LSTM network10.1109/TITS.2020.2981118 ↗
28. Huang, Y., Abeliuk, A., Morstatter, F., Atanasov, P., and Galstyan, A. (2020). Anchor attention for hybrid crowd forecasts aggregation. arXiv.
29. Informative scene decomposition for crowd analysis, comparison and simulation guidance10.1145/3386569.3392407 ↗
30. A review on classifying abnormal behavior in crowd scene10.1016/j.jvcir.2018.11.035 ↗
31. Goel, S., Koundal, D., and Nijhawan, R. (2024). Learning Models in Crowd Analysis: A Review. Arch. Comput. Methods Eng., 1–19.10.1007/s11831-024-10151-1 ↗
32. Abnormal Crowd Behavior Detection Using Optimized Pyramidal Lucas-Kanade Technique10.32604/iasc.2023.029119 ↗
33. A new YOLO-based method for real-time crowd detection from video and performance analysis of YOLO models10.1007/s11554-023-01276-w ↗
34. Abnormal behavior learning based on edge computing toward a crowd monitoring system10.1109/MNET.014.2000523 ↗
35. A motion feature descriptor for abnormal behavior detection
36. Human behavior recognition based on two-stream network fusion and spatio-temporal convolution
37. Abnormal behavior detection using streak flow acceleration10.1007/s10489-021-02881-7 ↗
38. Clarke, K.C. (2021). Cellular automata and agent-based models. Handbook of Regional Science, Springer Berlin Heidelberg.10.1007/978-3-662-60723-7_63 ↗
39. Crowd evacuation model based on improved PSO algorithm
40. A cellular-automaton agent-hybrid model for emergency evacuation of people in public places10.1109/ACCESS.2020.2986012 ↗
41. Dynamic detection of abnormalities in video analysis of crowd behavior with DBSCAN and neural networks10.25046/aj010510 ↗
42. Tay, N.C., Connie, T., Ong, T.S., Goh, K.O.M., and Teh, P.S. (2018, January 29–30). A robust abnormal behavior detection method using convolutional neural network. Proceedings of the Computational Science and Technology: 5th ICCST 2018, Kota Kinabalu, Malaysia.10.1007/978-981-13-2622-6_4 ↗
43. NWPU-crowd: A large-scale benchmark for crowd counting and localization10.1109/TPAMI.2020.3013269 ↗
44. Koonce, B. (2021). ResNet 50. Convolutional Neural Networks with Swift for Tensorflow: Image Recognition and Dataset Categorization, Apress.10.1007/978-1-4842-6168-2 ↗
45. Behera, N.K.S., Sa, P.K., Muhammad, K., and Bakshi, S. (2023). Large-Scale Person Re-Identification for Crowd Monitoring in Emergency. IEEE Transactions on Automation Science and Engineering, IEEE.10.1109/TASE.2023.3318007 ↗
46. Estimating crowd density with edge intelligence based on lightweight convolutional neural networks10.1016/j.eswa.2022.117823 ↗
47. Song, H. ST-InNet: Deep spatio-temporal inception networks for traffic flow prediction in smart cities10.1109/TITS.2022.3179789 ↗
48. Margin Learning Embedded Prediction for Video Anomaly Detection with A Few Anomalies
49. Hajj pilgrimage abnormal crowd movement monitoring using optical flow and FCNN10.1186/s40537-023-00779-4 ↗
50. Garg, S., Sharma, S., Dhariwal, S., Priya, W.D., Singh, M., and Ramesh, S. (2024). Human crowd behaviour analysis based on video segmentation and classification using expectation–maximization with deep learning architectures. Multimed. Tools Appl., 1–23.10.1007/s11042-024-18630-0 ↗
51. Crowd anomaly detection using aggregation of ensembles of fine-tuned convnets10.1016/j.neucom.2019.08.059 ↗
52. Deep-anomaly: Fully convolutional neural network for fast anomaly detection in crowded scenes10.1016/j.cviu.2018.02.006 ↗
53. A weakly supervised framework for abnormal behavior detection and localization in crowded scenes10.1016/j.neucom.2019.11.087 ↗
54. Crowd abnormal behavior detection based on optical flow and trajectory
55. Crowd abnormal behavior detection and location based on deep spatio-temporal convolutional neural network
56. Local distinguishability aggrandizing network for human anomaly detection10.1016/j.neunet.2019.11.002 ↗
57. Pinaya, W.H.L., Vieira, S., Garcia-Dias, R., and Mechelli, A. (2020). Autoencoders. Machine Learning, Academic Press.
58. A review of deep learning techniques for crowd behavior analysis10.1007/s11831-022-09772-1 ↗
59. Review of abnormal behavior detection of crowd based on deep learning
60. Scene perception guided crowd anomaly detection10.1016/j.neucom.2020.07.019 ↗
61. Extracting macroscopic quantities in crowd behaviour with deep learning10.1088/1402-4896/ad423e ↗
62. Nguyen, T.N., and Meunier, J. (2019, January 27–28). Anomaly detection in video sequence with appearance-motion correspondence. Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea.10.1109/ICCV.2019.00136 ↗
63. Wei, H., Li, K., Li, H., Lyu, Y., and Hu, X. (2019). Detecting video anomaly with a stacked convolutional LSTM framework. International Conference on Computer Vision Systems, Springer International Publishing.10.1007/978-3-030-34995-0_30 ↗
64. Probabilistic memory autoencoder network for abnormal behavior detection in surveillance videos
65. Spatiotemporal anomaly detection using deep learning for real-time video surveillance10.1109/TII.2019.2938527 ↗
66. Deep stacked denoising autoencoder for unsupervised anomaly detection in video surveillance10.1117/1.JEI.32.3.033015 ↗
67. Variational abnormal behavior detection with motion consistency10.1109/TIP.2021.3130545 ↗
68. DEVDAN: Deep evolving denoising autoencoder10.1016/j.neucom.2019.07.106 ↗
69. Abnormal behavior detection in videos using deep learning10.1007/s10586-018-2114-2 ↗
70. Generative neural networks for anomaly detection in crowded scenes10.1109/TIFS.2018.2878538 ↗
71. Review of the Research Status of Image Anomaly Detection
72. Learning normal patterns via adversarial attention-based autoencoder for abnormal event detection in videos10.1109/TMM.2019.2950530 ↗
73. NM-GAN: Noise-modulated generative adversarial network for video anomaly detection10.1016/j.patcog.2021.107969 ↗
74. Integrating prediction and reconstruction for anomaly detection10.1016/j.patrec.2019.11.024 ↗
75. BMAN: Bidirectional multi-scale aggregation networks for abnormal event detection10.1109/TIP.2019.2948286 ↗
76. f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks10.1016/j.media.2019.01.010 ↗
77. Detection Method for Crowd Abnormal Behavior Based on Long Short-Term Memory Network
78. Video Anomaly Event Detection Fusing FCN and LSTM
79. Crowd anomaly detection with LSTMs using optical features and domain knowledge for improved inferring10.1007/s00371-021-02100-x ↗
80. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. Adv. Neural Inf. Process. Syst.
81. Ye, Z., Li, Y., Cui, Z., Liu, Y., Li, L., Wang, L., and Zhang, C. (2023, January 24–28). Unsupervised Video Anomaly Detection with Self-Attention Based Feature Aggregating. Proceedings of the 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC), Bilbao, Spain.10.1109/ITSC57777.2023.10421863 ↗
82. Video Human Abnormal Behavior Detection Model of Autoencoder Fusing Attention Mechanism
83. Zhang, J., Qi, X., and Ji, G. (2022, January 26–27). Self Attention Based Bi-Directional Long Short-Term Memory Auto Encoder for Video Anomaly Detection. Proceedings of the 2021 Ninth International Conference on Advanced Cloud and Big Data (CBD), Xi’an, China.10.1109/CBD54617.2021.00027 ↗
84. Generative Adversarial Networks for Abnormal Event Detection in Videos Based on Self-Attention Mechanism10.1109/ACCESS.2021.3110798 ↗
85. Attention-Guided Generator with Dual Discriminator GAN for Real-Time Video Anomaly Detection10.1016/j.engappai.2023.107830 ↗
86. Scale-Aware CNN for Crowd Density Estimation and Crowd Behavior Analysis10.1016/j.compeleceng.2022.108569 ↗
87. Abnormal Behavior Detection Based on Dynamic Pedestrian Centroid Model: Case Study on U-Turn and Fall-Down10.1109/TITS.2023.3263586 ↗
88. (2024, September 20). UCSD Anomaly Detection Dataset. Available online: http://www.svcl.ucsd.edu/projects/anomaly/dataset.htm.
89. (2024, September 23). UMN Crowd Dataset. Available online: https://mha.cs.umn.edu/proj_events.shtml#crowd.
90. Context Recovery and Knowledge Retrieval: A Novel Two-Stream Framework for Video Anomaly Detection10.1109/TIP.2024.3372466 ↗
91. Anomaly Detection Method Based on Temporal Spatial Information Enhancement10.1088/1361-6501/ad0e3f ↗