Innovative Approaches to Traffic Anomaly Detection and Classification Using AI
Source abstract
Abstract
Video anomaly detection plays a crucial role in intelligent transportation systems by enhancing urban mobility and safety. This review provides a comprehensive analysis of recent advancements in artificial intelligence methods applied to traffic anomaly detection, including convolutional and recurrent neural networks (CNNs and RNNs), autoencoders, Transformers, generative adversarial networks (GANs), and multimodal large language models (MLLMs). We compare their performance across real-world applications, highlighting patterns such as the superiority of Transformer-based models in temporal context understanding and the growing use of multimodal inputs for robust detection. Key challenges identified include dependence on large labeled datasets, high computational costs, and limited model interpretability. The review outlines how recent research is addressing these issues through semi-supervised learning, model compression techniques, and explainable AI. We conclude with future directions focusing on scalable, real-time, and interpretable solutions for practical deployment.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Network Security and Intrusion Detection · Time Series Analysis and Forecasting
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Autonomous Mobility and Perception Lab (AMPL), Departamento de Ingeniería de Sistemas y Automática, Universidad Carlos III de Madrid, Av. de la Universidad, 30, 28911 Madrid, Spain
Technological Institute of Aragón, Calle Maria de Luna, 7-8, 50018 Zaragoza, Spain
Funding
Spanish Government · European Union NextGenerationEU/PRTR
References (72)
Show source-provided references
1. Xiang, S., Zhu, M., Cheng, D., Li, E., Zhao, R., Ouyang, Y., Chen, L., and Zheng, Y. (2023, January 7–14). Semi-supervised credit card fraud detection via attribute-driven graph representation. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.10.1609/aaai.v37i12.26702 ↗
2. Ram, P., and Gray, A.G. (2017, January 14). Fraud detection with density estimation trees. Proceedings of the KDD 2017 Workshop on Anomaly Detection in Finance, PMLR, Halifax, NS, Canada.
3. Hung, Y.H. (2022). Developing an Anomaly Detection System for Automatic Defective Products’ Inspection. Processes, 10.10.3390/pr10081476 ↗
4. Anomaly detection during milk processing by autoencoder neural network based on near-infrared spectroscopy10.1016/j.jfoodeng.2021.110510 ↗
5. Efficient anomaly detection from medical signals and images10.1007/s10772-019-09610-z ↗
6. Tschuchnig, M.E., and Gadermayr, M. (2021, January 20–21). Anomaly detection in medical imaging-a mini review. Proceedings of the Data Science—Analytics and Applications: Proceedings of the 4th International Data Science Conference–iDSC2021, Virtual.
7. Dias, M.A., Silva, E.A.d., Azevedo, S.C.d., Casaca, W., Statella, T., and Negri, R.G. (2019). An incongruence-based anomaly detection strategy for analyzing water pollution in images from remote sensing. Remote Sens., 12.10.3390/rs12010043 ↗
8. LSTM-autoencoder-based anomaly detection for indoor air quality time-series data10.1109/JSEN.2022.3230361 ↗
9. Bawaneh, M., and Simon, V. (2019, January 19–21). Anomaly detection in smart city traffic based on time series analysis. Proceedings of the 2019 International Conference on Software, Telecommunications and Computer Networks (SoftCOM), Split, Croatia.10.23919/SOFTCOM.2019.8903822 ↗
10. SafeDrive: Online driving anomaly detection from large-scale vehicle data10.1109/TII.2017.2674661 ↗
11. Cobilean, V., Mavikumbure, H.S., Wickramasinghe, C.S., Varghese, B.J., Pennington, T., and Manic, M. (2023, January 16–19). Anomaly Detection for In-Vehicle Communication Using Transformers. Proceedings of the IECON 2023—49th Annual Conference of the IEEE Industrial Electronics Society, Singapore.10.1109/IECON51785.2023.10311788 ↗
12. Redmon, J. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
13. Trajectory-based anomalous event detection10.1109/TCSVT.2008.2005599 ↗
14. Morris, B., and Trivedi, M. (2009, January 20–25). Learning trajectory patterns by clustering: Experimental studies and comparative evaluation. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPRW.2009.5206559 ↗
15. Morris, B.T. (2010). Understanding Activity from Trajectory Patterns, University of California.
16. Pathak, D., Sharang, A., and Mukerjee, A. (2015, January 5–9). Anomaly localization in topic-based analysis of surveillance videos. Proceedings of the 2015 IEEE Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV.2015.58 ↗
17. Sultani, W., Chen, C., and Shah, M. (2018, January 18–23). Real-world anomaly detection in surveillance videos. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00678 ↗
18. Doshi, K., and Yilmaz, Y. (2020, January 14–19). Fast unsupervised anomaly detection in traffic videos. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.1109/CVPRW50498.2020.00320 ↗
19. Wang, S., Anastasiu, D.C., Tang, Z., Chang, M.C., Yao, Y., Zheng, L., Rahman, M.S., Arya, M.S., Sharma, A., and Chakraborty, P. (2024, January 17–18). The 8th AI City Challenge. Proceedings of the The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Seattle, WA, USA.
20. Minnikhanov, R., Dagaeva, M., Anikin, I., Bolshakov, T., Makhmutova, A., and Mingulov, K. (2020, January 17). Detection of traffic anomalies for a safety system of smart city. Proceedings of the CEUR Workshop Proceedings, Bergen, Norway.
21. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
22. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11–14). Ssd: Single shot multibox detector. Proceedings of the Computer Vision—ECCV 2016: 14th European Conference, Amsterdam, The Netherlands. Proceedings, Part I 14.10.1007/978-3-319-46448-0_2 ↗
23. Grabner, H., Grabner, M., and Bischof, H. (2006, January 4–7). Real-time tracking via on-line boosting. Proceedings of the British Machine Vision Conference, Edinburgh, UK.10.5244/C.20.6 ↗
24. Babenko, B., Yang, M.H., and Belongie, S. (2009, January 20–25). Visual tracking with online multiple instance learning. Proceedings of the 2009 IEEE Conference on computer vision and Pattern Recognition, Miami, FL, USA.10.1109/CVPRW.2009.5206737 ↗
25. High-speed tracking with kernelized correlation filters10.1109/TPAMI.2014.2345390 ↗
26. Bolme, D.S., Beveridge, J.R., Draper, B.A., and Lui, Y.M. (2010, January 13–18). Visual object tracking using adaptive correlation filters. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.10.1109/CVPR.2010.5539960 ↗
27. Aboah, A. (2021, January 20–25). A vision-based system for traffic anomaly detection using deep learning and decision trees. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.10.1109/CVPRW53098.2021.00475 ↗
28. Jocher, G., Chaurasia, A., Stoken, A., Borovec, J., Kwon, Y., Michael, K., Xie, T., Fang, J. (2020, January 01). YOLOv5 by Ultralytics. Available online: https://zenodo.org/records/7347926.
29. Naphade, M., Wang, S., Anastasiu, D.C., Tang, Z., Chang, M.C., Yang, X., Yao, Y., Zheng, L., Chakraborty, P., and Lopez, C.E. (2021, January 19–25). The 5th AI City Challenge. Proceedings of the The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Nashville, TN, USA.10.1109/CVPRW53098.2021.00482 ↗
30. Visual recognition for urban traffic data retrieval and analysis in major events using convolutional neural networks10.1007/s43762-021-00031-w ↗
31. Khan, S.W., Hafeez, Q., Khalid, M.I., Alroobaea, R., Hussain, S., Iqbal, J., Almotiri, J., and Ullah, S.S. (2022). Anomaly detection in traffic surveillance videos using deep learning. Sensors, 22.10.3390/s22176563 ↗
32. A lightweight video anomaly detection model with weak supervision and adaptive instance selection10.1016/j.neucom.2024.128698 ↗
33. Zhang, Y., Zhou, D., Chen, S., Gao, S., and Ma, Y. (2016, January 27–30). Single-image crowd counting via multi-column convolutional neural network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.70 ↗
34. Singh, R., Sharma, N., Rajput, K., and Pokhariya, H.S. (2024, January 26–27). EfficientNet-B7 Enhanced Road Accident Detection Using CCTV Footage. Proceedings of the 2024 Asia Pacific Conference on Innovation in Technology (APCIT), Mysore, India.10.1109/APCIT62007.2024.10673607 ↗
35. Learning spatial patterns and temporal dependencies for traffic accident severity prediction: A deep learning approach10.1016/j.knosys.2024.111406 ↗
36. Anomaly detection in surveillance videos using deep autoencoder
37. Modeling, clustering, and segmenting video with mixtures of dynamic textures10.1109/TPAMI.2007.70738 ↗
38. Lu, C., Shi, J., and Jia, J. (2013, January 1–8). Abnormal event detection at 150 fps in matlab. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia.10.1109/ICCV.2013.338 ↗
39. Ramachandra, B., Jones, M., and Vatsavai, R. (2020, January 1–5). Learning a distance function with a Siamese network to localize anomalies in videos. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Snowmass Village, CO, USA.10.1109/WACV45572.2020.9093417 ↗
40. Liu, W., Luo, W., Lian, D., and Gao, S. (2018, January 18–23). Future frame prediction for anomaly detection–a new baseline. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00684 ↗
41. Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5–9). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015: 18th International Conference, Munich, Germany. Proceedings, Part III 18.
42. Lee, S., Kim, H.G., and Ro, Y.M. (2018, January 15–20). STAN: Spatio-temporal adversarial networks for abnormal event detection. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.10.1109/ICASSP.2018.8462388 ↗
43. Dual discriminator generative adversarial network for video anomaly detection10.1109/ACCESS.2020.2993373 ↗
44. Transanomaly: Video anomaly detection using video vision transformer10.1109/ACCESS.2021.3109102 ↗
45. Deshpande, K., Punn, N.S., Sonbhadra, S.K., and Agarwal, S. (2022, January 22–26). Anomaly detection in surveillance videos using transformer based attention model. Proceedings of the International Conference on Neural Information Processing, IIT, Indore, India.10.1007/978-981-99-1648-1_17 ↗
46. A Three-Stage Anomaly Detection Framework for Traffic Videos10.1155/2022/9463559 ↗
47. Cvit: A convolution vision transformer for video abnormal behavior detection and localization10.1007/s42979-023-02294-y ↗
48. Leporowski, B., Bakhtiarnia, A., Bonnici, N., Muscat, A., Zanella, L., Wang, Y., and Iosifidis, A. (2023). Audio-Visual Dataset and Method for Anomaly Detection in Traffic Videos. arXiv.10.1109/ICIP51287.2024.10647874 ↗
49. Lu, X., Zhang, D., and Xiao, J. (2024). A Hybrid Model for Traffic Incident Detection based on Generative Adversarial Networks and Transformer Model. arXiv.
50. I-880 field experiment: Analysis of incident data10.3141/1603-10 ↗
51. Dissipative Structure Properties of Traffic Flow in Expressway Weaving Areas10.7307/ptt.v36i4.525 ↗
52. Rossi, L., Bernuzzi, V., Fontanini, T., Bertozzi, M., and Prati, A. (2024, January 14–19). Memory-Augmented Online Video Anomaly Detection. Proceedings of the ICASSP 2024–2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea.10.1109/ICASSP48485.2024.10447554 ↗
53. DoTA: Unsupervised detection of traffic anomaly in driving videos10.1109/TPAMI.2022.3150763 ↗
54. Kim, J., Yoon, S., Choi, T., and Sull, S. (2023). Unsupervised video anomaly detection based on similarity with predefined text descriptions. Sensors, 23.10.3390/s23146256 ↗
55. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18–24). Learning transferable visual models from natural language supervision. Proceedings of the International Conference on Machine Learning, PMLR, Online.
56. Shao, Y., Cai, H., Long, X., Lang, W., Wang, Z., Wu, H., Wang, Y., Yin, J., Yang, Y., and Lv, Y. (2024). AccidentBlip2: Accident Detection with Multi-View MotionBlip2. arXiv.
57. DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Driving
58. Arefeen, M.A., Debnath, B., Uddin, M.Y.S., and Chakradhar, S. (2024, January 16–22). ViTA: An Efficient Video-to-Text Algorithm using VLM for RAG-based Video Analysis System. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPRW63382.2024.00232 ↗
59. Piadyk, Y., Rulff, J., Brewer, E., Hosseini, M., Ozbay, K., Sankaradas, M., Chakradhar, S., and Silva, C. (2023). StreetAware: A High-Resolution Synchronized Multimodal Urban Scene Dataset. Sensors, 23.10.3390/s23073710 ↗
60. Extract-Transform-Load for Video Streams10.14778/3598581.3598600 ↗
61. Using Multimodal Large Language Models (MLLMs) for Automated Detection of Traffic Safety-Critical Events10.3390/vehicles6030074 ↗
62. Malla, S., Choi, C., Dwivedi, I., Choi, J.H., and Li, J. (2023, January 2–7). Drama: Joint risk localization and captioning in driving. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.10.1109/WACV56688.2023.00110 ↗
63. Duan, Z., Cheng, H., Xu, D., Wu, X., Zhang, X., Ye, X., and Xie, Z. (2024, January 16–22). Cityllava: Efficient fine-tuning for vlms in city scenario. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPRW63382.2024.00713 ↗
64. Liu, H., Li, C., Li, Y., Li, B., Zhang, Y., Shen, S., and Lee, Y.J. (2024, January 01). Llava-Next: Improved Reasoning, OCR, and World Knowledge. Available online: https://llava-vl.github.io/blog/2024-01-30-llava-next/.
65. Wu, C., Gan, Y., Ge, Y., Lu, Z., Wang, J., Feng, Y., Luo, P., and Shan, Y. (2024). Llama pro: Progressive llama with block expansion. arXiv.10.18653/v1/2024.acl-long.352 ↗
66. Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2021). Lora: Low-rank adaptation of large language models. arXiv.
67. Wang, H., Qin, J., Bastola, A., Chen, X., Suchanek, J., Gong, Z., and Razi, A. (2024). VisionGPT: LLM-Assisted Real-Time Anomaly Detection for Safe Visual Navigation. arXiv.
68. Ashqar, H.I., Jaber, A., Alhadidi, T.I., and Elhenawy, M. (2024). Advancing Object Detection in Transportation with Multimodal Large Language Models (MLLMs): A Comprehensive Review and Empirical Testing. arXiv.
69. Anomaly detection in road traffic using visual surveillance: A survey10.1145/3417989 ↗
70. Bogdoll, D., Nitsche, M., and Zöllner, J.M. (2022, January 19–24). Anomaly detection in autonomous driving: A survey. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPRW56347.2022.00495 ↗
71. Chew, J.V.L., and Asli, M.F. (2024, January 20–24). A survey on vehicular traffic flow anomaly detection using machine learning. Proceedings of the ITM Web of Conferences, Marrakech, Morocco.10.1051/itmconf/20246301023 ↗
72. El Manaa, I., Benjelloun, F., Sabri, M.A., Yahyaouy, A., and Aarab, A. (2022, January 28–30). Road traffic anomaly detection: A survey. Proceedings of the International Conference on Digital Technologies and Applications, Fez, Morocco.10.1007/978-3-031-01942-5_77 ↗