A Novel Hybrid Deep Learning–Probabilistic Framework for Real-Time Crash Detection from Monocular Traffic Video
Source abstract
Abstract
The rapid evolution of autonomous vehicle technologies has amplified the need for crash detection that operates robustly under complex traffic conditions with minimal latency. We propose a hybrid temporal hierarchy that augments a Region-based Convolutional Neural Network (R-CNN) with an adaptive time-variant Kalman filter (with total-variation prior), a Hidden Markov Model (HMM) for state stabilization, and a lightweight Artificial Neural Network (ANN) for learned temporal refinement, enabling real-time crash detection from monocular video. Evaluated on simulated traffic in CARLA and real-world driving in Istanbul, the full temporal stack achieves the best precision–recall balance, yielding 83.47% F1 offline and 82.57% in real time (corresponding to 94.5% and 91.2% detection accuracy, respectively). Ablations are consistent and interpretable: removing the HMM reduces F1 by 1.85–2.16 percentage points (pp), whereas removing the ANN has a larger impact of 2.94–4.58 pp, indicating that the ANN provides the largest marginal gains—especially under real-time constraints. The transition from offline to real time incurs a modest overall loss (−0.90 pp F1), driven more by recall than precision. Compared to strong single-frame baselines, YOLOv10 attains 82.16% F1 and a real-time Transformer detector reaches 82.41% F1, while our full temporal stack remains slightly ahead in real time and offers a more favorable precision–recall trade-off. Notably, integrating the ANN into the HMM-based pipeline improves accuracy by 2.2%, while the time-variant Kalman configuration reduces detection lag by approximately 0.5 s—an improvement that directly addresses the human reaction time gap. Under identical conditions, the best RCNN-based configuration yields AP@0.50 ≈ 0.79 with an end-to-end latency of 119 ± 21 ms per frame (~8–9 FPS). Overall, coupling deep learning with probabilistic reasoning yields additive temporal benefits and advances deployable, camera-only crash detection that is cost-efficient and scalable for intelligent transportation systems.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Autonomous Vehicle Technology and Safety · Anomaly Detection Techniques and Applications · Video Surveillance and Tracking Methods
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Software Engineering, Faculty of Engineering, Istanbul Topkapı University, Altunizade İstanbul 34662, Türkiye
Department of Computer Engineering, Faculty of Engineering and Architecture, Istanbul Beykent University, Ayazağa İstanbul 34398, Türkiye
References (50)
Show source-provided references
1. Anderson, J.M., Nidhi, K., Stanley, K.D., Sorensen, P., Samaras, C., and Oluwatola, O.A. (2014). Autonomous Vehicle Technology: A Guide for Policymakers, Rand Corporation.
2. Feypell, V., Methorst, R., and Hughes, T. (2025, February 25). Non-Motor Pedestrian Accidents: A Hidden Issue. Available online: https://www.ictct.net/wp-content/uploads/23-Hague-2010/ictct_document_nr_740.pdf.
3. Ranney, T.A., Garrott, W.R., and Goodman, M.J. (2001). NHTSA Driver Distraction Research: Past, Present, and Future, SAE Technical Paper.
4. Legal aspects of autonomous driving
5. World Health Organization (2023). Global Status Report on Road Safety 2023: Time for a New Normal, World Health Organization. Available online: https://www.who.int/teams/social-determinants-of-health/safety-and-mobility/global-status-report-on-road-safety-2023.
6. Faster RCNN based robust vehicle detection algorithm for identifying and classifying vehicles10.1007/s11554-023-01344-1 ↗
7. A review of vehicle detection techniques for intelligent vehicles10.1109/TNNLS.2021.3128968 ↗
8. Pedestrian and vehicle behaviour prediction in autonomous vehicle system—A review10.1016/j.eswa.2023.121983 ↗
9. Dalal, N., and Triggs, B. (2005, January 20–26). Histograms of oriented gradients for human detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA.
10. An improved pedestrian detection algorithm integrating haar-like features and hog descriptors10.1155/2013/546206 ↗
11. Object Detection in Traffic Videos: A Survey10.1109/TITS.2023.3258683 ↗
12. On the detection of vehicular crashes-system characteristics and architecture10.1109/25.992078 ↗
13. A trainable system for object detection10.1023/A:1008162616689 ↗
14. Galvao, L.G., Abbod, M., Kalganova, T., Palade, V., and Huda, M.N. (2021). Pedestrian and Vehicle Detection in Autonomous Vehicle Perception Systems—A Review. Sensors, 21.10.3390/s21217267 ↗
15. A Review of Object Detection in Traffic Scenes Based on Deep Learning
16. Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23–28). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.81 ↗
17. Hasan, Y., Arif, M.U., Asif, A., and Raza, R.H. (2016, January 6–7). Comparative analysis of vehicle detection in urban traffic environment using Haar cascaded classifiers and blob statistics. Proceedings of the 2016 Future Technologies Conference (FTC), San Francisco, CA, USA.10.1109/FTC.2016.7821660 ↗
18. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
19. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11–14). Ssd: Single shot multibox detector. Proceedings of the Computer Vision—ECCV 2016: 14th European Conference, Amsterdam, The Netherlands.10.1007/978-3-319-46448-0_2 ↗
20. Jiang, Z., Zhao, L., Li, S., and Jia, Y. (2020). Real-time object detection method based on improved YOLOv4-tiny. arXiv.
21. Tan, M., Pang, R., and Le, Q.V. (2020, January 13–19). Efficientdet: Scalable and efficient object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01079 ↗
22. Bochkovskiy, A., Wang, C.-Y., and Liao, H.-Y.M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv.
23. Xiang, Z. (2024, January 21–23). Research on target detection algorithm in autonomous driving scenarios based on improved YOLOv5. Proceedings of the International Conference on Computer Vision, Robotics, and Automation Engineering (CRAE 2024), Kunming, China.10.1117/12.3041775 ↗
24. Jocher, G., Alex, S., Ayush, C., Jirka, B., Yonghye, K., Kalen, M., Liu, C., Jiacong, F., Abhiram, V., and Piotr, S. (2021). Ultralytics/yolov5: v6. 0-YOLOv5n’Nano’models, Roboflow integration, TensorFlow export, OpenCV DNN support. Zenodo.
25. Chughtai, B.R., and Jalal, A. (2024, January 19–20). Traffic surveillance system: Robust multiclass vehicle detection and classification. Proceedings of the 2024 5th International Conference on Advancements in Computational Sciences (ICACS), Lahore, Pakistan.10.1109/ICACS60934.2024.10473304 ↗
26. Khan, S.W., Hafeez, Q., Khalid, M.I., Alroobaea, R., Hussain, S., Iqbal, J., Almotiri, J., and Ullah, S.S. (2022). Anomaly detection in traffic surveillance videos using deep learning. Sensors, 22.10.3390/s22176563 ↗
27. On-road vehicle detection: A review10.1109/TPAMI.2006.104 ↗
28. Kukreja, V., Kumar, D., Kaur, A. (2020, January 5–7). GAN-based synthetic data augmentation for increased CNN performance in Vehicle Number Plate Recognition. Proceedings of the 2020 4th International Conference on Electronics, Communication and Aerospace Technology (ICECA), Coimbatore, India.10.1109/ICECA49313.2020.9297625 ↗
29. Patel, J. (2022). Scenario Generation for Vehicles Using Deep Learning. [Master’s Thesis, KTH Royal Institute of Technology].
30. Deep Learning for Image Super-Resolution: A Survey10.1109/TPAMI.2020.2982166 ↗
31. Anomaly Detection in Road Traffic Using Visual Surveillance: A Survey10.1145/3417989 ↗
32. Multi-sensor data fusion for vehicle detection in autonomous vehicle applications10.2352/ISSN.2470-1173.2018.17.AVM-257 ↗
33. A monolithic forward-view MEMS laser scanner with decoupled raster scanning and enlarged scanning angle for micro LiDAR applications10.1109/JMEMS.2020.3001921 ↗
34. Moayed, Z. (2020). Automated Multiview Safety Analysis at Complex Road Intersections. [Ph.D. Thesis, Auckland University of Technology].
35. Tahir, N.U.A., Zhang, Z., Asim, M., Chen, J., and ELAffendi, M. (2024). Object detection in autonomous vehicles under adverse weather: A review of traditional and deep learning approaches. Algorithms, 17.10.3390/a17030103 ↗
36. Liang, S., and Zhang, X. (2024, January 22–24). An Object Detection System for Automatic Driving: MEC-YOLO Based on “Cloud-Edge-End”. Proceedings of the 2024 6th International Conference on Natural Language Processing (ICNLP), Xi’an, China.10.1109/ICNLP60986.2024.10692461 ↗
37. Real-time traffic control and monitoring10.1016/j.prime.2023.100211 ↗
38. Transfer Learning-Based Anomaly Detection System for Autonomous Vehicle
39. Li, Y., Zhao, W., and Fan, H. (2022). A spatio-temporal graph neural network approach for traffic flow prediction. Mathematics, 10.10.3390/math10101754 ↗
40. Huo, Y., Zhang, H., Tian, Y., Wang, Z., Wu, J., and Yao, X. (2024). A spatiotemporal graph neural network with graph adaptive and attention mechanisms for traffic flow prediction. Electronics, 13.10.3390/electronics13010212 ↗
41. Erkartal, B., and Yılmaz, A. (2025, January 11–13). Generating Crush Signals in Vehicle Traffic using RCNN, Hidden Markov Chain and ANN. Proceedings of the International Conference on Science, Engineering Management and IT (SEMIT 2025), Dubai, United Arab Emirates. Presented at the conference; accepted for publication (in press).
42. Hmidani, O., and Alaoui, E.I. (2022, January 12–14). A comprehensive survey of the R-CNN family for object detection. Proceedings of the 2022 5th International Conference on Advanced Communication Technologies and Networking (CommNet), Marrakech, Morocco.10.1109/CommNet56067.2022.9993862 ↗
43. Norris, J.R. (1998). Markov Models, No. 2, Cambridge University Press.
44. Jurafsky, D., and Martin, J.H. (2019). Vector Semantics and Embeddings. Speech and Language Processing, [3rd ed.]. Available online: https://web.stanford.edu/~jurafsky/slp3/.
45. Philosophy and the practice of Bayesian statistics10.1111/j.2044-8317.2011.02037.x ↗
46. Deschaud, J.E. (2021). KITTI-CARLA: A KITTI-like dataset generated by CARLA Simulator. arXiv.
47. Weber, M., and Perona, P. (2022). Caltech Cars 1999, CaltechDATA.
48. Philip, B., Updike, P., and Perona, P. (2022). Caltech Cars 2001, CaltechDATA.
49. Niranjan, D.R., and Vinay Karthik, B.C. (2021, January 7–9). Deep learning based object detection model for autonomous driving research using carla simulator. Proceedings of the 2021 2nd International Conference on Smart Electronics and Communication (ICOSEC), Trichy, India.10.1109/ICOSEC51865.2021.9591747 ↗
50. World Report on Road Traffic Injury Prevention by World Health Organization and World Bank (WHO, Geneva, April)10.1080/0144164042000231404 ↗