EAR-CCPM-Net: A Cross-Modal Collaborative Perception Network for Early Accident Risk Prediction
Source abstract
Abstract
Early traffic accident risk prediction in complex road environments poses significant challenges due to the heterogeneous nature and incomplete semantic alignment of multimodal data. To address this, we propose a novel Early Accident Risk Cross-modal Collaborative Perception Mechanism Network (EAR-CCPM-Net) that integrates hierarchical fusion modules and cross-modal attention mechanisms to enable semantic interaction between visual, motion, and textual modalities. The model is trained and evaluated on the newly constructed CAP-DATA dataset, incorporating advanced preprocessing techniques such as bilateral filtering and a rigorous MINI-Train-Test sampling protocol. Experimental results show that EAR-CCPM-Net achieves an AUC of 0.853, AP of 0.758, and improves the Time-to-Accident (TTA0.5) from 3.927 s to 4.225 s, significantly outperforming baseline methods. These findings demonstrate that EAR-CCPM-Net effectively enhances early-stage semantic perception and prediction accuracy, providing an interpretable solution for real-world traffic risk anticipation.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Occupational Health and Safety Research · Risk and Safety Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Serdang 43400, Selangor, Malaysia
References (54)
Show source-provided references
1. World Health Organization (2018, May 06). Save Lives: A Road Safety Technical Package. Available online: https://apps.who.int/iris/bitstream/handle/10665/255199/9789241511704-eng.pdf.
2. World Health Organization (2019). Global Status Report on Road Safety 2018, World Health Organization.
3. Heydari, S., Hickford, A., McIlroy, R., Turner, J., and Bachani, A.M. (2019). Road Safety in Low-Income Countries: State of Knowledge and Future Directions. Sustainability, 11.10.3390/su11226249 ↗
4. A Time-Embedded Attention-based Transformer for Crash Likelihood Prediction at Intersections Using Connected Vehicle Data10.1016/j.trc.2024.104831 ↗
5. Advanced Analytics for Predicting Traffic Collision Severity Assessment
6. Traffic Accident Duration Prediction Using Multi-mode Data and Ensemble Deep Learning10.1016/j.heliyon.2024.e25957 ↗
7. Elsayed, H.A.G., and Syed, L. (2017, January 22–23). An Automatic Early Risk Classification of Hard Coronary Heart Diseases Using Framingham Scoring Model. Proceedings of the Second International Conference on Internet of Things, Data and Cloud Computing, Cambridge, UK.10.1145/3018896.3036384 ↗
8. A Data-driven Approach for Collision Risk Early Warning in Vessel Encounter Situations Using Attention-BiLSTM10.1109/ACCESS.2020.3031722 ↗
9. Fang, J., Li, L.L., Yang, K., Zheng, Z., Xue, J., and Chua, T.S. (2022). Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmark. arXiv.
10. Fang, S., Liu, J., Ding, M., Cui, Y., Lv, C., Hang, P., and Sun, J. (2024). Towards Interactive and Learnable Cooperative Driving Automation: A Large Language Model-Driven Decision-Making Framework. arXiv.10.1109/TVT.2025.3552922 ↗
11. Jamshidi, H., Jazani, R.K., Khani Jeihooni, A., Alibabaei, A., Alamdari, S., and Kalyani, M.N. (2023). Facilitators and Barriers to Collaboration Between Pre-hospital Emergency and Emergency Department in Traffic Accidents: A qualitative study. BMC Emerg. Med., 23.10.1186/s12873-023-00828-4 ↗
12. A Review on Action Recognition for Accident Detection in Smart City Transportation Systems10.1186/s43067-023-00124-y ↗
13. Vehicle trajectory data processing, analytics, and applications: A survey
14. Wang, L., Xiao, M., Lv, J., and Liu, J. (2024). Analysis of Influencing Factors of Traffic Accidents on Urban Ring Road based on the SVM Model Optimized by Bayesian Method. PLoS ONE, 19.10.1371/journal.pone.0310044 ↗
15. Khan, S.W., Hafeez, Q., Khalid, M.I., Alroobaea, R., Hussain, S., Iqbal, J., Almotiri, J., and Ullah, S.S. (2022). Anomaly Detection in Traffic Surveillance Videos Using Deep Learning. Sensors, 22.10.3390/s22176563 ↗
16. Hasan, F., and Huang, H. (2023). MALS-Net: A Multi-head Attention-based LSTM Sequence-to-sequence Network for Socio-temporal Interaction Modelling and Trajectory Prediction. Sensors, 23.10.3390/s23010530 ↗
17. Leon, F., and Gavrilescu, M. (2021). A Review of Tracking and Trajectory Prediction Methods for Autonomous Driving. Mathematics, 9.10.3390/math9060660 ↗
18. Yang, D., Huang, S., Xu, Z., Li, Z., Wang, S., Li, M., Wang, Y., Liu, Y., Yang, K., and Chen, Z. (2023, January 2–3). Aide: A Vision-driven Multi-view, Multimodal, Multi-tasking Dataset for Assistive Driving Perception. Proceedings of the IEEE/CVF International Conference on Computer Vision, Paris, France.10.1109/ICCV51070.2023.01871 ↗
19. A Comparative Review on Multimodal Sensors Fusion Based on Deep Learning10.1016/j.sigpro.2023.109165 ↗
20. The fusion of deep learning and fuzzy systems: A state-of-the-art survey10.1109/TFUZZ.2021.3062899 ↗
21. Marchetti, F., Mordan, T., Becattini, F., Seidenari, L., Bimbo, A.D., and Alahi, A. (IEEE Trans. Intell. Veh., 2024). CrossFeat: Semantic Cross-modal Attention for Pedestrian Behavior Forecasting, IEEE Trans. Intell. Veh., early access.10.1109/TIV.2024.3449046 ↗
22. Deep Spatio-temporal 3D Dilated Dense Neural Network for Traffic Flow Prediction10.1016/j.eswa.2023.121394 ↗
23. Liu, Z., Yang, N., Wang, Y., Li, Y., Zhao, X., and Wang, F.Y. (2023). Enhancing Traffic Object Detection in Variable Illumination with RGB-Event Fusion. arXiv.10.1109/TITS.2024.3456108 ↗
24. Transfuser: Imitation with Transformer-based Sensor Fusion for Autonomous Driving10.1109/TPAMI.2022.3200245 ↗
25. Tan, H., and Bansal, M. (2019). Lxmert: Learning Cross-modality Encoder Representations from Transformers. arXiv.10.18653/v1/D19-1514 ↗
26. Lu, J., Batra, D., Parikh, D., and Lee, S. (2019). ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks. arXiv.
27. Chen, Y.C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., Cheng, Y., and Liu, J. (2020). Uniter: Universal Image-text Representation Learning. European Conference on Computer Vision, Springer International Publishing.10.1007/978-3-030-58577-8_7 ↗
28. Zadeh, A.B., Liang, P.P., Poria, S., Cambria, E., and Morency, L.P. (2018, January 15–20). Multimodal language analysis in the wild: Cmu-mosei Dataset and Interpretable Dynamic Fusion Graph. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia. Volume 1: Long Papers.
29. Tsai, Y.H.H., Bai, S., Liang, P.P., Kolter, J.Z., Morency, L.P., and Salakhutdinov, R. (August, January 28). Multimodal Transformer for Unaligned Multimodal Language Sequences. Proceedings of the Conference. Association for Computational Linguistics. Meeting, Florence, Italy.
30. Rahman, W., Hasan, M.K., Lee, S., Zadeh, A., Mao, C., Morency, L.P., and Hoque, E. (2020, January 5–10). Integrating Multimodal Information in Large Pretrained Transformers. Proceedings of the Conference. Association for Computational Linguistics. Meeting, Online.10.18653/v1/2020.acl-main.214 ↗
31. Li, W., Liu, H., Tang, H., Wang, P., and Van Gool, L. (2022, January 18–24). Mhformer: Multi-hypothesis Transformer for 3d Human Pose Estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.10.1109/CVPR52688.2022.01280 ↗
32. Multi-model Traffic Prediction in Smart Cities Using Graph Neural Networks and Transformer-Based Multi-Source Visual Fusion for Intelligent Transportation Management
33. Li, L., Dou, Y., and Zhou, J. (2023, January 1–3). Traffic Accident Detection based on Multimodal Knowledge Graphs. Proceedings of the 2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence (RICAI), Hangzhou, China.10.1109/RICAI60863.2023.10489808 ↗
34. A Multimodal Coupled Graph Attention Network for Joint Traffic Event Detection and Sentiment Classification10.1109/TITS.2022.3205477 ↗
35. Multimodal Learning with Graphs10.1038/s42256-023-00624-6 ↗
36. STGAT: Spatial-temporal Graph Attention Networks for Traffic Flow Prediction10.1109/ACCESS.2020.3011186 ↗
37. Zhang, Y., Dong, X., Shang, L., Zhang, D., and Wang, D. (2020, January 22–25). A Multimodal Graph Neural Network Approach to Traffic Risk Prediction in Smart Urban Sensing. Proceedings of the 2020 17th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON), Como, Italy.10.1109/SECON48991.2020.9158447 ↗
38. Ulutan, O., Iftekhar, A.S.M., and Manjunath, B.S. (2020, January 13–19). Vsgnet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutions. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.10.1109/CVPR42600.2020.01363 ↗
39. Toward Region-aware Attention Learning for Scene Graph Generation10.1109/TNNLS.2021.3086066 ↗
40. Sultana, S., and Ahmed, B. (2021, January 5–7). Robust Nighttime Road Lane Line Detection Using Bilateral Filter and SAGC Under Challenging Conditions. Proceedings of the 2021 IEEE 13th International Conference on Computer Research and Development (ICCRD), Beijing, China.10.1109/ICCRD51685.2021.9386516 ↗
41. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention Is All You Need. Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA.
42. Language Conditioned Multi-Scale Visual Attention Networks for Visual Grounding10.1016/j.imavis.2024.105242 ↗
43. On Information and Sufficiency10.1214/aoms/1177729694 ↗
44. VII
45. Judd, T., Ehinger, K., Durand, F., and Torralba, A. (October, January 27). Learning to Predict Where Humans Look. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision, Kyoto, Japan.10.1109/ICCV.2009.5459462 ↗
46. Quantitative Analysis of Human-model Agreement in Visual Saliency Modeling: A Comparative Study10.1109/TIP.2012.2210727 ↗
47. Chan, F.H., Chen, Y.T., Xiang, Y., and Sun, M. (2016, January 20–24). Anticipating Accidents in Dashcam Videos. Proceedings of the Computer Vision–ACCV 2016: 13th Asian Conference on Computer Vision, Taipei, Taiwan. Revised Selected Papers, Part IV 13.
48. Suzuki, T., Kataoka, H., Aoki, Y., and Satoh, Y. (2018, January 18–22). Anticipating Traffic Accidents with Adaptive Loss and Large-scale Incident db. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00371 ↗
49. Bao, W., Yu, Q., and Kong, Y. (2020, January 12–16). Uncertainty-based Traffic Accident Anticipation with Spatio-temporal Relational Learning. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA.10.1145/3394171.3413827 ↗
50. Bao, W., Yu, Q., and Kong, Y. (2021, January 10–17). Drive: Deep Reinforced Accident Anticipation with Visual Explanation. Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada.10.1109/ICCV48922.2021.00752 ↗
51. Xia, Y., Zhang, D., Kim, J., Nakayama, K., Zipser, K., and Whitney, D. (2018, January 2–6). Predicting Driver Attention in Critical Situations. Proceedings of the Computer vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia. Revised Selected Papers, Part v 14.
52. Predicting the Driver’s Focus of Attention: The DR (eye) VE Project10.1109/TPAMI.2018.2845370 ↗
53. Revisiting Video Saliency Prediction in the Deep Learning Era10.1109/TPAMI.2019.2924417 ↗
54. DADA: Driver Attention Prediction in Driving Accident Scenarios10.1109/TITS.2020.3044678 ↗