Multimodal Feature-Assisted Continuous Driver Behavior Analysis and Solving for Edge-Enabled Internet of Connected Vehicles Using Deep Learning
Source abstract
Abstract
The emerging technology of internet of connected vehicles (IoCV) introduced many new solutions for accident prevention and traffic safety by monitoring the behavior of drivers. In addition, monitoring drivers’ behavior to reduce accidents has attracted considerable attention from industry and academic researchers in recent years. However, there are still many issues that have not been addressed due to the lack of feature extraction. To this end, in this paper, we propose the multimodal driver analysis internet of connected vehicles (MODAL-IoCV) approach for analyzing drivers’ behavior using a deep learning method. This approach includes three consecutive phases. In the first phase, the hidden Markov model (HMM) is proposed to predict vehicle motion and lane changes. In the second phase, SqueezeNet is proposed to perform feature extraction from these classes. Lastly, in the final phase, tri-agent-based soft actor critic (TA-SAC) is proposed for recommendation and route planning, in which each driver is precisely handled by an edge node for personalized assistance. Finally, detailed experimental results prove that our proposed MODAL-IoCV method can achieve high performance in terms of latency, accuracy, false alarm rate, and motion prediction error compared to existing works.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Autonomous Vehicle Technology and Safety · Traffic control and management · Traffic Prediction and Management Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Information Systems and Technology, College of Computer Science and Engineering, University of Jeddah, Jeddah 23218, Saudi Arabia
Department of Computer Science and Artificial Intelligence, College of Computer Science and Engineering, University of Jeddah, Jeddah 21959, Saudi Arabia
Department of Mathematics & Mechanics, Saint Petersburg State University, 199178 St. Petersburg, Russia
Department of Communication Networks and Data Transmission, St. Petersburg State University of Telecommunications, 193232 St. Petersburg, Russia
Department of Applied Probability and Informatics, Peoples’ Friendship University of Russia (RUDN University), 6 Miklukho-Maklaya St., 117198 Moscow, Russia
Computer Science Department, Community College, Shaqra University, Shaqra 11961, Saudi Arabia
Funding
This research was funded by Distinct Research Grant (DRG), grant number UJ-02-045-DR.
References (36)
Show source-provided references
1. Driving Behavior Clustering for Hazardous Material Transportation Based on Genetic Fuzzy C-Means Algorithm10.1109/ACCESS.2020.2964648 ↗
2. Adaptive comfort-oriented vehicle lateral control with online controller adjustments according to driver behavior and look-ahead dynamics
3. Building a Lane Merge Coordination for Connected Vehicles Using Deep Reinforcement Learning10.1109/JIOT.2020.3017931 ↗
4. Le, D.T., Dang, K.Q., Nguyen, Q.L.T., Alhelaly, S., and Muthanna, A. (2021). A Behavior-Based Malware Spreading Model for Vehicle-to-Vehicle Communications in VANET Networks. Electronics, 10.10.3390/electronics10192403 ↗
5. Evaluating the impact of Heavy Goods Vehicle driver monitoring and coaching to reduce risky behavior10.1016/j.aap.2020.105754 ↗
6. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations10.1016/j.jcp.2018.10.045 ↗
7. Driver Behavior Soft-Sensor Based on Neurofuzzy Systems and Weighted Projection on Principal Components10.1109/JSEN.2020.2995921 ↗
8. An Effective Bio-Signal-Based Driver Behavior Monitoring System Using a Generalized Deep Learning Approach10.1109/ACCESS.2020.3011003 ↗
9. Wu, R., Zheng, X., Xu, Y., Wu, W., Li, G., Xu, Q., and Nie, Z. (2019). Modified Driving Safety Field Based on Trajectory Prediction Model for Pedestrian–Vehicle Collision. Sustainability, 11.10.3390/su11226254 ↗
10. Li, Y., Wang, F., Ke, H., Wang, L.-L., and Xu, C.-C. (2019). A Driver’s Physiology Sensor-Based Driving Risk Prediction Method for Lane-Changing Process Using Hidden Markov Model. Sensors, 19.10.3390/s19122670 ↗
11. Validation of a Real-Time Capable Multibody Vehicle Dynamics Formulation for Automotive Testing Frameworks Based on Simulation10.1109/ACCESS.2020.3040232 ↗
12. DMD: A Large-Scale Multi-modal Driver Monitoring Dataset for Attention and Alertness Analysis
13. A driver behavior assessment and recommendation system for connected vehicles to produce safer driving environments through a “follow the leader” approach10.1016/j.aap.2020.105460 ↗
14. Terán, J., Navarro, L., Quintero M., C.G., and Pardo, M. (2020). Intelligent Driving Assistant Based on Road Accident Risk Map Analysis and Vehicle Telemetry. Sensors, 20.10.3390/s20061763 ↗
15. Comparison of Three Traffic Management Plans Showing Shadow and Police Vehicle Effects on Driver Behavior at Highway Single Lane Closures10.1177/0361198120925473 ↗
16. Zahid, M., Chen, Y., Jamal, A., Al-Ofi, K.A., and Al-Ahmadi, H.M. (2020). Adopting Machine Learning and Spatial Analysis Techniques for Driver Risk Assessment: Insights from a Case Study. Int. J. Environ. Res. Public Health, 17.10.3390/ijerph17145193 ↗
17. Robust Obstacle Detection and Recognition for Driver Assistance Systems10.1109/TITS.2019.2909275 ↗
18. Evaluating advanced driver-assistance system trainings using driver performance, attention allocation, and neural efficiency measures10.1016/j.apergo.2019.103036 ↗
19. Wickramanayake, S., Bandara, H.M., and Samarasekara, N.A. (2020). Real-Time Monitoring and Driver Feedback to Promote Fuel Efficient Driving. arXiv Prepr.
20. RBO-EM: Reduced Broadcast Overhead Scheme for Emergency Message Dissemination in VANETs10.1109/ACCESS.2020.3025212 ↗
21. Smartphone Sensors-Based Abnormal Driving Behaviors Detection: Serial-Feature Network10.1109/JSEN.2020.3036862 ↗
22. Attention Based Vehicle Trajectory Prediction10.1109/TIV.2020.2991952 ↗
23. HCF: A Hybrid CNN Framework for Behavior Detection of Distracted Drivers10.1109/ACCESS.2020.3001159 ↗
24. An Adaptive Batch-Image Based Driver Status Monitoring System on a Lightweight GPU-Equipped SBC10.1109/ACCESS.2020.3035393 ↗
25. Rough Set Based Method for Vehicle Collision Risk Assessment Through Inferring Driver’s Braking Actions in Near-Crash Situations10.1109/MITS.2019.2903539 ↗
26. Davoli, L., Martalò, M., Cilfone, A., Belli, L., Ferrari, G., Presta, R., Montanari, R., Mengoni, M., Giraldi, L., and Amparore, E.G. (2020). On Driver Behavior Recognition for Increased Safety: A Roadmap. Safety, 6.10.3390/safety6040055 ↗
27. Lobo, A., Ferreira, S., and Couto, A. (2020). Exploring Monitoring Systems Data for Driver Distraction and Drowsiness Research. Sensors, 20.10.3390/s20143836 ↗
28. Abbas, Q., and Alsheddy, A. (2020). Driver Fatigue Detection Systems Using Multi-Sensors, Smartphone, and Cloud-Based Computing Platforms: A Comparative Analysis. Sensors, 21.10.3390/s21010056 ↗
29. Analysis of Road-User Interaction by Extraction of Driver Behavior Features Using Deep Learning10.1109/ACCESS.2020.2965940 ↗
30. Methodology and Mobile Application for Driver Behavior Analysis and Accident Prevention10.1109/TITS.2019.2918328 ↗
31. Modelling braking behavior and accident probability of drivers under increasing time pressure conditions10.1016/j.aap.2019.105401 ↗
32. Using Asymmetric Theory to Identify Heterogeneous Drivers’ Behavior Characteristics Through Traffic Oscillation10.1109/ACCESS.2019.2930762 ↗
33. XGBoost Algorithm-Based Monitoring Model for Urban Driving Stress: Combining Driving Behavior, Driving Environment, and Route Familiarity10.1109/ACCESS.2021.3055551 ↗
34. Driver behavior detection and classification using deep convolutional neural networks10.1016/j.eswa.2020.113240 ↗
35. Cloud-Based Driver Monitoring System Using a Smartphone10.1109/JSEN.2020.2975382 ↗
36. Chen, L.-W., and Chen, H.-M. (2020). Driver Behavior Monitoring and Warning With Dangerous Driving Detection Based on the Internet of Vehicles. IEEE Trans. Intell. Transp. Syst., 1–10.