Driver Fatigue and Distracted Driving Detection Using Random Forest and Convolutional Neural Network
Source abstract
Abstract
Driver fatigue and distracted driving are the two most common causes of major accidents. Thus, the on-board monitoring of driving behaviors is key in the development of intelligent vehicles. In this paper, we propose an approach which detects driver fatigue and distracted driving behaviors using vision-based techniques. For driver fatigue detection, a single shot scale-invariant face detector (S3FD) is first used to detect the face in the image and then the face alignment network (FAN) is utilized to extract facial features. After that, the facial features are used to determine the driver’s yawns, head posture, and the opening or closing of their eyes. Finally, the random forest technique is used to analyze the driving conditions. For distracted driving detection, a convolutional neural network (CNN) is used to classify various distracted driving behaviors. Also, Adam optimizer is used to reinforce optimization performance. Compared with existing methods, our approach is more accurate and efficient. Moreover, distracted driving can be detected in real-time running on the embedded hardware.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Sleep and Work-Related Fatigue · IoT and GPS-based Vehicle Safety Systems · Autonomous Vehicle Technology and Safety
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electrical Engineering, National Chung Cheng University, Chiayi 62102, Taiwan
Department of Computer Science and Information Engineering, National Taipei University of Technology, Taipei 10608, Taiwan
Department of Computer Science and Information Engineering, National United University, Miaoli 36003, Taiwan
Funding
Ministry of Science and Technology of Taiwan
References (38)
Show source-provided references
1. Lee, K.W., Yoon, H.S., Song, J.M., and Park, K.R. (2018). Convolutional neural network-based classification of driver’s emotion during aggressive and smooth driving using multi-modal camera sensors. Sensors, 18.10.3390/s18040957 ↗
2. Lin, H.Y., Dai, J.M., Wu, L.T., and Chen, L.Q. (2020). A vision based driver assistance system with forward collision and overtaking detection. Sensors, 20.10.3390/s20185139 ↗
3. Dong, B.T., and Lin, H.Y. (2021, January 10–12). An on-board monitoring system for driving fatigue and distraction detection. Proceedings of the 22nd IEEE International Conference on Industrial Technology (ICIT), Valencia, Spain.10.1109/ICIT46573.2021.9453676 ↗
4. Driver distraction detection methods: A literature review and framework10.1109/ACCESS.2021.3073599 ↗
5. (2020, January 05). Distract CNN. Available online: https://github.com/nkkumawat/Driver-Distraction-Detection/branches.
6. Zhang, S., Zhu, X., Lei, Z., Shi, H., Wang, X., and Li, S.Z. (2017, January 22–29). S3fd: Single shot scale-invariant face detector. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.30 ↗
7. Bulat, A., and Tzimiropoulos, G. (2017, January 22–29). How far are we from solving the 2d 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks). Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.10.1109/ICCV.2017.116 ↗
8. Li, Z., Li, S., Cheng, B., and Shi, J. (2017). Online detection of driver fatigue using steering wheel angles for real driving conditions. Sensors, 17.10.3390/s17030495 ↗
9. Eeg-based drowsiness detection for safe driving using chaotic features and statistical tests10.4103/2228-7477.95297 ↗
10. Babaeian, M., and Mozumdar, M. (2019, January 7–9). Driver drowsiness detection algorithms using electrocardiogram data analysis. Proceedings of the 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA.10.1109/CCWC.2019.8666467 ↗
11. On-road detection of driver fatigue and drowsiness during medium-distance journeys10.3390/e23020135 ↗
12. Hypo-driver: A multiview driver fatigue and distraction level detection system
13. Danisman, T., Bilasco, I.M., Djeraba, C., and Ihaddadene, N. (2010, January 3–5). Drowsy driver detection system using eye blink patterns. Proceedings of the 2010 International Conference on Machine and Web Intelligence, Algiers, Algeria.10.1109/ICMWI.2010.5648121 ↗
14. Abtahi, S., Hariri, B., and Shirmohammadi, S. (2011, January 10–12). Driver drowsiness monitoring based on yawning detection. Proceedings of the 2011 IEEE International Instrumentation and Measurement Technology Conference, Hangzhou, China.10.1109/IMTC.2011.5944101 ↗
15. Savas, B.K., and Becerikli, Y. (2018, January 25–27). Real time driver fatigue detection based on SVM algorithm. Proceedings of the 2018 6th International Conference on Control Engineering Information Technology (CEIT), Istanbul, Turkey.10.1109/CEIT.2018.8751886 ↗
16. Ou, W., Shih, M., Chang, C., Yu, X., and Fan, C. (2015, January 6–8). Intelligent video-based drowsy driver detection system under various illuminations and embedded software implementation. Proceedings of the 2015 IEEE International Conference on Consumer Electronics, Taipei, Taiwan.10.1109/ICCE-TW.2015.7216850 ↗
17. A smartphone-based drowsiness detection and warning system for automotive drivers10.1109/TITS.2018.2879609 ↗
18. Qiao, Y., Zeng, K., Xu, L., and Yin, X. (2016, January 9–12). A smartphone-based driver fatigue detection using fusion of multiple real-time facial features. Proceedings of the 2016 13th IEEE Annual Consumer Communications Networking Conference (CCNC), Las Vegas, NV, USA.10.1109/CCNC.2016.7444761 ↗
19. Galarza, E.E., Egas, F.D., Silva, F.M., Velasco, P.M., and Galarza, E.D. (2018, January 10–12). Real time driver drowsiness detection based on driver’s face image behavior using a system of human computer interaction implemented in a smartphone. Proceedings of the International Conference on Information Technology & Systems (ICITS 2018), Libertad City, Ecuador.10.1007/978-3-319-73450-7_53 ↗
20. Zhang, W., and Su, J. (2017–1, January 27). Driver yawning detection based on long short term memory networks. Proceedings of the 2017 IEEE Symposium Series on Computational Intelligence (SSCI), Honolulu, HI, USA.10.1109/SSCI.2017.8285343 ↗
21. Akrout, B., and Mahdi, W. (2016, January 5–7). Yawning detection by the analysis of variational descriptor for monitoring driver drowsiness. Proceedings of the 2016 International Image Processing, Applications and Systems (IPAS), Hammamet, Tunisia.10.1109/IPAS.2016.7880127 ↗
22. Abouelnaga, Y., Eraqi, H.M., and Moustafa, M.N. (2017). Real-time distracted driver posture classification. arXiv.
23. Baheti, B., Gajre, S., and Talbar, S. (2018, January 18–22). Detection of distracted driver using convolutional neural network. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.10.1109/CVPRW.2018.00150 ↗
24. Kose, N., Kopuklu, O., Unnervik, A., and Rigoll, G. (2019, January 27–30). Real-time driver state monitoring using a cnn based spatio-temporal approach. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.10.1109/ITSC.2019.8917460 ↗
25. Jain, A., Koppula, H.S., Raghavan, B., Soh, S., and Saxena, A. (2015, January 7–13). Car that knows before you do: Anticipating maneuvers via learning temporal driving models. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.10.1109/ICCV.2015.364 ↗
26. Distracted driver detection and classification
27. Majdi, M.S., Ram, S., Gill, J.T., and Rodríguez, J.J. (2018, January 8–10). Drive-net: Convolutional network for driver distraction detection. Proceedings of the 2018 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI), Las Vegas, NV, USA.10.1109/SSIAI.2018.8470309 ↗
28. Moslemi, N., Azmi, R., and Soryani, M. (2019, January 6–7). Driver distraction recognition using 3d convolutional neural networks. Proceedings of the 2019 4th International Conference on Pattern Recognition and Image Analysis (IPRIA), Tehran, Iran.10.1109/PRIA.2019.8786012 ↗
29. Anber, S., Alsaggaf, W., and Shalash, W. (2022). A hybrid driver fatigue and distraction detection model using AlexNet based on facial features. Electronics, 11.10.3390/electronics11020285 ↗
30. Newell, A., Yang, K., and Deng, J. (2016, January 11–14). Stacked hourglass networks for human pose estimation. Proceedings of the 14th European Conference on Computer Vision (ECCV2016), Amsterdam, The Netherlands.10.1007/978-3-319-46484-8_29 ↗
31. Dlib-ml: A machine learning toolkit
32. Blinking extraction in eye gaze system for stereoscopy movies10.1515/phys-2019-0053 ↗
33. Relangi, S., Nilesh, M., Kumar, K., and Naveen, A. (2019, January 21–22). Full length driver drowsiness detection model—Utilising driver specific judging parameters. Proceedings of the International Conference on Intelligent Manufacturing and Energy Sustainability (ICIMES 2019), Hyderabad, India.10.1007/978-981-15-1616-0_77 ↗
34. Abtahi, S., Omidyeganeh, M., Shirmohammadi, S., and Hariri, B. (2014, January 19–21). Yawdd: A yawning detection dataset. Proceedings of the 5th ACM Multimedia Systems Conference, Singapore.10.1145/2557642.2563678 ↗
35. Efficient and compact face descriptor for driver drowsiness detection10.1016/j.eswa.2020.114334 ↗
36. Bakheet, S., and Al-Hamadi, A. (2021). A framework for instantaneous driver drowsiness detection based on improved HOG features and Naïve Bayesian classification. Brain Sci., 11.10.3390/brainsci11020240 ↗
37. (2020, January 05). 10 Classes. Available online: https://www.kaggle.com/competitions/state-farm-distracted-driver-detection/data.
38. Zhang, B. (2016). Apply and compare different classical image classification method: Detect distracted driver. CS 229 Project Report, Stanford University.