Research on a Real-Time Driver Fatigue Detection Algorithm Based on Facial Video Sequences
Source abstract
Abstract
The research on driver fatigue detection is of great significance to improve driving safety. This paper proposes a real-time comprehensive driver fatigue detection algorithm based on facial landmarks to improve the detection accuracy, which detects the driver’s fatigue status by using facial video sequences without equipping their bodies with other intelligent devices. A tasks-constrained deep convolutional network is constructed to detect the face region based on 68 key points, which can solve the optimization problem caused by the different convergence speeds of each task. According to the real-time facial video images, the eye feature of the eye aspect ratio (EAR), mouth aspect ratio (MAR) and percentage of eye closure time (PERCLOS) are calculated based on facial landmarks. A comprehensive driver fatigue assessment model is established to assess the fatigue status of drivers through eye/mouth feature selection. After a series of comparative experiments, the results show that this proposed algorithm achieves good performance in both accuracy and speed for driver fatigue detection.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Sleep and Work-Related Fatigue · Ergonomics and Musculoskeletal Disorders · Traffic and Road Safety
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Mechanical and Automotive Engineering, Zhaoqing University, Zhaoqing 526021, China
College of Mechanical and Equipment Engineering, Hebei University of Engineering, Handan 056021, China
Guangdong Zhaoqing Institute of Quality Inspection & Metrology, Zhaoqing 526000, China
Department of Engineering, University of Cambridge, Cambridge CB2 1PZ, UK
Funding
Teaching Quality and Reform of Higher Vocational Education Project of Guangdong Province · Characteristic innovation project of Guangdong Provincial Department of Education · Zhaoqing Research and Development Technology and Application of Energy Conserva-tion and Environmental Protection Ecological Governance · Quality Engineering Project of Zhaoqing University · Special Projects in Key Fields of Colleges and Universities of Guangdong Province
References (36)
Show source-provided references
1. Automatic detection of driver impairment based on pupillary light relex10.1109/TITS.2018.2871262 ↗
2. Rear-End Road Crash Characteristics Analysis Based on Chinese In-Depth Crash Study Data
3. The link between fatigue and safety10.1016/j.aap.2009.11.011 ↗
4. Traffic accidents involving fatigue driving and their extent of casualties10.1016/j.aap.2015.10.033 ↗
5. Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness10.1016/j.neubiorev.2012.10.003 ↗
6. The significance of the development of road safety performance indicators related to driver fatigue10.1016/j.trpro.2020.03.024 ↗
7. Real-time detection method of driver fatigue state based on deep learning of face video10.1007/s11042-021-10930-z ↗
8. Exploration of the effects of task-related fatigue on eye-motion features and its value in improving driver fatigue-related technology10.1016/j.trf.2021.03.014 ↗
9. Monitoring drivers’ sleepy status at night based on machine vision10.1007/s11042-016-4103-x ↗
10. Ren, S., Cao, X., Wei, Y., and Sun, J. (2014, January 23–28). Face Alignment at 3000 FPS via Regressing Local Binary Features. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.218 ↗
11. Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks10.1109/LSP.2016.2603342 ↗
12. Driver Fatigue Detection through Chaotic Entropy Analysis of Cortical Sources Obtained From Scalp EEG Signals10.1109/TITS.2018.2890332 ↗
13. Zhang, C., Ma, J., Zhao, J., Liu, P., Cong, F., Liu, T., Li, Y., Sun, L., and Chang, R. (2020). Decoding Analysis of Alpha Oscillation Networks on Maintaining Driver Alertness. Entropy, 22.10.3390/e22070787 ↗
14. Detection of Malicious PDF Files Using a Two-Stage Machine Learning Algorithm10.1049/cje.2020.10.002 ↗
15. Hao, Z., Wan, G., Tian, Y., Tang, Y., Dai, T., Liu, M., and Wei, R. (2019, January 30–31). Research on Driver Fatigue Detection Method Based on Parallel Convolution Neural Network. Proceedings of the 2019 IEEE International Conference on Power, Intelligent Computing and Systems (ICPICS), Shenyang, China.10.1109/ICPICS47731.2019.8942528 ↗
16. Liu, W., Sun, H., and Shen, W. (2010, January 16–18). Driver fatigue detection through pupil detection and yawing analysis. Proceedings of the 2010 International Conference on Bioinformatics and Biomedical Technology, Chengdu, China.
17. Driver drowsiness detection using heart rate and behavior methods: A study10.1016/B978-0-12-819314-3.00011-2 ↗
18. Villanueva, A., Benemerito, R.L.L., Cabug-Os, M.J.M., Chua, R.B., Rebeca, C.K.D.C., and Miranda, M. (2019, January 24–25). Somnolence Detection System Utilizing Deep Neural Network. Proceedings of the 2019 International Conference on Information and Communications Technology (ICOIACT), Yogyakarta, Indonesia.10.1109/ICOIACT46704.2019.8938460 ↗
19. Real Time Driver Fatigue Detection System Based on Multi-Task ConNN10.1109/ACCESS.2020.2963960 ↗
20. Driver Fatigue Detection Based On Facial Feature Analysis10.1142/S0218001421500348 ↗
21. A dynamic center and multi threshold point based stable feature extraction network for driver fatigue detection utilizing EEG signals10.1007/s11571-020-09601-w ↗
22. Wang, F., Xu, Q., and Fu, R. (2019). Study on the Effect of Man-Machine Response Mode to Relieve Driving Fatigue Based on EEG and EOG. Sensors, 19.10.3390/s19224883 ↗
23. A Complex Network-Based Broad Learning System for Detecting Driver Fatigue From EEG Signals
24. Driver fatigue detection based on prefrontal EEG using multi-entropy measures and hybrid model10.1016/j.bspc.2021.102857 ↗
25. He, J., Chen, J., Liu, J., and Li, H. (2019, January 6–8). A Lightweight Architecture for Driver Status Monitoring via Convolutional Neural Networks. Proceedings of the 2019 IEEE International Conference on Robotics and Biomimetics (ROBIO), Dali, China.10.1109/ROBIO49542.2019.8961845 ↗
26. Carlotta Olivetti, E., Violante, M.G., Vezzetti, E., Marcolin, F., and Eynard, B. (2020). Engagement evaluation in a virtual learning environment via facial expression recognition and self-reports: A Preliminary Approach. Appl. Sci., 10.10.3390/app10010314 ↗
27. Useche, O., and El-Sheikh, E. (2015, January 27–30). An Intelligent System Framework for Measuring Attention Levels of Students in Online Course Environments. Proceedings of the International Conference on Artificial Intelligence (ICAI), The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp), Las Vegas, NV, USA.
28. Nonis, F., Dagnes, N., Marcolin, F., and Vezzetti, E. (2019). 3D Approaches and Challenges in Facial Expression Recognition Algorithms—A Literature Review. Appl. Sci., 9.10.3390/app9183904 ↗
29. Facial expressions recognition based on convolutional neural networks for mobile virtual reality10.1145/3013971.3014025 ↗
30. A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting10.1006/jcss.1997.1504 ↗
31. Zhang, Z., Luo, P., Loy, C.C., and Tang, X. (2014). Facial Landmark Detection by Deep Multi-task Learning. European Conference on Computer Vision, Springer.10.1007/978-3-319-10599-4_7 ↗
32. Cheng, R., Zhao, Y., and DAI, Y. (2012). An On-Board Embedded Driver Fatigue Warning System Based on Adaboost Method, Acta Scientiarum Naturalium Universitatis Pekinensis.
33. Design of multi-feature fusion driver fatigue state detection system based on FPGA
34. Dinges, D.F., and Grace, R. (1998). PERCLOS: A Valid Psychophysiological Measure of Alertness as Assessed by Psychomotor Vigilance, Technical Briefs.
35. Assessment of Driver Mental Fatigue Using Facial Landmarks10.1109/ACCESS.2019.2947692 ↗
36. (2021, October 07). Fddb Dataset Official. Available online: http://vis-www.cs.umass.edu/fddb/index.html.