Computer Vision-Based Drowsiness Detection Using Handcrafted Feature Extraction for Edge Computing Devices
Source abstract
Abstract
Drowsy driving contributes to over 6000 fatal incidents annually in the US, underscoring the need for effective, non-intrusive drowsiness detection. This study seeks to address detection challenges, particularly in non-standard head positions. Our innovative approach leverages computer vision by combining facial feature detection using Dlib, head pose estimation with the HOPEnet model, and analyses of the percentage of eyelid closure over time (PERCLOS) and the percentage of mouth opening over time (POM). These are integrated with traditional machine learning models, such as Support Vector Machines, Random Forests, and XGBoost. These models were chosen for their ability to process detailed information from facial landmarks, head poses, PERCLOS, and POM. They achieved a high overall accuracy of 86.848% in detecting drowsiness, with a small overall model size of 5.05 MB and increased computational efficiency. The models were trained on the National Tsing Hua University Driver Drowsiness Detection Dataset, making them highly suitable for devices with a limited computational capacity. Compared to the baseline model from the literature, which achieved an accuracy of 84.82% and a larger overall model size of 37.82 MB, the method proposed in this research shows a notable improvement in the efficiency of the model with relatively similar accuracy. These findings provide a framework for future studies, potentially improving sleepiness detection systems and ultimately saving lives by enhancing road safety.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Sleep and Work-Related Fatigue · Non-Invasive Vital Sign Monitoring · EEG and Brain-Computer Interfaces
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Computer Science Department, BINUS Graduate Program—Master of Computer Science, Bina Nusantara University, Jakarta 11480, Indonesia
Department of Electrical, Electronics and Communication Engineering, Faculty of Engineering, Tokyo City University, Tokyo 158-8557, Japan
References (46)
Show source-provided references
1. Wessel, R. (2024, March 01). What Are Driver Drowsiness Detection Systems?|TomTom Blog. TomTom. Available online: https://www.tomtom.com/newsroom/explainers-and-insights/driver-drowsiness-detection-systems/.
2. Deep CNN: A Machine Learning Approach for Driver Drowsiness Detection Based on Eye State
3. Coxworth, B. (2024, March 01). Anti Sleep Pilot Detects Drowsy Drivers. New Atlas. Available online: https://newatlas.com/anti-sleep-pilot-monitors-driver-fatigue/17439/.
4. Driver’s drowsiness detection through computer vision: A review10.1007/978-3-030-02840-4_22 ↗
5. Insurance Information Institute (2024, March 01). Facts + Statistics: Drowsy Driving. Available online: https://www.iii.org/fact-statistic/facts-statistics-drowsy-driving.
6. Artificial neural network systems
7. Chen, L., and Zheng, W. (2023). Research on railway dispatcher fatigue detection method based on deep learning with multi-feature fusion. Electronics, 12.10.3390/electronics12102303 ↗
8. Deep learning application pros and cons over algorithm
9. Phan, A.-C., Nguyen, N.-H.-Q., Trieu, T.-N., and Phan, T.-C. (2021). An efficient approach for detecting driver drowsiness based on deep learning. Appl. Sci., 11.10.3390/app11188441 ↗
10. Bhatt, D., Patel, C., Talsania, H., Patel, J., Vaghela, R., Pandya, S., Modi, K., and Ghayvat, H. (2021). CNN variants for computer vision: History, architecture, application, challenges and future Scope. Electronics, 10.10.3390/electronics10202470 ↗
11. Robust drowsiness detection for vehicle driver using deep convolutional neural network
12. Weng, C.-H., Lai, Y.-H., and Lai, S.-H. (2016, January 20–24). Driver drowsiness detection via a hierarchical temporal deep belief network. Proceedings of the Computer Vision—ACCV 2016 Workshops, Taipei, Taiwan.10.1007/978-3-319-54526-4_9 ↗
13. Convolutional neural networks: An overview and application in radiology10.1007/s13244-018-0639-9 ↗
14. Driver fatigue detection based on convolutional neural networks using EM-CNN10.1155/2020/7251280 ↗
15. Jabbar, R., Shinoy, M., Kharbeche, M., Al-Khalifa, K., Krichen, M., and Barkaoui, K. (2020, January 2–5). Driver drowsiness detection model using convolutional neural networks techniques for android application. Proceedings of the 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT), IoT, Doha, Qatar.10.1109/ICIoT48696.2020.9089484 ↗
16. Real-time driver-drowsiness detection system using facial features10.1109/ACCESS.2019.2936663 ↗
17. Junaedi, S., and Akbar, H. (2017, January 10–12). Driver drowsiness detection based on face feature and PERCLOS. Proceedings of the International Conference on Computation in Science and Engineering, Bandung, Indonesia.
18. Ursulescu, O., Ilie, B., and Simion, G. (2018, January 8–9). Driver drowsiness detection based on eye analysis. Proceedings of the 2018 International Symposium on Electronics and Telecommunications (ISETC), Timisoara, Romania.10.1109/ISETC.2018.8583852 ↗
19. Driver fatigue detection method based on eye states with pupil and iris segmentation10.1109/ACCESS.2020.3025818 ↗
20. A novel approach for driver fatigue detection based on visual characteristics analysis10.1007/s12652-021-03311-9 ↗
21. Kahlon, M., and Ganesan, S. (2018, January 3–5). Driver drowsiness detection system based on binary eyes image data. Proceedings of the 2018 IEEE International Conference on Electro/Information Technology (EIT), Rochester, MI, USA.10.1109/EIT.2018.8500272 ↗
22. Ye, M., Zhang, W., Cao, P., and Liu, K. (2021). Driver fatigue detection based on residual channel attention network and head pose estimation. Appl. Sci., 11.10.3390/app11199195 ↗
23. Sheikh, A.A., and Mir, J. (2021, January 20–21). Machine learning inspired vision-based drowsiness detection using eye and body motion features. Proceedings of the 2021 13th International Conference on Information & Communication Technology and System (ICTS), Surabaya, Indonesia.10.1109/ICTS52701.2021.9608977 ↗
24. Bakheet, S., and Al-Hamadi, A. (2021). A framework for instantaneous driver drowsiness detection based on improved HOG features and naïve bayesian classification. Brain Sci., 11.10.3390/brainsci11020240 ↗
25. Intelligent driver drowsiness detection for traffic safety based on multi CNN deep model and facial subsampling10.1109/TITS.2021.3134222 ↗
26. CDC (2024, March 01). Dangers of Drowsy Driving, Available online: https://stacks.cdc.gov/view/cdc/35742.
27. Rivelli, E., and Kempken, M. (2024, March 01). Drowsy Driving 2021 Facts; Statistics. Bankrate. Available online: https://www.bankrate.com/insurance/car/drowsy-driving-statistics/#drowsy-driving-statistics.
28. Wallis, K., and Johnson, T. (2024, March 01). Why Am I Excessively Sleepy? Drowsiness Explained. 31 August 2022. Available online: https://www.healthgrades.com/right-care/symptoms-and-conditions/drowsiness.
29. Andrew Currin (2024, March 01). NHTSA. National Highway Traffic Safety Administration, Available online: https://www.nhtsa.gov/risky-driving/drowsy-driving.
30. Trop, J. (2024, March 01). Drowsy Driving: Worse Than Drunk Driving? US News. Available online: https://cars.usnews.com/cars-trucks/advice/best-cars-blog/2016/12/drowsy-driving-worse-than-drunk-driving.
31. Gwak, J., Hirao, A., and Shino, M. (2020). An investigation of early detection of driver drowsiness using ensemble machine learning Based on Hybrid Sensing. Appl. Sci., 10.10.3390/app10082890 ↗
32. Real-time driver fatigue or drowsiness detection system using face image stream
33. Amodio, A., Ermidoro, M., Maggi, D., and Savaresi, S.M. (2018, January 12–15). A low-cost system for dynamic analysis of pupillary light response for a driver drowsiness detection system. Proceedings of the 2018 European Control Conference (ECC), Limassol, Cyprus.10.23919/ECC.2018.8550621 ↗
34. Barney, N. (2024, March 01). What Is Face Detection and How Does It Work?. Tech Target., Available online: https://www.techtarget.com/searchenterpriseai/definition/face-detection.
35. Face detection techniques: A review10.1007/s10462-018-9650-2 ↗
36. Lu, W., and Yang, M. (2019, January 15–16). Face detection based on viola-jones algorithm applying composite features. Proceedings of the 2019 International Conference on Robotics and Intelligent System (ICRIS), Haikou, China.10.1109/ICRIS.2019.00029 ↗
37. Bodini, M. (2019). A review of facial landmark extraction in 2D images and videos using deep learning. Big Data Cogn. Comput., 3.10.3390/bdcc3010014 ↗
38. Facial landmark detection: A literature survey10.1007/s11263-018-1097-z ↗
39. Vonikakis, V., and Winkler, S. (2020, January 25–28). Identity-invariant facial landmark frontalization for facial expression analysis. Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates.10.1109/ICIP40778.2020.9190989 ↗
40. Deep learning for head pose estimation: A survey10.1007/s42979-023-01796-z ↗
41. Lee, S. (2024, March 01). Head Pose Estimation with Hopenet. Medium. 11 August 2021. Available online: https://towardsdatascience.com/head-pose-estimation-with-hopenet-5e62ace254d5.
42. Zhang, F., Su, J., Geng, L., and Xiao, Z. (2017, January 17–19). Driver fatigue detection based on eye state recognition. Proceedings of the 2017 International Conference on Machine Vision and Information Technology (CMVIT), Singapore.10.1109/CMVIT.2017.25 ↗
43. Chang, R.C.-H., Wang, C.-Y., Chen, W.-T., and Chiu, C.-D. (2022). Drowsiness detection system based on PERCLOS and facial physiological signal. Sensors, 22.10.3390/s22145380 ↗
44. PERCLOS-based technologies for detecting drowsiness: Current evidence and future directions10.1093/sleepadvances/zpad006 ↗
45. A Survey on state-of-the-art drowsiness detection techniques10.1109/ACCESS.2019.2914373 ↗
46. Tipprasert, W., Charoenpong, T., Chianrabutra, C., and Sukjamsri, C. (2019, January 16–18). A Method of driver’s eyes closure and yawning detection for drowsiness analysis by infrared camera. Proceedings of the 2019 First International Symposium on Instrumentation, Control, Artificial Intelligence, and Robotics (ICA-SYMP), Bangkok, Thailand.10.1109/ICA-SYMP.2019.8646001 ↗