Contactless Pulse Rate Assessment: Results and Insights for Application in Driving Simulators
Source abstract
Abstract
Remote photoplethysmography (rPPG) offers a promising solution for non-contact driver monitoring by detecting subtle blood flow-induced facial color changes from video. However, motion artifacts in dynamic driving environments remain key challenges. This study presents an rPPG framework that combines signal processing techniques before and after applying Eulerian Video Magnification (EVM) for pulse rate (PR) estimation in driving simulators. While not novel, the approach offers insights into the efficiency of the EVM method and its time complexity. We compare results of the proposed rPPG approach against reference Empatica E4 data and also compare it with existing achievements from the literature. Additionally, the possible bias of the Empatica E4 is further assessed using an independent dataset with both the Empatica E4 and the Faros 360 measurements. EVM slightly improves PR estimation, reducing the mean absolute error (MAE) from 6.48 bpm to 5.04 bpm (the lowest MAE (~2 bpm) was achieved under strict conditions) with an additional time required for EVM of about 20 s for 30 s sequence. Furthermore, statistically significant differences are identified between younger and older drivers in both reference and rPPG data. Our findings demonstrate the feasibility of using rPPG-based PR monitoring, encouraging further research in driving simulations.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Non-Invasive Vital Sign Monitoring · Heart Rate Variability and Autonomic Control · Sleep and Work-Related Fatigue
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Electrical Engineering, University of Belgrade, 11120 Belgrade, Serbia
Vinča Institute of Nuclear Sciences—National Institute of the Republic of Serbia, University of Belgrade, 11351 Belgrade, Serbia
Faculty of Electrical Engineering, University of Ljubljana, 1000 Ljubljana, Slovenia
Funding
Ministry of Science, Technological Development and Innovation of the Republic of Serbia · lovenian Research Agency within program ICT4QL
References (90)
Show source-provided references
1. Renner, P., Gleichauf, J., and Winkelmann, S. (2024, January 1–4). Non-Contact In-Car Monitoring of Heart Rate: Evaluating the Eulerian Video Magnification Algorithm in a Driving Simulator Study. Proceedings of the Mensch und Computer 2024, Karlsruhe, Germany.10.1145/3670653.3677493 ↗
2. Nijskens, L., van der Hurk, S.E., van den Broek, S.P., Louvenberg, S., Souman, J.L., Bos, J.E., and ter Haar, F.B. (2024, January 13). An EO/IR monitoring system for noncontact physiological signal analysis in automated vehicles. Proceedings of the SPIE Autonomous Systems for Security and Defence, Edinburgh, UK.10.1117/12.3033956 ↗
3. Continuous Monitoring of Heart Rate Variability in Free-Living Conditions Using Wearable Sensors: Exploratory Observational Study10.2196/53977 ↗
4. Simulation-based driver scoring and profiling system10.1016/j.heliyon.2024.e40310 ↗
5. Boboc, R.G., Butilă, E.V., and Butnariu, S. (2024). Leveraging wearable sensors in virtual reality driving simulators: A review of techniques and applications. Sensors, 24.10.3390/s24134417 ↗
6. Photoplethysmography revisited: From contact to noncontact, from point to imaging10.1109/TBME.2015.2476337 ↗
7. Dudarev, V., Barral, O., Zhang, C., Davis, G., and Enns, J.T. (2023). On the reliability of wearable technology: A tutorial on measuring heart rate and heart rate variability in the wild. Sensors, 23.10.3390/s23135863 ↗
8. Ronca, V., Martinez-Levy, A.C., Vozzi, A., Giorgi, A., Aricò, P., Capotorto, R., Borghini, G., Babiloni, F., and Di Flumeri, G. (2023). Wearable technologies for electrodermal and cardiac activity measurements: A comparison between fitbit sense, empatica E4 and shimmer GSR3+. Sensors, 23.10.20944/preprints202306.0378.v1 ↗
9. Measuring pulse rate with a webcam
10. Kwon, S., Kim, H., and Park, K.S. (September, January 28). Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone. Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, USA.
11. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation10.1364/OE.18.010762 ↗
12. Ernst, H., Scherpf, M., Malberg, H., and Schmidt, M. (2021). Optimal color channel combination across skin tones for remote heart rate measurement in camera-based photoplethysmography. Biomed. Signal Process. Control, 68.10.1016/j.bspc.2021.102644 ↗
13. Eulerian video magnification for revealing subtle changes in the world10.1145/2185520.2185561 ↗
14. Miljković, N., and Trifunović, D. (2014, January 25–27). Pulse rate assessment: Eulerian video magnification vs. electrocardiography recordings. Proceedings of the 12th Symposium on Neural Network Applications in Electrical Engineering (NEUREL), Belgrade, Serbia.10.1109/NEUREL.2014.7011447 ↗
15. Phase-based video motion processing10.1145/2461912.2461966 ↗
16. Balakrishnan, G., Durand, F., and Guttag, J. (2013, January 23–28). Detecting pulse from head motions in video. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.440 ↗
17. Lomaliza, J.P., and Park, H. (2016, January 8–10). Detecting pulse from head motions using smartphone camera. Proceedings of the International Conference on Advanced Engineering Theory and Applications, Busan, Vietnam.10.1007/978-3-319-50904-4_24 ↗
18. Gruden, T., Pececnik, K.S., Jakus, G., and Sodnik, J. (2022, January 29). Quantifying Drivers’ Physiological Responses to Take-Over Requests in Conditionally Automated Vehicles. Proceedings of the Human-Computer Interaction Slovenia 2022, Ljubljana, Slovenia.
19. Raybaut, P. (2025, August 26). Spyder-Documentation. Available online: https://www.spyder-ide.org/.
20. Array programming with NumPy10.1038/s41586-020-2649-2 ↗
21. Bradski, G., and Kaehler, A. (2000). OpenCV. Dr. Dobb’s J. Softw. Tools, 3, Available online: https://github.com/opencv/opencv/wiki/CiteOpenCV.
22. SciPy 1.0: Fundamental algorithms for scientific computing in Python10.1038/s41592-019-0686-2 ↗
23. Scikit-learn: Machine learning in Python
24. Van Rossum, G. (2020). The Python Library Reference, Release 3.8. 2., Python Software Foundation.
25. McCarthy, C., Pradhan, N., Redpath, C., and Adler, A. (2016, January 29–31). Validation of the Empatica E4 wristband. Proceedings of the 2016 IEEE EMBS International Student Conference (ISC), Ottawa, ON, Canada.10.1109/EMBSISC.2016.7508621 ↗
26. Validity of the Empatica E4 wristband to measure heart rate variability (HRV) parameters: A comparison to electrocardiography (ECG)10.1007/s10916-020-01648-w ↗
27. Ambulatory heart rate variability monitoring: Comparisons between the empatica e4 wristband and holter electrocardiogram
28. Jocher, G., Chaurasia, A., and Qiu, J. (2025, August 26). YOLOv8 Docs by Ultralytics (Version 8.0. 0). [software]. Available online: https://github.com/ultralytics/ultralytics.
29. An analysis of the Viola-Jones face detection algorithm10.5201/ipol.2014.104 ↗
30. Rapid object detection using a boosted cascade of simple features
31. Li, X., Komulainen, J., Zhao, G., Yuen, P.C., and Pietikäinen, M. (2016, January 4–8). Generalized face anti-spoofing by detecting pulse from face videos. Proceedings of the 2016 23rd International Conference on Pattern Recognition (ICPR), Cancun, Mexico.
32. Gonzalez, R.C. (2009). Digital Image Processing, Pearson Education India.10.1117/1.3115362 ↗
33. The architecture and performance of the face and eyes detection system based on the Haar cascade classifiers10.1007/s10044-009-0150-5 ↗
34. Rudinskaya, E., and Paringer, R. (2020). Face Detection Accuracy Study Based on Race and Gender Factor Using Haar Cascades, CEUR Workshop Proceedings.
35. Yu, S.G., Kim, S.E., Kim, N.H., Suh, K.H., and Lee, E.C. (2021). Pulse rate variability analysis using remote photoplethysmography signals. Sensors, 21.10.3390/s21186241 ↗
36. Unifying frame rate and temporal dilations for improved remote pulse detection10.1016/j.cviu.2021.103246 ↗
37. Uncertainties in the analysis of heart rate variability: A systematic review10.1109/RBME.2023.3271595 ↗
38. Clinically accurate fetal ECG parameters acquired from maternal abdominal sensors10.1016/j.ajog.2011.02.066 ↗
39. Developed with the Special Contribution of the European Heart Rhythm Association (EHRA), Endorsed by the European Association for Cardio-Thoracic Surgery (EACTS), Authors/Task Force Members, Camm, A.J., Kirchhof, P., Lip, G.Y.H., Schotten, U., Savelieva, I., Ernst, S., and Van Gelder, I.C. (2010). Guidelines for the management of atrial fibrillation: The Task Force for the Management of Atrial Fibrillation of the European Society of Cardiology (ESC). Eur. Heart J., 31, 2369–2429.
40. The efficiency of 10-second resting heart rate for the evaluation of short-term heart rate variability indices10.1111/j.1540-8159.2011.03178.x ↗
41. A new algorithm for fetal heart rate detection: Fractional order calculus approach10.1016/j.medengphy.2023.104007 ↗
42. Webcam-based, non-contact, real-time measurement for the physiological parameters of drivers10.1016/j.measurement.2017.01.007 ↗
43. Speed up Eulerian Video Motion Magnification10.24017/science.2017.3.14 ↗
44. Klabunde, R. (2011). Cardiovascular Physiology Concepts, Lippincott Williams & Wilkins.
45. Wang, C., Pun, T., and Chanel, G. (2018). A comparative survey of methods for remote heart rate detection from frontal face videos. Front. Bioeng. Biotechnol., 6.10.3389/fbioe.2018.00033 ↗
46. Lim, K.S., Moya-Bello, E., and Chavarria-Zamora, L. (2021, January 24–26). Resource Optimization of the Eulerian Video Magnification Algorithm Towards an Embedded Architecture. Proceedings of the 2021 IEEE URUCON, Montevideo, Uruguay.10.1109/URUCON53396.2021.9647386 ↗
47. Zhang, K., Jin, X., and Wu, A. (2017, January 19–22). Accelerating Eulerian video magnification using FPGA. Proceedings of the 2017 19th International Conference on Advanced Communication Technology (ICACT), PyeongChang, Republic of Korea.10.23919/ICACT.2017.7890151 ↗
48. Bishop, C.M., and Nasrabadi, N.M. (2006). Pattern Recognition and Machine Learning, Springer. Number 4.
49. Huang, R., Hong, K.S., Yang, D., and Huang, G. (2022). Motion artifacts removal and evaluation techniques for functional near-infrared spectroscopy signals: A review. Front. Neurosci., 16.10.3389/fnins.2022.878750 ↗
50. Sathyapriya, L., Murali, L., and Manigandan, T. (2014, January 8–10). Analysis and detection R-peak detection using Modified Pan-Tompkins algorithm. Proceedings of the 2014 IEEE International Conference on Advanced Communications, Control and Computing Technologies, Ramanathapuram, India.10.1109/ICACCCT.2014.7019490 ↗
51. An optimally designed digital differentiator based preprocessor for R-peak detection in electrocardiogram signal10.1016/j.bspc.2018.09.005 ↗
52. Physiological responses to simulated and on-road driving10.1016/j.ijpsycho.2011.06.012 ↗
53. Kohlhaas, M., Seidlmayer, L., and Kaspar, M. (2021). A Specialized System for Arrhythmia Detection for Basic Research in Cardiology. German Medical Data Sciences: Bringing Data to Life, IOS Press.10.3233/SHTI210041 ↗
54. Biofingerprint detection of corona virus using Raman spectroscopy: A novel approach10.1007/s42452-023-05419-3 ↗
55. Comparison of values of Pearson’s and Spearman’s correlation coefficients on the same sets of data
56. Smith, L.I. (2002). A Tutorial on Principal Components Analysis, University of Otago.
57. Yu, Z., Li, X., and Zhao, G. (2019). Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks. arXiv.
58. Garbarino, M., Lai, M., Bender, D., Picard, R.W., and Tognetti, S. (2014, January 3–5). Empatica E3—A wearable wireless multi-sensor device for real-time computerized biofeedback and data acquisition. Proceedings of the 2014 4th International Conference on Wireless Mobile Communication and Healthcare-Transforming Healthcare Through Innovations in Mobile and Wireless Technologies (MOBIHEALTH), Athens, Greece.10.4108/icst.mobihealth.2014.257418 ↗
59. Heart rate estimation from wrist-worn photoplethysmography: A review10.1109/JSEN.2019.2914166 ↗
60. Xiao, H., Liu, T., Sun, Y., Li, Y., Zhao, S., and Avolio, A. (2024). Remote photoplethysmography for heart rate measurement: A review. Biomed. Signal Process. Control., 88.10.1016/j.bspc.2023.105608 ↗
61. Unsupervised skin tissue segmentation for remote photoplethysmography10.1016/j.patrec.2017.10.017 ↗
62. Benezeth, Y., Krishnamoorthy, D., Monsalve, D.J.B., Nakamura, K., Gomez, R., and Mitéran, J. (2024). Video-based heart rate estimation from challenging scenarios using synthetic video generation. Biomed. Signal Process. Control., 96.10.1016/j.bspc.2024.106598 ↗
63. Implementation of haar cascade classifier and eye aspect ratio for driver drowsiness detection using raspberry Pi10.13189/ujeee.2019.061609 ↗
64. Investigating sources of inaccuracy in wearable optical heart rate sensors10.1038/s41746-020-0226-6 ↗
65. Validity of the Empatica E4 wristband to estimate resting-state heart rate variability in a lab-based context10.1016/j.ijpsycho.2022.10.003 ↗
66. Medarević, J., Miljković, N., Stojmenova Pečečnik, K., and Sodnik, J. (2025). Distress Detection in VR environment using Empatica E4 wristband and Bittium Faros 360. Front. Physiol., 16.10.3389/fphys.2025.1480018 ↗
67. Effectiveness of the chest strap electrocardiogram to detect atrial fibrillation10.1016/j.amjcard.2019.02.028 ↗
68. Hess, M.R., and Kromrey, J.D. (2004, January 12–16). Robust confidence intervals for effect sizes: A comparative study of Cohen’s d and Cliff’s delta under non-normality and heterogeneous variances. Proceedings of the Annual Meeting of the American Educational Research Association, San Diego, CA, USA.
69. Tang, J., Chen, K., Wang, Y., Shi, Y., Patel, S., McDuff, D., and Liu, X. (2023, January 24–27). Mmpd: Multi-domain mobile video physiology dataset. Proceedings of the 2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Sydney, Australia.10.1109/EMBC40787.2023.10340857 ↗
70. Lee, H., Cho, A., and Whang, M. (2021). Fusion method to estimate heart rate from facial videos based on RPPG and RBCG. Sensors, 21.10.3390/s21206764 ↗
71. Ma, X., Tang, J., Jiang, Z., Cheng, S., Shi, Y., Li, D., Zhang, T., Liu, H., Chen, L., and Zhao, Q. (2025). Non-Contact Health Monitoring During Daily Personal Care Routines. arXiv.
72. Contactless heart rate estimation from face videos
73. Lee, H., Ko, H., Chung, H., Nam, Y., Hong, S., and Lee, J. (2022). Real-time realizable mobile imaging photoplethysmography. Sci. Rep., 12.10.1038/s41598-022-11265-x ↗
74. Fallet, S., Schoenenberger, Y., Martin, L., Braun, F., Moser, V., and Vesin, J.M. (2017, January 24–27). Imaging photoplethysmography: A real-time signal quality index. Proceedings of the 2017 Computing in Cardiology (CinC), Rennes, France.10.22489/CinC.2017.142-139 ↗
75. Twenty-four hour time domain heart rate variability and heart rate: Relations to age and gender over nine decades10.1016/S0735-1097(97)00554-8 ↗
76. Ruba, M., Jeyakumar, V., Gurucharan, M.K., Kousika, V., and Viveka, S. (2020, January 10–11). Non-contact pulse rate measurement using facial videos. Proceedings of the 2020 IEEE International Conference on Advances and Developments in Electrical and Electronics Engineering (ICADEE), Coimbatore, India.
77. Non-contact video-based estimation of heart rate variability spectrogram from hemoglobin composition10.1007/s10015-017-0382-1 ↗
78. Ravindran, K.K., Della Monica, C., Atzori, G., Lambert, D., Revell, V., and Dijk, D.J. (2022, January 11–15). Evaluating the Empatica E4 derived heart rate and heart rate variability measures in older men and women. Proceedings of the 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Glasgow, UK.10.1109/EMBC48229.2022.9871559 ↗
79. Comparison of Apple watch vs KardiaMobile: A tale of two devices10.1016/j.cjco.2022.07.011 ↗
80. The normal range and determinants of the intrinsic heart rate in man10.1093/cvr/4.2.160 ↗
81. Reimer, B., Mehler, B.L., Pohlmeyer, A.E., Coughlin, J.F., and Dusek, J.A. (2006). The use of heart rate in a driving simulator as an indicator of age-related differences in driver workload. Adv. Transp. Stud. Int. J., 9–20. Available online: https://www.atsinternationaljournal.com/2006-issues/the-use-of-heart-rate-in-a-driving-simulator-as-an-indicator-of-age-related-differences-in-driver-workload/.
82. Nešković, Đ.D., Stojmenova Pečečnik, K., Sodnik, J., and Miljković, N. (2025). Dataset comprising extracted R, G, and B components for assessment of remote photopletismography (Version 1) [Data set]. Zenodo.
83. Contactless physiological signals extraction based on skin color magnification10.1117/1.JEI.26.6.063003 ↗
84. Garg, D., Goel, P., Pandya, S., Ganatra, A., and Kotecha, K. (December, January 30). A deep learning approach for face detection using YOLO. Proceedings of the 2018 IEEE Punecon, Pune, India.10.1109/PUNECON.2018.8745376 ↗
85. Skin segmentation using color pixel classification: Analysis and comparison10.1109/TPAMI.2005.17 ↗
86. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27–30). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.91 ↗
87. The Evaluation of Remote Monitoring Technology Across Participants with Different Skin Tones
88. Talukdar, D., de Deus, L.F., and Sehgal, N. (2023). Evaluation of Remote Monitoring Technology across different skin tone participants. MedRxiv.10.1101/2023.04.02.23288057 ↗
89. Das, R., Negi, G., and Smeaton, A.F. (2021). Detecting deepfake videos using euler video magnification. arXiv.10.2352/ISSN.2470-1173.2021.4.MWSF-272 ↗
90. Hernandez-Ortega, J., Tolosana, R., Fierrez, J., and Morales, A. (2022). Deepfakes detection based on heart rate estimation: Single-and multi-frame. Handbook of Digital Face Manipulation and Detection: From DeepFakes to Morphing Attacks, Springer International Publishing.10.1007/978-3-030-87664-7_12 ↗