Performance Evaluation of rPPG Approaches with and without the Region-of-Interest Localization Step
Source abstract
Abstract
Traditionally, the first step in physiological measurements based on remote photoplethysmography (rPPG) is localizing the region of interest (ROI) that contains a desired pulsatile information. Recently, approaches that do not require this step have been proposed. The purpose of this study was to evaluate the performance of selected approaches with and without ROI localization step in rPPG signal extraction. The Viola-Jones face detector and Kanade–Lucas–Tomasi tracker (VK) in combination with (a) a region-of-interest (ROI) cropping, (b) facial landmarks, (c) skin-color segmentation, and (d) skin detection based on maximization of mutual information and an approach without ROI localization step (Full Video Pulse (FVP)) were studied. Final rPPG signals were extracted using selected model-based and data-driven rPPG algorithms. The performance of the approaches was tested on three publicly available data sets offering compressed and uncompressed video recordings covering various scenarios. The success rates of pulse waveform signal extraction range from 88.37% (VK with skin-color segmentation) to 100% (FVP). In challenging scenarios (skin tone, lighting conditions, exercise), there were no statistically significant differences between the studied approaches in terms of SNR. The best overall performance in terms of RMSE was achieved by a combination of VK with ROI cropping and the model-based rPPG algorithm. Results indicate that the selection of the ROI localization approach does not significantly affect rPPG measurements if combined with a robust algorithm for rPPG signal extraction.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Non-Invasive Vital Sign Monitoring · Heart Rate Variability and Autonomic Control · Hemodynamic Monitoring and Therapy
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Mechanical Engineering, University of Ljubljana, Aškerčeva cesta 6, 1000 Ljubljana, Slovenia
Funding
Javna Agencija za Raziskovalno Dejavnost RS
References (48)
Show source-provided references
1. Remote plethysmographic imaging using ambient light10.1364/OE.16.021434 ↗
2. Photoplethysmography revisited: From contact to noncontact, from point to imaging10.1109/TBME.2015.2476337 ↗
3. Pulse rate estimation using imaging photoplethysmography: Generic framework and comparison of methods on a publicly available dataset10.1088/2057-1976/aabd09 ↗
4. Heart rate measurement based on a time-lapse image10.1016/j.medengphy.2006.09.006 ↗
5. Living-skin classification via remote-PPG10.1109/TBME.2017.2676160 ↗
6. Unsupervised skin tissue segmentation for remote photoplethysmography10.1016/j.patrec.2017.10.017 ↗
7. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation10.1364/OE.18.010762 ↗
8. Zhao, C., Mei, P., Xu, S., Li, Y., and Feng, Y. (2019, January 27–28). Performance evaluation of visual object detection and tracking algorithms used in remote photoplethysmography. Proceedings of the IEEE International Conference on Computer Vision Workshops, Seoul, Korea.10.1109/ICCVW.2019.00204 ↗
9. Mestha, L.K., Kyal, S., Xu, B., Lewis, L.E., and Kumar, V. (2014, January 26–30). Towards continuous monitoring of pulse rate in neonatal intensive care unit with a webcam. Proceedings of the 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Chicago, IL, USA.10.1109/EMBC.2014.6944455 ↗
10. Kovac, J., Peer, P., and Solina, F. (2003). Human Skin Color Clustering for Face Detection, IEEE.
11. bin Abdul Rahman, N.A., Wei, K.C., and See, J. (2007). RGB-H-CbCr Skin Colour Model for Human Face Detection, Faculty of Information Technology, Multimedia University.
12. A new fast skin color detection technique
13. Asthana, A., Zafeiriou, S., Cheng, S., and Pantic, M. (2013, January 23–28). Robust discriminative response map fitting with constrained local models. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.442 ↗
14. Kazemi, V., and Sullivan, J. (2014, January 23–28). One millisecond face alignment with an ensemble of regression trees. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.241 ↗
15. Conaire, C.O., O’Connor, N.E., and Smeaton, A.F. (2007, January 17–22). Detector adaptation by maximising agreement between independent data sources. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.10.1109/CVPR.2007.383448 ↗
16. Full video pulse extraction10.1364/BOE.9.003898 ↗
17. Single-element remote-PPG10.1109/TBME.2018.2882396 ↗
18. Li, P., Benezeth, Y., Nakamura, K., Gomez, R., Li, C., and Yang, F. (2018, January 29–31). Comparison of region of interest segmentation methods for video-based heart rate measurements. Proceedings of the 2018 IEEE 18th International Conference on Bioinformatics and Bioengineering (BIBE), Taichung, Taiwan.10.1109/BIBE.2018.00034 ↗
19. Optimizing remote photoplethysmography using adaptive skin segmentation for real-time heart rate monitoring10.1109/ACCESS.2019.2922304 ↗
20. Li, P., Benezeth, Y., Nakamura, K., Gomez, R., and Yang, F. (2019, January 25–27). Model-based Region of Interest Segmentation for Remote Photoplethysmography. Proceedings of the 14th International Conference on Computer Vision Theory and Applications, Prague, Czech Republic.10.5220/0007389800002108 ↗
21. Robust pulse rate from chrominance-based rPPG10.1109/TBME.2013.2266196 ↗
22. Skin segmentation based on graph cuts10.1016/S1007-0214(09)70106-3 ↗
23. A fast and accurate unconstrained face detector10.1109/TPAMI.2015.2448075 ↗
24. Bradski, G.R. (1998, January 19–21). Real time face and object tracking as a component of a perceptual user interface. Proceedings of the Fourth IEEE Workshop on Applications of Computer Vision. WACV’98 (Cat. No. 98EX201), Princeton, NJ, USA.
25. Independent component analysis: Algorithms and applications10.1016/S0893-6080(00)00026-5 ↗
26. Viola, P., and Jones, M. (2001, January 8–14). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, Kauai, HI, USA.
27. Lucas, B.D., and Kanade, T. (1981, January 24–28). An iterative image registration technique with an application to stereo vision. Proceedings of the IJCAI’81, 7th International Joint Conference on Artificial Intelligence—Volume 2, Vancouver, BC, Canada.
28. Henriques, J.F., Caseiro, R., Martins, P., and Batista, J. (2012). Exploiting the circulant structure of tracking-by-detection with kernels. European Conference on Computer Vision, Springer.10.1007/978-3-642-33765-9_50 ↗
29. Bertinetto, L., Valmadre, J., Golodetz, S., Miksik, O., and Torr, P.H. (July, January 26). Staple: Complementary learners for real-time tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.
30. Stricker, R., Müller, S., and Gross, H.M. (2014, January 25–29). Non-contact video-based pulse rate measurement on a mobile service robot. Proceedings of the 23rd IEEE International Symposium on Robot and Human Interactive Communication, Edinburgh, UK.10.1109/ROMAN.2014.6926392 ↗
31. Algorithmic principles of remote PPG10.1109/TBME.2016.2609282 ↗
32. Woyczyk, A., Fleischhauer, V., and Zaunseder, S. (2020, January 14–19). Skin Segmentation using Active Contours and Gaussian Mixture Models for Heart Rate Detection in Videos. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.1109/CVPRW50498.2020.00164 ↗
33. Statistical color models with application to skin detection10.1023/A:1013200319198 ↗
34. A novel algorithm for remote photoplethysmography: Spatial subspace rotation10.1109/TBME.2015.2508602 ↗
35. Hoffman, W.F.C., and Lakens, D. (2019). Public Benchmark Dataset for Testing rPPG Algorithm Performance, 4TU.Centre for Research Data.
36. Pilz, C.S., Zaunseder, S., Krajewski, J., and Blazek, V. (2018, January 18–22). Local group invariance for heart rate estimation from face videos in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA.10.1109/CVPRW.2018.00172 ↗
37. Shi, J. (1994, January 21–23). Good features to track. Proceedings of the 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.
38. Li, X., Chen, J., Zhao, G., and Pietikainen, M. (2014, January 24–27). Remote heart rate measurement from face videos under realistic situations. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.543 ↗
39. Rabin, J., Delon, J., Gousseau, Y., and Moisan, L. (2010, January 17–20). MAC-RANSAC: A robust algorithm for the recognition of multiple objects. Proceedings of the Fifth International Symposium on 3D Data Processing, Visualization and Transmission (3DPTV 2010), Paris, France.
40. A new look at the essence of the imaging photoplethysmography10.1038/srep10494 ↗
41. Mironenko, Y., Kalinin, K., Kopeliovich, M., and Petrushan, M. (2020, January 14–19). Remote Photoplethysmography: Rarely Considered Factors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.10.1109/CVPRW50498.2020.00156 ↗
42. Feasibility of assessing ultra-short-term pulse rate variability from video recordings10.7717/peerj.8342 ↗
43. An open-source remote heart rate imaging method with practical apparatus and algorithms10.3758/s13428-019-01256-8 ↗
44. DistancePPG: Robust non-contact vital signs monitoring using a camera10.1364/BOE.6.001565 ↗
45. Exploiting spatial redundancy of image sensor for motion robust rPPG10.1109/TBME.2014.2356291 ↗
46. Speth, J., Vance, N., Flynn, P., Bowyer, K., and Czajka, A. (2021). Remote Pulse Estimation in the Presence of Face Masks. arXiv.10.1109/CVPRW56347.2022.00226 ↗
47. Rubins, U., Miscuks, A., and Lange, M. (2017). Simple and convenient remote photoplethysmography system for monitoring regional anesthesia effectiveness. EMBEC & NBC 2017, Springer.10.1007/978-981-10-5122-7_95 ↗
48. Non-contact imaging of peripheral hemodynamics during cognitive and psychological stressors10.1038/s41598-020-67647-6 ↗