Contactless Vital Sign Monitoring System for Heart and Respiratory Rate Measurements with Motion Compensation Using a Near-Infrared Time-of-Flight Camera
Source abstract
Abstract
This study describes a contactless vital sign monitoring (CVSM) system capable of measuring heart rate (HR) and respiration rate (RR) using a low-power, indirect time-of-flight (ToF) camera. The system takes advantage of both the active infrared illumination as well as the additional depth information from the ToF camera to compensate for the motion-induced artifacts during the HR measurements. The depth information captures how the user is moving with respect to the camera and, therefore, can be used to differentiate where the intensity change in the raw signal is from the underlying heartbeat or motion. Moreover, from the depth information, the system can acquire respiration rate by directly measuring the motion of the chest wall during breathing. We also conducted a pilot human study using this system with 29 participants of different demographics such as age, gender, and skin color. Our study shows that with depth-based motion compensation, the success rate (system measurement within 10% of reference) of HR measurements increases to 75%, as compared to 35% when motion compensation is not used. The mean HR deviation from the reference also drops from 21 BPM to −6.25 BPM when we apply the depth-based motion compensation. In terms of the RR measurement, our system shows a mean deviation of 1.7 BPM from the reference measurement. The pilot human study shows the system performance is independent of skin color but weakly dependent on gender and age.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Non-Invasive Vital Sign Monitoring · Hemodynamic Monitoring and Therapy · Healthcare Technology and Patient Monitoring
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Electrical Engineering, University of Michigan, Ann Arbor, MI 48109, USA
School of Medicine, Case Western Reserve University, Cleveland, OH 44106, USA
Department of Computer Science and Engineering, University of California, La Jolla, San Diego, CA 92093, USA
Omni Science Inc., Dexter, MI 48130, USA
References (41)
Show source-provided references
1. Remote plethysmographic imaging using ambient light10.1364/OE.16.021434 ↗
2. A heart rate monitoring framework for real-world drivers using remote photoplethysmography10.1109/JBHI.2020.3026481 ↗
3. Balakrishnan, G., Dur, F., and Guttag, J. (2013, January 23–28). Detecting pulse from head motions in video. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.440 ↗
4. Illumination variation-resistant video-based heart rate measurement using joint blind source separation and ensemble empirical mode decomposition10.1109/JBHI.2016.2615472 ↗
5. Negishi, T., Abe, S., Matsui, T., Liu, H., Kurosawa, M., Kirimoto, T., and Sun, G. (2020). Contactless vital signs measurement system using RGB-thermal image sensors and its clinical screening test on patients with seasonal influenza. Sensors, 20.10.3390/s20082171 ↗
6. DistancePPG: Robust non-contact vital signs monitoring using a camera10.1364/BOE.6.001565 ↗
7. Remote sensing of multiple vital signs using a CMOS camera-equipped infrared thermography system and its clinical application in rapidly screening patients with suspected infectious diseases10.1016/j.ijid.2017.01.007 ↗
8. Development of a Novel Web Camera-Based Contact-Free Major Depressive Disorder Screening System Using Autonomic Nervous Responses Induced by a Mental Task and Its Clinical Application10.3389/fphys.2021.642986 ↗
9. Algorithmic principles of remote PPG10.1109/TBME.2016.2609282 ↗
10. Exploiting spatial redundancy of image sensor for motion robust rPPG10.1109/TBME.2014.2356291 ↗
11. Robust pulse rate from chrominance-based rPPG10.1109/TBME.2013.2266196 ↗
12. A novel algorithm for remote photoplethysmography: Spatial subspace rotation10.1109/TBME.2015.2508602 ↗
13. Improved motion robustness of remote-PPG by using the blood volume pulse signature10.1088/0967-3334/35/9/1913 ↗
14. Motion resistant image-photoplethysmography based on spectral peak tracking algorithm10.1109/ACCESS.2018.2828133 ↗
15. Motion-resistant remote imaging photoplethysmography based on the optical properties of skin10.1109/TCSVT.2014.2364415 ↗
16. A Review of Deep Learning-Based Contactless Heart Rate Measurement Methods10.3390/s21113719 ↗
17. Cardiovascular assessment by imaging photoplethysmography—A review10.1515/bmt-2017-0119 ↗
18. Melanin absorption spectroscopy: New method for noninvasive skin investigation and melanoma detection10.1117/1.2844710 ↗
19. Motion robust remote-PPG in infrared10.1109/TBME.2015.2390261 ↗
20. Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise10.1117/1.3602852 ↗
21. Magdalena, N.E., Marks, T.K., Mansour, H., and Veeraraghavan, A. (2018, January 18–22). SparsePPG: Towards driver monitoring using camera-based vital signs estimation in near-infrared. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA.
22. Extraction of heart rate from functional near-infrared spectroscopy in infants10.1117/1.JBO.19.6.067010 ↗
23. Remote vital parameter monitoring in neonatology–robust, unobtrusive heart rate detection in a realistic clinical scenario
24. Nahler, C., Feldhofer, B., Ruether, M., Holweg, G., and Druml, N. (2018, January 29–31). Exploring the usage of Time-of-Flight Cameras for contact and remote Photoplethysmography. Proceedings of the 2018 21st Euromicro Conference on Digital System Design (DSD), Prague, Czech Republic.10.1109/DSD.2018.00079 ↗
25. Non-contact heart rate and heart rate variability measurements: A review10.1016/j.bspc.2014.03.004 ↗
26. Robust respiration detection from remote photoplethysmography10.1364/BOE.7.004941 ↗
27. An assessment of algorithms to estimate respiratory rate from the electrocardiogram and photoplethysmogram10.1088/0967-3334/37/4/610 ↗
28. Bernacchia, N., Scalise, L., Casacanditella, L., Ercoli, I., Marchionni, P., and Tomasini, E.P. (2014, January 11–12). Non contact measurement of heart and respiration rates based on Kinect™. Proceedings of the 2014 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Lisboa, Portugal.10.1109/MeMeA.2014.6860065 ↗
29. Procházka, A., Schätz, M., Vyšata, O., and Vališ, M. (2016). Microsoft kinect visual and depth sensors for breathing and heart rate analysis. Sensors, 16.10.3390/s16070996 ↗
30. Zanuttigh, P., Marin, G., Dal Mutto, C., Dominio, F., Minto, L., and Cortelazzo, G.M. (2016). Time-of-Flight and Structured Light Depth Cameras, Springer. Technology and Applications.10.1007/978-3-319-30973-6 ↗
31. Martinez, L.F., Paez, G., and Strojnik, M. (2011, January 15–19). Optimal wavelength selection for noncontact reflection photoplethysmography. Proceedings of the 22nd Congress of the International Commission for Optics: Light for the Development of the World, Puebla, Mexico.
32. New insights into the origin of remote PPG signals in visible light and infrared
33. Asthana, A., Zafeiriou, S., Cheng, S., and Pantic, M. (2014, January 23–28). Incremental face alignment in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.240 ↗
34. Zhu, X., and Ramanan, D. (2012, January 16–21). Face detection, pose estimation, and landmark localization in the wild. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.
35. So-Ling, C., and Li, L. (2001, January 6). A multi-layered reflection model of natural human skin. Proceedings of the Computer Graphics International 2001, Hong Kong, China.
36. Analysis of human faces using a measurement-based skin reflectance model10.1145/1141911.1141987 ↗
37. Shi, J. (1994, January 21–23). Good features to track. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.
38. The validity and practicality of sun-reactive skin types I through VI10.1001/archderm.1988.01670060015008 ↗
39. Keel, M.S., Kim, D., Kim, Y., Bae, M., Ki, M., Chung, B., Son, S., Lee, H., Jo, H., and Shin, S.C. (2021, January 13–22). A 4-tap 3.5 μm 1.2 Mpixel Indirect Time-of-Flight CMOS Image Sensor with Peak Current Mitigation and Multi-User Interference Cancellation. Proceedings of the 2021 IEEE International Solid-State Circuits Conference (ISSCC), San Francisco, CA, USA.10.1109/ISSCC42613.2021.9365854 ↗
40. Wang, J., Warnecke, J.M., Haghi, M., and Deserno, T.M. (2020). Unobtrusive health monitoring in private spaces: The smart vehicle. Sensors, 20.10.3390/s20092442 ↗
41. Jiang, C., Yabroff, K.R., Deng, L., Perimbeti, S., and Han, X. (2021). Prevalence of Underlying Medical Conditions Associated with Severe COVID-19 Illness in Adult Cancer Survivors in the United States. JNCI J. Natl. Cancer Inst., djab012.10.1093/jnci/djab012 ↗