Remote Monitoring of Vital Signs in Diverse Non-Clinical and Clinical Scenarios Using Computer Vision Systems: A Review
Source abstract
Abstract
Techniques for noncontact measurement of vital signs using camera imaging technologies have been attracting increasing attention. For noncontact physiological assessments, computer vision-based methods appear to be an advantageous approach that could be robust, hygienic, reliable, safe, cost effective and suitable for long distance and long-term monitoring. In addition, video techniques allow measurements from multiple individuals opportunistically and simultaneously in groups. This paper aims to explore the progress of the technology from controlled clinical scenarios with fixed monitoring installations and controlled lighting, towards uncontrolled environments, crowds and moving sensor platforms. We focus on the diversity of applications and scenarios being studied in this topic. From this review it emerges that automatic multiple regions of interest (ROIs) selection, removal of noise artefacts caused by both illumination variations and motion artefacts, simultaneous multiple person monitoring, long distance detection, multi-camera fusion and accepted publicly available datasets are topics that still require research to enable the technology to mature into many real-world applications.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Non-Invasive Vital Sign Monitoring · Healthcare Technology and Patient Monitoring · ECG Monitoring and Analysis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Engineering, University of South Australia, Mawson Lakes, SA 5095, Australia
Electrical Engineering Technical College, Middle Technical University, Al Doura, Baghdad 10022, Iraq
Joint and Operations Analysis Division, Defence Science and Technology Group, Melbourne, VIC 3207, Australia
References (100+)
Show source-provided references
1. Ambient and unobtrusive cardiorespiratory monitoring techniques10.1109/RBME.2015.2414661 ↗
2. Video-Based Heart Rate Measurement: Recent Advances and Future Prospects10.1109/TIM.2018.2879706 ↗
3. Zhao, F., Li, M., Qian, Y., and Tsien, J.Z. (2013). Remote measurements of heart and respiration rates for telemedicine. PLoS ONE, 8.10.1371/journal.pone.0071384 ↗
4. Non-contact heart rate and heart rate variability measurements: A review10.1016/j.bspc.2014.03.004 ↗
5. Remote heart rate measurement using low-cost RGB face video: A technical literature review10.1007/s11704-016-6243-6 ↗
6. A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems10.1136/jamia.2010.005264 ↗
7. Photoplethysmography revisited: From contact to noncontact, from point to imaging10.1109/TBME.2015.2476337 ↗
8. Extraction of respiratory signals from the electrocardiogram and photoplethysmogram: Technical and physiological determinants10.1088/1361-6579/aa670e ↗
9. Remote sensing of physiological signs using a machine vision system10.1080/03091902.2017.1313326 ↗
10. Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit—A pilot study10.1016/j.earlhumdev.2013.09.016 ↗
11. Unobtrusive acquisition of cardiorespiratory signals10.1007/s11818-017-0112-x ↗
12. Monitoring of cardiorespiratory signal: Principles of remote measurements and review of methods10.1109/ACCESS.2017.2735419 ↗
13. Scalise, L. (2012). Non contact heart monitoring. Advances in Electrocardiograms-Methods and Analysis, IntechOpen.10.5772/22937 ↗
14. Electrodeless measurements of the effective resistivity of the human torso and head by magnetic induction10.1109/TBME.1968.4502577 ↗
15. Guardo, R., Trudelle, S., Adler, A., Boulay, C., and Savard, P. (1995, January 20–23). Contactless recording of cardiac related thoracic conductivity changes. Proceedings of the 1995 IEEE 17th International Conference of the Engineering in Medicine and Biology Society, Montreal, QC, Canada.
16. Richer, A., and Adler, A. (2005, January 16–19). Eddy current based flexible sensor for contactless measurement of breathing. Proceedings of the 2005 IEEE Instrumentation and Measurement Technology Conference Proceedings, Ottawa, ON, Canada.
17. Mobile noncontact monitoring of heart and lung activity10.1109/TBCAS.2008.915633 ↗
18. Vetter, P., Leicht, L., Leonhardt, S., and Teichmann, D. (2017, January 27–28). Integration of an electromagnetic coupled sensor into a driver seat for vital sign monitoring: Initial insight. Proceedings of the 2017 IEEE International Conference on Vehicular Electronics and Safety (ICVES), Vienna, Austria.10.1109/ICVES.2017.7991923 ↗
19. Remote sensing of vital sign of human body with radio frequency10.1007/s40012-016-0154-4 ↗
20. Ultra-wide patch antenna array design at 60 GHz band for remote vital sign monitoring with Doppler radar principle10.1007/s10762-016-0344-z ↗
21. Scalise, L., Marchionni, P., and Ercoli, I. (May, January 30). Optical method for measurement of respiration rate. Proceedings of the 2010 IEEE International Workshop on Medical Measurements and Applications, Ottawa, ON, Canada.10.1109/MEMEA.2010.5480208 ↗
22. A digital signal processor for Doppler radar sensing of vital signs10.1109/MEMB.2002.1044188 ↗
23. Frequency-Tracking CW doppler radar solving small-angle approximation and null point issues in non-contact vital signs monitoring10.1109/TBCAS.2016.2647560 ↗
24. A Concurrent Dual-Beam Phased-Array Doppler Radar Using MIMO Beamforming Techniques for Short-Range Vital-Signs Monitoring10.1109/TAP.2019.2893337 ↗
25. An optical measurement method for the simultaneous assessment of respiration and heart rates in preterm infants10.1063/1.4845635 ↗
26. Cardiorespiratory interactions: Noncontact assessment using laser Doppler vibrometry10.1111/psyp.12638 ↗
27. Characterization of common carotid artery blood-flow waveforms in normal human subjects10.1088/0967-3334/20/3/301 ↗
28. An ultrasonic contactless sensor for breathing monitoring10.3390/s140815371 ↗
29. A study on a non-contacting respiration signal monitoring system using Doppler ultrasound
30. Yang, M., Liu, Q., Turner, T., and Wu, Y. (2008, January 23–28). Vital sign estimation from passive thermal video. Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, AK, USA.
31. Adaptive Eulerian Video Processing of Thermal Video: An Experimental Analysis10.1109/TIM.2017.2684518 ↗
32. Blind signal separation: Statistical principles10.1109/5.720250 ↗
33. Exploratory study to evaluate respiratory rate using a thermal imaging camera10.1159/000490546 ↗
34. Luca, C., Corciovă, C., Andriţoi, D., and Ciorap, R. (2018, January 17–20). The Use of Thermal Imaging Techniques as a Method of Monitoring the New Born. Proceedings of the 6th International Conference on Advancements of Medicine and Health Care through Technology, Cluj-Napoca, Romania.
35. Neonatal non-contact respiratory monitoring based on real-time infrared thermography10.1186/1475-925X-10-93 ↗
36. Hybrid optical imaging technology for long-term remote monitoring of skin perfusion and temperature behavior10.1117/1.JBO.19.1.016012 ↗
37. Contact-Free respiration rate monitoring using a pan–tilt thermal camera for stationary bike telerehabilitation sessions10.1109/JSYST.2014.2336372 ↗
38. Remote monitoring of breathing dynamics using infrared thermography10.1364/BOE.6.004378 ↗
39. Contact-Free measurement of cardiac pulse based on the analysis of thermal imagery10.1109/TBME.2007.891930 ↗
40. Interacting with human physiology10.1016/j.cviu.2006.11.018 ↗
41. Sun, N., Garbey, M., Merla, A., and Pavlidis, I. (2005, January 20–25). Imaging the cardiovascular pulse. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA.
42. Chekmenev, S.Y., Farag, A.A., Miller, W.M., Essock, E.A., and Bhatnagar, A. (2009). Multiresolution approach for noncontact measurements of arterial pulse using thermal imaging. Augmented Vision Perception in Infrared, Springer Science and Business Media LLC.10.1007/978-1-84800-277-7_4 ↗
43. Abbas, A.K., Heiman, K., Orlikowsky, T., and Leonhardt, S. (2009, January 7–12). Non-Contact Respiratory Monitoring Based on Real-Time IR-Thermography. Proceedings of the World Congress on Medical Physics and Biomedical Engineering, Munich, Germany.10.1007/978-3-642-03882-2_346 ↗
44. Real-time computer vision with OpenCV10.1145/2184319.2184337 ↗
45. McDuff, D.J., Estepp, J.R., Piasecki, A.M., and Blackford, E.B. (2015, January 25–29). A survey of remote optical photoplethysmographic imaging methods. Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy.10.1109/EMBC.2015.7319857 ↗
46. Computer-Vision-Guided human pulse rate estimation: A Review10.1109/RBME.2016.2551778 ↗
47. Heart rate estimation using facial video: A review10.1016/j.bspc.2017.07.004 ↗
48. Cardiovascular assessment by imaging photoplethysmography—A review10.1515/bmt-2017-0119 ↗
49. Nakajima, K., Osa, A., and Miike, H. (November, January 30). A method for measuring respiration and physical activity in bed by optical flow analysis. Proceedings of the 19th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. ‘Magnificent Milestones and Emerging Opportunities in Medical Engineering’ (Cat. No. 97CH36136), Chicago, IL, USA.
50. Frigola, M., Amat, J., and Pagès, J. (2002, January 9–12). Vision based respiratory monitoring system. Proceedings of the 10th Mediterranean Conference on Control and Automation (MED 2002), Lisbon, Portugal.
51. Balakrishnan, G., Durand, F., and Guttag, J. (2013, January 23–28). Detecting pulse from head motions in video. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.10.1109/CVPR.2013.440 ↗
52. Viola, P., and Jones, M. (2001, January 8–14). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, USA.
53. Lucas, B.D., and Kanade, T. (1981, January 24–28). An Iterative Image Registration Technique with an Application to Stereo Vision. Proceedings of the 7th International Joint Conference on Artificial Intelligence, Vancouver, BC, Canada.
54. Shan, L., and Yu, M. (2013, January 16–18). Video-based heart rate measurement using head motion tracking and ICA. Proceedings of the 2013 6th International Congress on Image and Signal Processing (CISP), Hangzhou, China.10.1109/CISP.2013.6743978 ↗
55. Irani, R., Nasrollahi, K., and Moeslund, T.B. (2014, January 5–8). Improved pulse detection from head motions using DCT. Proceedings of the 2014 International Conference on Computer Vision Theory and Applications (VISAPP), Lisbon, Portugal.
56. Heartbeat rate measurement from facial video10.1109/MIS.2016.20 ↗
57. Lomaliza, J.-P., and Park, H. (2016, January 8–10). Detecting Pulse from Head Motions Using Smartphone Camera. Proceedings of the International Conference on Advanced Engineering Theory and Applications, Busan, Vietnnam.10.1007/978-3-319-50904-4_24 ↗
58. Lomaliza, J.-P., and Park, H. (2019). Improved Heart-Rate Measurement from Mobile Face Videos. Electronics, 8.10.3390/electronics8060663 ↗
59. He, X., Goubran, R.A., and Liu, X.P. (2016, January 24–27). Wrist pulse measurement and analysis using Eulerian video magnification. Proceedings of the 2016 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Las Vegas, NV, USA.10.1109/BHI.2016.7455830 ↗
60. Eulerian video magnification for revealing subtle changes in the world10.1145/2185520.2185561 ↗
61. Contactless cardiac activity detection based on head motion magnification10.1142/S0219467817500012 ↗
62. Remote respiratory monitoring system based on developing motion magnification technique10.1016/j.bspc.2016.05.002 ↗
63. Al-Naji, A., Gibson, K., Lee, S.-H., and Chahl, J. (2017). Real time apnoea monitoring of children using the microsoft kinect sensor: A pilot study. Sensors, 17.10.3390/s17020286 ↗
64. Al-Naji, A., and Chahl, J. (2018). Detection of cardiopulmonary activity and related abnormal events using microsoft kinect sensor. Sensors, 18.10.3390/s18030920 ↗
65. An efficient motion magnification system for real-time applications10.1007/s00138-018-0916-0 ↗
66. Observations on the finger volume pulse recorded photoelectrically
67. Algorithmic principles of remote PPG10.1109/TBME.2016.2609282 ↗
68. Remote plethysmographic imaging using ambient light10.1364/OE.16.021434 ↗
69. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation10.1364/OE.18.010762 ↗
70. Advancements in noncontact, multiparameter physiological measurements using a webcam10.1109/TBME.2010.2086456 ↗
71. Pursche, T., Krajewski, J., and Moeller, R. (2012, January 13–16). Video-based heart rate measurement from human faces. Proceedings of the 2012 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA.10.1109/ICCE.2012.6161965 ↗
72. Lewandowska, M., Rumiński, J., Kocejko, T., and Nowak, J. (2011, January 18–21). Measuring pulse rate with a webcam—A non-contact method for evaluating cardiac activity. Proceedings of the 2011 Federated Conference on Computer Science and Information Systems (FedCSIS), Szczecin, Poland.
73. Independent component analysis: Algorithms and applications10.1016/S0893-6080(00)00026-5 ↗
74. Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise10.1117/1.3602852 ↗
75. Feng, L., Po, L.-M., Xu, X., and Li, Y. (2014, January 20–23). Motion artifacts suppression for remote imaging photoplethysmography. Proceedings of the 2014 19th International Conference on Digital Signal Processing, Hong Kong, China.10.1109/ICDSP.2014.6900813 ↗
76. Video-based human heart rate measurement using joint blind source separation10.1016/j.bspc.2016.08.020 ↗
77. Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate10.1016/j.bspc.2013.05.010 ↗
78. Robust pulse rate from chrominance-based rPPG10.1109/TBME.2013.2266196 ↗
79. Improved motion robustness of remote-PPG by using the blood volume pulse signature10.1088/0967-3334/35/9/1913 ↗
80. Motion-resistant remote imaging photoplethysmography based on the optical properties of skin10.1109/TCSVT.2014.2364415 ↗
81. A novel algorithm for remote photoplethysmography: Spatial subspace rotation10.1109/TBME.2015.2508602 ↗
82. Robust heart rate from fitness videos10.1088/1361-6579/aa6d02 ↗
83. Wu, B.-F., Huang, P.-W., Tsou, T.-Y., Lin, T.-M., and Chung, M.-L. (2017, January 21–23). Camera-based heart rate measurement using continuous wavelet transform. Proceedings of the 2017 International Conference on System Science and Engineering (ICSSE), Ho Chi Minh City, Vietnam.10.1109/ICSSE.2017.8030826 ↗
84. Motion resistant image-photoplethysmography based on spectral peak tracking algorithm10.1109/ACCESS.2018.2828133 ↗
85. Xie, K., Fu, C.-H., Liang, H., Hong, H., and Zhu, X. (2019, January 28–30). Non-contact Heart Rate Monitoring for Intensive Exercise Based on Singular Spectrum Analysis. Proceedings of the 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), San Jose, CA, USA.10.1109/MIPR.2019.00048 ↗
86. McDuff, D.J., Blackford, E.B., Estepp, J.R., and Nishidate, I. (2018). A Fast Non-Contact Imaging Photoplethysmography Method Using a Tissue-Like Model. Optical Diagnostics and Sensing XVIII: Toward Point-of-Care Diagnostics, International Society for Optics and Photonics.10.1117/12.2285937 ↗
87. Fallet, S., Schoenenberger, Y., Martin, L., Braun, F., Moser, V., and Vesin, J.-M. (2017, January 24–27). Imaging photoplethysmography: A real-time signal quality index. Proceedings of the 2017 Computing in Cardiology (CinC), Rennes, France.10.22489/CinC.2017.142-139 ↗
88. Motion robust PPG-imaging through color channel mapping10.1364/BOE.7.001737 ↗
89. Image sensor-based heart rate evaluation from face reflectance using Hilbert–Huang transform10.1109/JSEN.2014.2347397 ↗
90. Face-based heart rate signal decomposition and evaluation using multiple linear regression10.1109/JSEN.2015.2500032 ↗
91. Lee, D., Kim, J., Kwon, S., and Park, K. (2015, January 25–29). Heart rate estimation from facial photoplethysmography during dynamic illuminance changes. Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy.10.1109/EMBC.2015.7318963 ↗
92. Non-contact video-based vital sign monitoring using ambient light and auto-regressive models10.1088/0967-3334/35/5/807 ↗
93. Illumination variation-resistant video-based heart rate measurement using joint blind source separation and ensemble empirical mode decomposition10.1109/JBHI.2016.2615472 ↗
94. Illumination variation interference suppression in remote PPG using PLS and MEMD10.1049/el.2016.3611 ↗
95. Li, X., Chen, J., Zhao, G., and Pietikainen, M. (2014, January 23–28). Remote heart rate measurement from face videos under realistic situations. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.10.1109/CVPR.2014.543 ↗
96. DistancePPG: Robust non-contact vital signs monitoring using a camera10.1364/BOE.6.001565 ↗
97. Remote monitoring of cardiorespiratory signals from a hovering unmanned aerial vehicle10.1186/s12938-017-0395-y ↗
98. Quality index evaluation of videos based on fuzzy interface system10.1049/iet-ipr.2016.0569 ↗
99. Chen, J.-H., Tang, I.-L., and Chang, C.-H. (2015, January 20–22). Enhancing the detection rate of inclined faces. Proceedings of the 2015 IEEE Trustcom/BigDataSE/ISPA, Helsinki, Finland.10.1109/Trustcom.2015.573 ↗
100. Borga, M., and Knutsson, H. (2001). A Canonical Correlation Approach to Blind Source Separation, Department of Biomedical Engineering, Linkping University. Report LiU-IMT-EX-0062.