Abstract

Photoplethysmographic imaging is an optical solution for non-contact cardiovascular monitoring from a distance. This camera-based technology enables physiological monitoring in situations where contact-based devices may be problematic or infeasible, such as ambulatory, sleep, and multi-individual monitoring. However, automatically extracting the blood pulse waveform signal is challenging due to the unknown mixture of relevant (pulsatile) and irrelevant pixels in the scene. Here, we propose a signal fusion framework, FusionPPG, for extracting a blood pulse waveform signal with strong temporal fidelity from a scene without requiring anatomical priors. The extraction problem is posed as a Bayesian least squares fusion problem, and solved using a novel probabilistic pulsatility model that incorporates both physiologically derived spectral and spatial waveform priors to identify pulsatility characteristics in the scene. Evaluation was performed on a 24-participant sample with various ages (9–60 years) and body compositions (fat% 30.0 ± 7.9, muscle% 40.4 ± 5.3, BMI 25.5 ± 5.2 kg·m−2). Experimental results show stronger matching to the ground-truth blood pulse waveform signal compared to the FaceMeanPPG (p < 0.001) and DistancePPG (p < 0.001) methods. Heart rates predicted using FusionPPG correlated strongly with ground truth measurements (r2 = 0.9952). A cardiac arrhythmia was visually identified in FusionPPG’s waveform via temporal analysis.

© 2016 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Comparison of speckleplethysmographic (SPG) and photoplethysmographic (PPG) imaging by Monte Carlo simulations and in vivo measurements

Cody E. Dunn, Ben Lertsakdadet, Christian Crouzet, Adrian Bahani, and Bernard Choi
Biomed. Opt. Express 9(9) 4306-4316 (2018)

DistancePPG: Robust non-contact vital signs monitoring using a camera

Mayank Kumar, Ashok Veeraraghavan, and Ashutosh Sabharwal
Biomed. Opt. Express 6(5) 1565-1588 (2015)

Robust respiration detection from remote photoplethysmography

Mark van Gastel, Sander Stuijk, and Gerard de Haan
Biomed. Opt. Express 7(12) 4941-4957 (2016)

References

  • View by:
  • |
  • |
  • |

  1. J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiol. Meas. 28(3), R1–R39 (2007).
    [Crossref] [PubMed]
  2. Y. Sun and N. Thakor, “Photoplethysmography revisited: from contact to noncontact, from point to imaging,” IEEE Trans. Biomed. Eng. 63(3), 463–477 (2016).
    [Crossref]
  3. J. Allen and K. Howell, “Microvascular imaging: techniques and opportunities for clinical physiological measurements,” Physiol. Meas. 35(7), R91 (2014).
    [Crossref] [PubMed]
  4. Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16(7), 077010 (2011).
    [Crossref] [PubMed]
  5. Y. Sun, S. Hu, V. Azorin-Peris, R. Kalawsky, and S. Greenwald, “Noncontact imaging photoplethysmography to effectively access pulse rate variability,” J. Biomed. Opt. 18(6), 061205 (2013).
    [Crossref]
  6. G. Gibert, D. D’Alessandro, and F. Lance, “Face detection method based on photoplethysmography,” in Proceedings of Adv. Video and Signal Based Surveillance (IEEE, 2013), pp. 449–453.
  7. R. van Luijtelaar, W. Wang, S. Stuijk, and G. de Haan, “Automatic ROI Detection for camera-based pulse-rate measurement,” in Proceedings of Asian Conf. Comput. Vision (Springer, 2014), pp. 360–374.
  8. W. Wang, S. Stuijk, and G. de Haan, “Unsupervised subject detection via remote PPG,” IEEE Trans. Biomed. Eng. 62(11), 2629–2637 (2015).
    [Crossref] [PubMed]
  9. E. Calvo-Gallego and G. de Haan, “Automatic ROI for remote photoplethysmography using PPG and color features,” in Proceedings of Int. Conf. Comput. Vision Theory Appl. (SCITEPRESS, 2015), pp. 357–364.
  10. M.-Z. Poh, D. J. McDuff, and R. W. Picard, “Non-contact, automated cardiac pulse measurements using video imaging and blind source separation,” Opt. Express 18(10), 10762–10774 (2010).
    [Crossref] [PubMed]
  11. A. A. Kamshilin, S. Miridonov, V. Teplov, R. Saarenheimo, and E. Nippolainen, “Photoplethysmographic imaging of high spatial resolution,” Biomed. Opt. Express 2(4), 996–1006 (2011).
    [Crossref] [PubMed]
  12. A. A. Kamshilin, E. Nippolainen, I. S. Sidorov, P. V. Vasilev, N. P. Erofeev, N. P. Podolian, and R. V. Romashko, “A new look at the essence of the imaging photoplethysmography,” Sci. Rep. 5, 10494 (2015).
    [Crossref] [PubMed]
  13. D. McDuff, S. Gontarek, and R. W. Picard, “Improvements in remote cardiopulmonary measurement using a five band digital camera,” IEEE Trans. Biomed. Eng. 61(10), 2593–2601 (2014).
    [Crossref] [PubMed]
  14. S. Xu, L. Sun, and G. K. Rohde, “Robust efficient estimation of heart rate pulse from video,” Biomed. Opt. Express 5(4), 1124–1135 (2014).
    [Crossref] [PubMed]
  15. W. Wang, S. Stuijk, and G. De Haan, “Exploiting spatial redundancy of image sensor for motion robust rPPG,” IEEE Trans. Biomed. Eng. 62(2), 415–425 (2015).
    [Crossref]
  16. R. R. Anderson and J. A. Parrish, “The optics of human skin,” J. Invest. Dermatol. 77(1), 13–19 (1981).
    [Crossref] [PubMed]
  17. M. Kumar, A. Veeraraghavan, and A. Sabharwal, “DistancePPG: Robust non-contact vital signs monitoring using a camera,” Biomed. Opt. Express 6(5), 1565–1588 (2015).
    [Crossref] [PubMed]
  18. D. McDuff, S. Gontarek, and R. W. Picard, “Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera,” IEEE Trans. Biomed. Eng. 61(12), 2948–2954 (2014).
    [Crossref] [PubMed]
  19. M. P. Tarvainen, P. O. Ranta-aho, and P. A. Karjalainen, “An advanced detrending method with application to HRV analysis,” IEEE Trans. Biomed. Eng. 49(2), 172–175 (2002).
    [Crossref] [PubMed]
  20. P. Fieguth, Statistical Image Processing and Multidimensional Modeling (Springer, 2010).
  21. A. Wong, A. Mishra, W. Zhang, P. Fieguth, and D. A. Clausi, “Stochastic image denoising based on Markov-chain Monte Carlo sampling,” Signal Process. 91(8), 2112–2120 (2011).
    [Crossref]
  22. R-B, “Easy Pulse sensor (version 1.1) overview (part 1),” http://embedded-lab.com/blog/?p=7336 ). (12Dec2014).
  23. G. de Haan and A. Van Leest, “Improved motion robustness of remote-ppg by using the blood volume pulse signature,” Physiol. Meas. 35(9), 1913 (2014).
    [Crossref] [PubMed]
  24. J. Wander and D. Morris, “A combined segmenting and non-segmenting approach to signal quality estimation for ambulatory photoplethysmography,” Physiol. Meas. 35(12), 2543–2561 (2014).
    [Crossref] [PubMed]
  25. B. P. Imholz, W. Wieling, G. A. van Montfrans, and K. H. Wesseling, “Fifteen years experience with finger arterial pressure monitoring,” Cardiovasc. Res. 38(3), 605–616 (1998).
    [Crossref] [PubMed]
  26. A. Schäfer and J. Vagedes, “How accurate is pulse rate variability as an estimate of heart rate variability?: A review on studies comparing photoplethysmographic technology with an electrocardiogram,” Int. J. Cardiol. 166(1), 15–29 (2013).
    [Crossref]
  27. L. V. Wang and H.-i. Wu, Biomedical Optics: Principles and Imaging (John Wiley & Sons, 2012).
  28. S. L. Jacques, “Optical properties of biological tissues: a review,” Physics in Med. and Biol. 58(11), R37–R61 (2013).
    [Crossref]

2016 (1)

Y. Sun and N. Thakor, “Photoplethysmography revisited: from contact to noncontact, from point to imaging,” IEEE Trans. Biomed. Eng. 63(3), 463–477 (2016).
[Crossref]

2015 (4)

W. Wang, S. Stuijk, and G. de Haan, “Unsupervised subject detection via remote PPG,” IEEE Trans. Biomed. Eng. 62(11), 2629–2637 (2015).
[Crossref] [PubMed]

A. A. Kamshilin, E. Nippolainen, I. S. Sidorov, P. V. Vasilev, N. P. Erofeev, N. P. Podolian, and R. V. Romashko, “A new look at the essence of the imaging photoplethysmography,” Sci. Rep. 5, 10494 (2015).
[Crossref] [PubMed]

W. Wang, S. Stuijk, and G. De Haan, “Exploiting spatial redundancy of image sensor for motion robust rPPG,” IEEE Trans. Biomed. Eng. 62(2), 415–425 (2015).
[Crossref]

M. Kumar, A. Veeraraghavan, and A. Sabharwal, “DistancePPG: Robust non-contact vital signs monitoring using a camera,” Biomed. Opt. Express 6(5), 1565–1588 (2015).
[Crossref] [PubMed]

2014 (6)

D. McDuff, S. Gontarek, and R. W. Picard, “Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera,” IEEE Trans. Biomed. Eng. 61(12), 2948–2954 (2014).
[Crossref] [PubMed]

G. de Haan and A. Van Leest, “Improved motion robustness of remote-ppg by using the blood volume pulse signature,” Physiol. Meas. 35(9), 1913 (2014).
[Crossref] [PubMed]

J. Wander and D. Morris, “A combined segmenting and non-segmenting approach to signal quality estimation for ambulatory photoplethysmography,” Physiol. Meas. 35(12), 2543–2561 (2014).
[Crossref] [PubMed]

D. McDuff, S. Gontarek, and R. W. Picard, “Improvements in remote cardiopulmonary measurement using a five band digital camera,” IEEE Trans. Biomed. Eng. 61(10), 2593–2601 (2014).
[Crossref] [PubMed]

S. Xu, L. Sun, and G. K. Rohde, “Robust efficient estimation of heart rate pulse from video,” Biomed. Opt. Express 5(4), 1124–1135 (2014).
[Crossref] [PubMed]

J. Allen and K. Howell, “Microvascular imaging: techniques and opportunities for clinical physiological measurements,” Physiol. Meas. 35(7), R91 (2014).
[Crossref] [PubMed]

2013 (3)

Y. Sun, S. Hu, V. Azorin-Peris, R. Kalawsky, and S. Greenwald, “Noncontact imaging photoplethysmography to effectively access pulse rate variability,” J. Biomed. Opt. 18(6), 061205 (2013).
[Crossref]

A. Schäfer and J. Vagedes, “How accurate is pulse rate variability as an estimate of heart rate variability?: A review on studies comparing photoplethysmographic technology with an electrocardiogram,” Int. J. Cardiol. 166(1), 15–29 (2013).
[Crossref]

S. L. Jacques, “Optical properties of biological tissues: a review,” Physics in Med. and Biol. 58(11), R37–R61 (2013).
[Crossref]

2011 (3)

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16(7), 077010 (2011).
[Crossref] [PubMed]

A. A. Kamshilin, S. Miridonov, V. Teplov, R. Saarenheimo, and E. Nippolainen, “Photoplethysmographic imaging of high spatial resolution,” Biomed. Opt. Express 2(4), 996–1006 (2011).
[Crossref] [PubMed]

A. Wong, A. Mishra, W. Zhang, P. Fieguth, and D. A. Clausi, “Stochastic image denoising based on Markov-chain Monte Carlo sampling,” Signal Process. 91(8), 2112–2120 (2011).
[Crossref]

2010 (1)

2007 (1)

J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiol. Meas. 28(3), R1–R39 (2007).
[Crossref] [PubMed]

2002 (1)

M. P. Tarvainen, P. O. Ranta-aho, and P. A. Karjalainen, “An advanced detrending method with application to HRV analysis,” IEEE Trans. Biomed. Eng. 49(2), 172–175 (2002).
[Crossref] [PubMed]

1998 (1)

B. P. Imholz, W. Wieling, G. A. van Montfrans, and K. H. Wesseling, “Fifteen years experience with finger arterial pressure monitoring,” Cardiovasc. Res. 38(3), 605–616 (1998).
[Crossref] [PubMed]

1981 (1)

R. R. Anderson and J. A. Parrish, “The optics of human skin,” J. Invest. Dermatol. 77(1), 13–19 (1981).
[Crossref] [PubMed]

Allen, J.

J. Allen and K. Howell, “Microvascular imaging: techniques and opportunities for clinical physiological measurements,” Physiol. Meas. 35(7), R91 (2014).
[Crossref] [PubMed]

J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiol. Meas. 28(3), R1–R39 (2007).
[Crossref] [PubMed]

Anderson, R. R.

R. R. Anderson and J. A. Parrish, “The optics of human skin,” J. Invest. Dermatol. 77(1), 13–19 (1981).
[Crossref] [PubMed]

Azorin-Peris, V.

Y. Sun, S. Hu, V. Azorin-Peris, R. Kalawsky, and S. Greenwald, “Noncontact imaging photoplethysmography to effectively access pulse rate variability,” J. Biomed. Opt. 18(6), 061205 (2013).
[Crossref]

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16(7), 077010 (2011).
[Crossref] [PubMed]

Calvo-Gallego, E.

E. Calvo-Gallego and G. de Haan, “Automatic ROI for remote photoplethysmography using PPG and color features,” in Proceedings of Int. Conf. Comput. Vision Theory Appl. (SCITEPRESS, 2015), pp. 357–364.

Chambers, J.

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16(7), 077010 (2011).
[Crossref] [PubMed]

Clausi, D. A.

A. Wong, A. Mishra, W. Zhang, P. Fieguth, and D. A. Clausi, “Stochastic image denoising based on Markov-chain Monte Carlo sampling,” Signal Process. 91(8), 2112–2120 (2011).
[Crossref]

D’Alessandro, D.

G. Gibert, D. D’Alessandro, and F. Lance, “Face detection method based on photoplethysmography,” in Proceedings of Adv. Video and Signal Based Surveillance (IEEE, 2013), pp. 449–453.

de Haan, G.

W. Wang, S. Stuijk, and G. de Haan, “Unsupervised subject detection via remote PPG,” IEEE Trans. Biomed. Eng. 62(11), 2629–2637 (2015).
[Crossref] [PubMed]

W. Wang, S. Stuijk, and G. De Haan, “Exploiting spatial redundancy of image sensor for motion robust rPPG,” IEEE Trans. Biomed. Eng. 62(2), 415–425 (2015).
[Crossref]

G. de Haan and A. Van Leest, “Improved motion robustness of remote-ppg by using the blood volume pulse signature,” Physiol. Meas. 35(9), 1913 (2014).
[Crossref] [PubMed]

R. van Luijtelaar, W. Wang, S. Stuijk, and G. de Haan, “Automatic ROI Detection for camera-based pulse-rate measurement,” in Proceedings of Asian Conf. Comput. Vision (Springer, 2014), pp. 360–374.

E. Calvo-Gallego and G. de Haan, “Automatic ROI for remote photoplethysmography using PPG and color features,” in Proceedings of Int. Conf. Comput. Vision Theory Appl. (SCITEPRESS, 2015), pp. 357–364.

Erofeev, N. P.

A. A. Kamshilin, E. Nippolainen, I. S. Sidorov, P. V. Vasilev, N. P. Erofeev, N. P. Podolian, and R. V. Romashko, “A new look at the essence of the imaging photoplethysmography,” Sci. Rep. 5, 10494 (2015).
[Crossref] [PubMed]

Fieguth, P.

A. Wong, A. Mishra, W. Zhang, P. Fieguth, and D. A. Clausi, “Stochastic image denoising based on Markov-chain Monte Carlo sampling,” Signal Process. 91(8), 2112–2120 (2011).
[Crossref]

P. Fieguth, Statistical Image Processing and Multidimensional Modeling (Springer, 2010).

Gibert, G.

G. Gibert, D. D’Alessandro, and F. Lance, “Face detection method based on photoplethysmography,” in Proceedings of Adv. Video and Signal Based Surveillance (IEEE, 2013), pp. 449–453.

Gontarek, S.

D. McDuff, S. Gontarek, and R. W. Picard, “Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera,” IEEE Trans. Biomed. Eng. 61(12), 2948–2954 (2014).
[Crossref] [PubMed]

D. McDuff, S. Gontarek, and R. W. Picard, “Improvements in remote cardiopulmonary measurement using a five band digital camera,” IEEE Trans. Biomed. Eng. 61(10), 2593–2601 (2014).
[Crossref] [PubMed]

Greenwald, S.

Y. Sun, S. Hu, V. Azorin-Peris, R. Kalawsky, and S. Greenwald, “Noncontact imaging photoplethysmography to effectively access pulse rate variability,” J. Biomed. Opt. 18(6), 061205 (2013).
[Crossref]

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16(7), 077010 (2011).
[Crossref] [PubMed]

Howell, K.

J. Allen and K. Howell, “Microvascular imaging: techniques and opportunities for clinical physiological measurements,” Physiol. Meas. 35(7), R91 (2014).
[Crossref] [PubMed]

Hu, S.

Y. Sun, S. Hu, V. Azorin-Peris, R. Kalawsky, and S. Greenwald, “Noncontact imaging photoplethysmography to effectively access pulse rate variability,” J. Biomed. Opt. 18(6), 061205 (2013).
[Crossref]

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16(7), 077010 (2011).
[Crossref] [PubMed]

Imholz, B. P.

B. P. Imholz, W. Wieling, G. A. van Montfrans, and K. H. Wesseling, “Fifteen years experience with finger arterial pressure monitoring,” Cardiovasc. Res. 38(3), 605–616 (1998).
[Crossref] [PubMed]

Jacques, S. L.

S. L. Jacques, “Optical properties of biological tissues: a review,” Physics in Med. and Biol. 58(11), R37–R61 (2013).
[Crossref]

Kalawsky, R.

Y. Sun, S. Hu, V. Azorin-Peris, R. Kalawsky, and S. Greenwald, “Noncontact imaging photoplethysmography to effectively access pulse rate variability,” J. Biomed. Opt. 18(6), 061205 (2013).
[Crossref]

Kamshilin, A. A.

A. A. Kamshilin, E. Nippolainen, I. S. Sidorov, P. V. Vasilev, N. P. Erofeev, N. P. Podolian, and R. V. Romashko, “A new look at the essence of the imaging photoplethysmography,” Sci. Rep. 5, 10494 (2015).
[Crossref] [PubMed]

A. A. Kamshilin, S. Miridonov, V. Teplov, R. Saarenheimo, and E. Nippolainen, “Photoplethysmographic imaging of high spatial resolution,” Biomed. Opt. Express 2(4), 996–1006 (2011).
[Crossref] [PubMed]

Karjalainen, P. A.

M. P. Tarvainen, P. O. Ranta-aho, and P. A. Karjalainen, “An advanced detrending method with application to HRV analysis,” IEEE Trans. Biomed. Eng. 49(2), 172–175 (2002).
[Crossref] [PubMed]

Kumar, M.

Lance, F.

G. Gibert, D. D’Alessandro, and F. Lance, “Face detection method based on photoplethysmography,” in Proceedings of Adv. Video and Signal Based Surveillance (IEEE, 2013), pp. 449–453.

McDuff, D.

D. McDuff, S. Gontarek, and R. W. Picard, “Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera,” IEEE Trans. Biomed. Eng. 61(12), 2948–2954 (2014).
[Crossref] [PubMed]

D. McDuff, S. Gontarek, and R. W. Picard, “Improvements in remote cardiopulmonary measurement using a five band digital camera,” IEEE Trans. Biomed. Eng. 61(10), 2593–2601 (2014).
[Crossref] [PubMed]

McDuff, D. J.

Miridonov, S.

Mishra, A.

A. Wong, A. Mishra, W. Zhang, P. Fieguth, and D. A. Clausi, “Stochastic image denoising based on Markov-chain Monte Carlo sampling,” Signal Process. 91(8), 2112–2120 (2011).
[Crossref]

Morris, D.

J. Wander and D. Morris, “A combined segmenting and non-segmenting approach to signal quality estimation for ambulatory photoplethysmography,” Physiol. Meas. 35(12), 2543–2561 (2014).
[Crossref] [PubMed]

Nippolainen, E.

A. A. Kamshilin, E. Nippolainen, I. S. Sidorov, P. V. Vasilev, N. P. Erofeev, N. P. Podolian, and R. V. Romashko, “A new look at the essence of the imaging photoplethysmography,” Sci. Rep. 5, 10494 (2015).
[Crossref] [PubMed]

A. A. Kamshilin, S. Miridonov, V. Teplov, R. Saarenheimo, and E. Nippolainen, “Photoplethysmographic imaging of high spatial resolution,” Biomed. Opt. Express 2(4), 996–1006 (2011).
[Crossref] [PubMed]

Parrish, J. A.

R. R. Anderson and J. A. Parrish, “The optics of human skin,” J. Invest. Dermatol. 77(1), 13–19 (1981).
[Crossref] [PubMed]

Picard, R. W.

D. McDuff, S. Gontarek, and R. W. Picard, “Improvements in remote cardiopulmonary measurement using a five band digital camera,” IEEE Trans. Biomed. Eng. 61(10), 2593–2601 (2014).
[Crossref] [PubMed]

D. McDuff, S. Gontarek, and R. W. Picard, “Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera,” IEEE Trans. Biomed. Eng. 61(12), 2948–2954 (2014).
[Crossref] [PubMed]

M.-Z. Poh, D. J. McDuff, and R. W. Picard, “Non-contact, automated cardiac pulse measurements using video imaging and blind source separation,” Opt. Express 18(10), 10762–10774 (2010).
[Crossref] [PubMed]

Podolian, N. P.

A. A. Kamshilin, E. Nippolainen, I. S. Sidorov, P. V. Vasilev, N. P. Erofeev, N. P. Podolian, and R. V. Romashko, “A new look at the essence of the imaging photoplethysmography,” Sci. Rep. 5, 10494 (2015).
[Crossref] [PubMed]

Poh, M.-Z.

Ranta-aho, P. O.

M. P. Tarvainen, P. O. Ranta-aho, and P. A. Karjalainen, “An advanced detrending method with application to HRV analysis,” IEEE Trans. Biomed. Eng. 49(2), 172–175 (2002).
[Crossref] [PubMed]

Rohde, G. K.

Romashko, R. V.

A. A. Kamshilin, E. Nippolainen, I. S. Sidorov, P. V. Vasilev, N. P. Erofeev, N. P. Podolian, and R. V. Romashko, “A new look at the essence of the imaging photoplethysmography,” Sci. Rep. 5, 10494 (2015).
[Crossref] [PubMed]

Saarenheimo, R.

Sabharwal, A.

Schäfer, A.

A. Schäfer and J. Vagedes, “How accurate is pulse rate variability as an estimate of heart rate variability?: A review on studies comparing photoplethysmographic technology with an electrocardiogram,” Int. J. Cardiol. 166(1), 15–29 (2013).
[Crossref]

Sidorov, I. S.

A. A. Kamshilin, E. Nippolainen, I. S. Sidorov, P. V. Vasilev, N. P. Erofeev, N. P. Podolian, and R. V. Romashko, “A new look at the essence of the imaging photoplethysmography,” Sci. Rep. 5, 10494 (2015).
[Crossref] [PubMed]

Stuijk, S.

W. Wang, S. Stuijk, and G. De Haan, “Exploiting spatial redundancy of image sensor for motion robust rPPG,” IEEE Trans. Biomed. Eng. 62(2), 415–425 (2015).
[Crossref]

W. Wang, S. Stuijk, and G. de Haan, “Unsupervised subject detection via remote PPG,” IEEE Trans. Biomed. Eng. 62(11), 2629–2637 (2015).
[Crossref] [PubMed]

R. van Luijtelaar, W. Wang, S. Stuijk, and G. de Haan, “Automatic ROI Detection for camera-based pulse-rate measurement,” in Proceedings of Asian Conf. Comput. Vision (Springer, 2014), pp. 360–374.

Sun, L.

Sun, Y.

Y. Sun and N. Thakor, “Photoplethysmography revisited: from contact to noncontact, from point to imaging,” IEEE Trans. Biomed. Eng. 63(3), 463–477 (2016).
[Crossref]

Y. Sun, S. Hu, V. Azorin-Peris, R. Kalawsky, and S. Greenwald, “Noncontact imaging photoplethysmography to effectively access pulse rate variability,” J. Biomed. Opt. 18(6), 061205 (2013).
[Crossref]

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16(7), 077010 (2011).
[Crossref] [PubMed]

Tarvainen, M. P.

M. P. Tarvainen, P. O. Ranta-aho, and P. A. Karjalainen, “An advanced detrending method with application to HRV analysis,” IEEE Trans. Biomed. Eng. 49(2), 172–175 (2002).
[Crossref] [PubMed]

Teplov, V.

Thakor, N.

Y. Sun and N. Thakor, “Photoplethysmography revisited: from contact to noncontact, from point to imaging,” IEEE Trans. Biomed. Eng. 63(3), 463–477 (2016).
[Crossref]

Vagedes, J.

A. Schäfer and J. Vagedes, “How accurate is pulse rate variability as an estimate of heart rate variability?: A review on studies comparing photoplethysmographic technology with an electrocardiogram,” Int. J. Cardiol. 166(1), 15–29 (2013).
[Crossref]

Van Leest, A.

G. de Haan and A. Van Leest, “Improved motion robustness of remote-ppg by using the blood volume pulse signature,” Physiol. Meas. 35(9), 1913 (2014).
[Crossref] [PubMed]

van Luijtelaar, R.

R. van Luijtelaar, W. Wang, S. Stuijk, and G. de Haan, “Automatic ROI Detection for camera-based pulse-rate measurement,” in Proceedings of Asian Conf. Comput. Vision (Springer, 2014), pp. 360–374.

van Montfrans, G. A.

B. P. Imholz, W. Wieling, G. A. van Montfrans, and K. H. Wesseling, “Fifteen years experience with finger arterial pressure monitoring,” Cardiovasc. Res. 38(3), 605–616 (1998).
[Crossref] [PubMed]

Vasilev, P. V.

A. A. Kamshilin, E. Nippolainen, I. S. Sidorov, P. V. Vasilev, N. P. Erofeev, N. P. Podolian, and R. V. Romashko, “A new look at the essence of the imaging photoplethysmography,” Sci. Rep. 5, 10494 (2015).
[Crossref] [PubMed]

Veeraraghavan, A.

Wander, J.

J. Wander and D. Morris, “A combined segmenting and non-segmenting approach to signal quality estimation for ambulatory photoplethysmography,” Physiol. Meas. 35(12), 2543–2561 (2014).
[Crossref] [PubMed]

Wang, L. V.

L. V. Wang and H.-i. Wu, Biomedical Optics: Principles and Imaging (John Wiley & Sons, 2012).

Wang, W.

W. Wang, S. Stuijk, and G. De Haan, “Exploiting spatial redundancy of image sensor for motion robust rPPG,” IEEE Trans. Biomed. Eng. 62(2), 415–425 (2015).
[Crossref]

W. Wang, S. Stuijk, and G. de Haan, “Unsupervised subject detection via remote PPG,” IEEE Trans. Biomed. Eng. 62(11), 2629–2637 (2015).
[Crossref] [PubMed]

R. van Luijtelaar, W. Wang, S. Stuijk, and G. de Haan, “Automatic ROI Detection for camera-based pulse-rate measurement,” in Proceedings of Asian Conf. Comput. Vision (Springer, 2014), pp. 360–374.

Wesseling, K. H.

B. P. Imholz, W. Wieling, G. A. van Montfrans, and K. H. Wesseling, “Fifteen years experience with finger arterial pressure monitoring,” Cardiovasc. Res. 38(3), 605–616 (1998).
[Crossref] [PubMed]

Wieling, W.

B. P. Imholz, W. Wieling, G. A. van Montfrans, and K. H. Wesseling, “Fifteen years experience with finger arterial pressure monitoring,” Cardiovasc. Res. 38(3), 605–616 (1998).
[Crossref] [PubMed]

Wong, A.

A. Wong, A. Mishra, W. Zhang, P. Fieguth, and D. A. Clausi, “Stochastic image denoising based on Markov-chain Monte Carlo sampling,” Signal Process. 91(8), 2112–2120 (2011).
[Crossref]

Wu, H.-i.

L. V. Wang and H.-i. Wu, Biomedical Optics: Principles and Imaging (John Wiley & Sons, 2012).

Xu, S.

Zhang, W.

A. Wong, A. Mishra, W. Zhang, P. Fieguth, and D. A. Clausi, “Stochastic image denoising based on Markov-chain Monte Carlo sampling,” Signal Process. 91(8), 2112–2120 (2011).
[Crossref]

Zhu, Y.

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16(7), 077010 (2011).
[Crossref] [PubMed]

Biomed. Opt. Express (3)

Cardiovasc. Res. (1)

B. P. Imholz, W. Wieling, G. A. van Montfrans, and K. H. Wesseling, “Fifteen years experience with finger arterial pressure monitoring,” Cardiovasc. Res. 38(3), 605–616 (1998).
[Crossref] [PubMed]

IEEE Trans. Biomed. Eng. (6)

W. Wang, S. Stuijk, and G. De Haan, “Exploiting spatial redundancy of image sensor for motion robust rPPG,” IEEE Trans. Biomed. Eng. 62(2), 415–425 (2015).
[Crossref]

D. McDuff, S. Gontarek, and R. W. Picard, “Improvements in remote cardiopulmonary measurement using a five band digital camera,” IEEE Trans. Biomed. Eng. 61(10), 2593–2601 (2014).
[Crossref] [PubMed]

D. McDuff, S. Gontarek, and R. W. Picard, “Remote detection of photoplethysmographic systolic and diastolic peaks using a digital camera,” IEEE Trans. Biomed. Eng. 61(12), 2948–2954 (2014).
[Crossref] [PubMed]

M. P. Tarvainen, P. O. Ranta-aho, and P. A. Karjalainen, “An advanced detrending method with application to HRV analysis,” IEEE Trans. Biomed. Eng. 49(2), 172–175 (2002).
[Crossref] [PubMed]

Y. Sun and N. Thakor, “Photoplethysmography revisited: from contact to noncontact, from point to imaging,” IEEE Trans. Biomed. Eng. 63(3), 463–477 (2016).
[Crossref]

W. Wang, S. Stuijk, and G. de Haan, “Unsupervised subject detection via remote PPG,” IEEE Trans. Biomed. Eng. 62(11), 2629–2637 (2015).
[Crossref] [PubMed]

Int. J. Cardiol. (1)

A. Schäfer and J. Vagedes, “How accurate is pulse rate variability as an estimate of heart rate variability?: A review on studies comparing photoplethysmographic technology with an electrocardiogram,” Int. J. Cardiol. 166(1), 15–29 (2013).
[Crossref]

J. Biomed. Opt. (2)

Y. Sun, S. Hu, V. Azorin-Peris, S. Greenwald, J. Chambers, and Y. Zhu, “Motion-compensated noncontact imaging photoplethysmography to monitor cardiorespiratory status during exercise,” J. Biomed. Opt. 16(7), 077010 (2011).
[Crossref] [PubMed]

Y. Sun, S. Hu, V. Azorin-Peris, R. Kalawsky, and S. Greenwald, “Noncontact imaging photoplethysmography to effectively access pulse rate variability,” J. Biomed. Opt. 18(6), 061205 (2013).
[Crossref]

J. Invest. Dermatol. (1)

R. R. Anderson and J. A. Parrish, “The optics of human skin,” J. Invest. Dermatol. 77(1), 13–19 (1981).
[Crossref] [PubMed]

Opt. Express (1)

Physics in Med. and Biol. (1)

S. L. Jacques, “Optical properties of biological tissues: a review,” Physics in Med. and Biol. 58(11), R37–R61 (2013).
[Crossref]

Physiol. Meas. (4)

G. de Haan and A. Van Leest, “Improved motion robustness of remote-ppg by using the blood volume pulse signature,” Physiol. Meas. 35(9), 1913 (2014).
[Crossref] [PubMed]

J. Wander and D. Morris, “A combined segmenting and non-segmenting approach to signal quality estimation for ambulatory photoplethysmography,” Physiol. Meas. 35(12), 2543–2561 (2014).
[Crossref] [PubMed]

J. Allen, “Photoplethysmography and its application in clinical physiological measurement,” Physiol. Meas. 28(3), R1–R39 (2007).
[Crossref] [PubMed]

J. Allen and K. Howell, “Microvascular imaging: techniques and opportunities for clinical physiological measurements,” Physiol. Meas. 35(7), R91 (2014).
[Crossref] [PubMed]

Sci. Rep. (1)

A. A. Kamshilin, E. Nippolainen, I. S. Sidorov, P. V. Vasilev, N. P. Erofeev, N. P. Podolian, and R. V. Romashko, “A new look at the essence of the imaging photoplethysmography,” Sci. Rep. 5, 10494 (2015).
[Crossref] [PubMed]

Signal Process. (1)

A. Wong, A. Mishra, W. Zhang, P. Fieguth, and D. A. Clausi, “Stochastic image denoising based on Markov-chain Monte Carlo sampling,” Signal Process. 91(8), 2112–2120 (2011).
[Crossref]

Other (6)

R-B, “Easy Pulse sensor (version 1.1) overview (part 1),” http://embedded-lab.com/blog/?p=7336 ). (12Dec2014).

L. V. Wang and H.-i. Wu, Biomedical Optics: Principles and Imaging (John Wiley & Sons, 2012).

P. Fieguth, Statistical Image Processing and Multidimensional Modeling (Springer, 2010).

E. Calvo-Gallego and G. de Haan, “Automatic ROI for remote photoplethysmography using PPG and color features,” in Proceedings of Int. Conf. Comput. Vision Theory Appl. (SCITEPRESS, 2015), pp. 357–364.

G. Gibert, D. D’Alessandro, and F. Lance, “Face detection method based on photoplethysmography,” in Proceedings of Adv. Video and Signal Based Surveillance (IEEE, 2013), pp. 449–453.

R. van Luijtelaar, W. Wang, S. Stuijk, and G. de Haan, “Automatic ROI Detection for camera-based pulse-rate measurement,” in Proceedings of Asian Conf. Comput. Vision (Springer, 2014), pp. 360–374.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1
Fig. 1 Processing pipeline of the proposed signal extraction method. Acquired frames were converted from reflectance to absorbance and detrended. Spectral-spatial probabilistic prior maps were computed and used to model the posterior distribution representing the pulsatility model. Bayesian least-squares optimization was used to generate the blood pulse waveform signal.
Fig. 2
Fig. 2 Quasi-periodic nature of a typical blood pulse waveform signal. The periodicity and dicrotic characteristics of the waveform result in predominantly harmonic frequencies in the power spectral density.
Fig. 3
Fig. 3 Signals extracted from all 23 participants using the proposed FusionPPG method (black), plotted against to the ground-truth FingerPPG waveform (gray, dotted).
Fig. 4
Fig. 4 Box plot comparison of the correlation (a) and normalized spectral entropy (b) between the signals extracted using the proposed (FusionPPG) and the two comparison (FaceMeanPPG, DistancePPG) methods. FusionPPG exhibited significantly higher correlation and significantly lower spectral entropy (i.e., higher spectral compactness) compared to FaceMeanPPG and DistancePPG. (***statistically significant difference, p < 0.001)
Fig. 5
Fig. 5 Correlation and Bland-Altman plots of the predicted heart rates using the extracted blood pulse waveform signal. The predicted heart rates were highly correlated to the ground-truth heart rate (r2 = 0.9952), and were in tight agreement (μ = −1.0 bpm, σ = 0.70 bpm). The outlier was omitted due to failed signal extraction.
Fig. 6
Fig. 6 Extracted waveforms from the proposed and comparison methods across four participants. The selected waveforms were those that exhibited the strongest correlation from DistancePPG (a), FaceMeanPPG (b), FusionPPG (c), and a participant with arrhythmia (d). FusionPPG was able to extract strong waveforms across all participants, enabling the visual assessment of a cardiac arrhythmia (at t = 6 s).
Fig. 7
Fig. 7 Typical pulsatility distribution based on spectral-spatial fusion. An original frame (a) is used to compute and overlay the probabilistic pulsatility distribution (b). The strongest pulsing was often observed in the neck region, contributing strongly to the blood pulse waveform extraction.

Equations (19)

Equations on this page are rendered with MathJax. Learn more.

x i = x i ( t ) k = δ ( t k T )
z ^ = arg min z ^ { E [ ( z ^ z ) T ( z ^ z ) | X ] }
= arg min z ^ ( z ^ z ) T ( z ^ z ) p ( z | X ) d z
z ^ ( z ^ z ) T ( z ^ z ) p ( z | X ) d z = 0
2 ( z ^ z ) p ( z | X ) d z = 0
z ^ p ( z | X ) d z z p ( z | X ) d z = 0
z ^ = z p ( z | X ) d z
p ^ ( z | X ) = i = 1 | X | W i δ ( | z x i | ) Y
Γ i ( f ) = | F i ( f ) | 2 | F i ( f ) | 2 d f
h i = f * Δ f f * + Δ f Γ i ( f ) d f + 2 f * Δ f 2 f * + Δ f Γ i ( f ) d f
W i harm = exp ( ( 1 h i ) 2 α h )
q i = max f { 1 f * Δ f f * + Δ f Γ i ( f ) d f }
W i nmag = exp ( q i 2 α q )
W i spat = exp ( Λ 2 α l )
W i = inf { k W i [ k ] | N i }
H ( z ^ ) = k = 0 N 1 Z [ k ] log Z [ k ]
ρ ( z ^ , y ) = | σ z ^ , y | | σ z ^ , σ y |
H R ^ = 60 F s Δ t
k η Z ^ k 1 k η Z ^ k

Metrics