Abstract

Imaging through a wavy water surface is a challenging task, as the wavy water surface introduces anisoplanatism effects difficult to model and track. A typical recovery method is usually involving multiple-stage processing on a pre-acquired image sequence. A new progressive restoration scheme is demonstrated, it can run simultaneously with image acquisition and mitigate both distortion and blur progressively. This method extends the anisotropic evolution in lucky region fusion with a novel progressive optical flow based de-warping scheme, centroid evolution. A comparison has been made with other state-of-art techniques, the proposed method can create comparable results, even with much less frames acquired. Experiments with real through-water scenes have also proved the effectiveness of the method.

© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Full Article  |  PDF Article
OSA Recommended Articles
Reconstruction of distorted underwater images using robust registration

Zhen Zhang and Xu Yang
Opt. Express 27(7) 9996-10008 (2019)

Correction of geometrically distorted underwater images using shift map analysis

Kalyan Kumar Halder, Manoranjan Paul, Murat Tahtali, Sreenatha G. Anavatti, and Manzur Murshed
J. Opt. Soc. Am. A 34(4) 666-673 (2017)

Restoration of images degraded by underwater turbulence using structure tensor oriented image quality (STOIQ) metric

A. V. Kanaev, W. Hou, S. R. Restaino, S. Matt, and S. Gładysz
Opt. Express 23(13) 17077-17090 (2015)

References

  • View by:
  • |
  • |
  • |

  1. R. E. Potter, “Observations from below a rough water surface to determine conditions of or above the surface waves,” (August 22, 1996).
  2. M. Alterman, Y. Y. Schechner, P. Perona, and J. Shamir, “Detecting motion through dynamic refraction,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 245–251 (2013).
    [Crossref] [PubMed]
  3. M. Alterman, Y. Y. Schechner, and Y. Swirski, “Triangulation in random refractive distortions,” in IEEE International Conference on Computational Photography (ICCP) (2013), pp. 1–10.
  4. H. Suiter, N. Flacco, P. Carter, K. Tong, R. Ries, and M. Gershenson, “Optics near the Snell angle in a water-to-air change of medium,” in Oceans 2007 (IEEE, 2007), pp. 1–12.
  5. D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
    [Crossref]
  6. M. Alterman, Y. Swirski, and Y. Y. Schechner, “STELLA MARIS: stellar marine refractive imaging sensor,” in 2014 IEEE International Conference on Computational Photography (ICCP) (IEEE, 2014), pp. 1–10.
    [Crossref]
  7. Y. Y. Schechner and N. Karpel, “Attenuating natural flicker patterns,” in Oceans ’04 MTS/IEEE Techno-Ocean ’04 (IEEE Cat. No.04CH37600) (2004), Vol. 3, pp. 1262–1268.
  8. H. Schultz and A. Corrada-Emmanuel, “System and method for imaging through an irregular water surface,” (December 2009).
  9. P. H. Barsic and C. R. Chinn, “Sea surface slope recovery through passive polarimetric imaging,” in 2012 Oceans (IEEE, 2012), pp. 1–9.
  10. A. Efros, V. Isler, J. Shi, and M. Visontai, “Seeing through water,” in Advances in Neural Information Processing Systems (2005), pp. 393–400.
  11. A. Donate, G. Dahme, and E. Ribeiro, “Classification of textures distorted by waterwaves,” in 18th International Conference on Pattern Recognition (ICPR’06) (2006), Vol. 2, pp. 421–424.
    [Crossref]
  12. A. Donate and E. Ribeiro, “Improved reconstruction of images distorted by water waves,” in Advances in Computer Graphics and Computer Vision, Communications in Computer and Information Science (Springer Berlin Heidelberg, 2007), pp. 264–277.
  13. Z. Wen, A. Lambert, D. Fraser, and H. Li, “Bispectral analysis and recovery of images distorted by a moving water surface,” Appl. Opt. 49(33), 6376–6384 (2010).
    [Crossref] [PubMed]
  14. Y. Tian and S. G. Narasimhan, “Seeing through water: image restoration using model-based tracking,” in 2009 IEEE 12th International Conference on Computer Vision (IEEE, 2009), pp. 2303–2310.
    [Crossref]
  15. O. Oreifej, G. Shu, T. Pace, and M. Shah, “A two-stage reconstruction approach for seeing through water,” in CVPR 2011 (IEEE, 2011), pp. 1153–1160.
  16. K. K. Halder, M. Tahtali, and S. G. Anavatti, “Simple and efficient approach for restoration of non-uniformly warped images,” Appl. Opt. 53(25), 5576–5584 (2014).
    [Crossref] [PubMed]
  17. M. Tahtali, D. Fraser, and A. Lambert, “Restoration of non-uniformly warped images using a typical frame as prototype,” in TENCON 2005 - 2005 IEEE Region 10 Conference (IEEE, 2005), pp. 1–6.
  18. K. K. Halder, M. Tahtali, and S. G. Anavatti, “Geometric correction of atmospheric turbulence-degraded video containing moving objects,” Opt. Express 23(4), 5091–5101 (2015).
    [Crossref] [PubMed]
  19. K. K. Halder, M. Paul, M. Tahtali, S. G. Anavatti, and M. Murshed, “Correction of geometrically distorted underwater images using shift map analysis,” J. Opt. Soc. Am. A 34(4), 666–673 (2017).
    [Crossref] [PubMed]
  20. Z. Li, Z. Murez, D. Kriegman, R. Ramamoorthi, and M. Chandraker, “Learning to see through turbulent water,” in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (2018), pp. 512–520.
    [Crossref]
  21. M. Tahtali, A. J. Lambert, and D. Fraser, “Restoration of nonuniformly warped images using accurate frame by frame shiftmap accumulation,” Proc. SPIE 6316, 631603 (2006).
    [Crossref]
  22. M. Tahtali, A. J. Lambert, and D. Fraser, “Progressive restoration of nonuniformly warped images by shiftmap prediction using Kalman filter,” in Adaptive Optics: Analysis and Methods/Computational Optical Sensing and Imaging/Information Photonics/Signal Recovery and Synthesis Topical Meetings on CD-ROM (2007) (Optical Society of America, 2007), p. SMC5.
  23. M. Tahtali, A. Lambert, and D. Fraser, “Self-tuning Kalman filter estimation of atmospheric warp,” Proc. SPIE 7076, 70760F (2008).
    [Crossref]
  24. M. Tahtali and A. Lambert, “Statistical turbulence approach to the covariance matrices in the shiftmap prediction using Kalman filter,” in Frontiers in Optics 2009/Laser Science XXV/Fall 2009 OSA Optics & Photonics Technical Digest (2009) (Optical Society of America, 2009), p. STuC2.
  25. K. K. Halder, M. Tahtali, and S. G. Anavatti, “Model-free prediction of atmospheric warp based on artificial neural network,” Appl. Opt. 53(30), 7087–7094 (2014).
    [Crossref] [PubMed]
  26. G. W. Carhart and M. A. Vorontsov, “Synthetic imaging: nonadaptive anisoplanatic image correction in atmospheric turbulence,” Opt. Lett. 23(10), 745–747 (1998).
    [Crossref] [PubMed]
  27. M. A. Vorontsov, “Parallel image processing based on an evolution equation with anisotropic gain: integrated optoelectronic architectures,” J. Opt. Soc. Am. A 16(7), 1623–1637 (1999).
    [Crossref]
  28. M. A. Vorontsov and G. W. Carhart, “Anisoplanatic imaging through turbulent media: image recovery by local information fusion from a set of short-exposure images,” J. Opt. Soc. Am. A 18(6), 1312–1324 (2001).
    [Crossref] [PubMed]
  29. M. Aubailly, M. A. Vorontsov, G. W. Carhart, and M. T. Valley, “Video enhancement through automated lucky-region fusion from a stream of atmospherically-distorted images,” in Computational Optical Sensing and Imaging (Optical Society of America, 2009), p. CThC3.
  30. M. Aubailly, M. Vorontsov, G. W. Carhart, and M. T. Valley, “Automated video enhancement from a stream of atmospherically- distorted images: the lucky-region fusion approach,” Proc. SPIE 7463, 74630C (2009).
    [Crossref]
  31. R. Zhang, “Imaging through wavy water surface,” (2018). https://doi.org/10.6084/m9.figshare.6843875.v1 .
  32. A. Bruhn, J. Weickert, and C. Schnörr, “Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods,” Int. J. Comput. Vis. 61(3), 211–231 (2005).
    [Crossref]
  33. C. Liu, “Beyond pixels: exploring new representations and applications for motion analysis,” Massachusetts Institute of Technology (2009).
  34. M. Chen, W. Lu, Q. Chen, K. J. Ruchala, and G. H. Olivera, “A simple fixed-point approach to invert a deformation field,” Med. Phys. 35(1), 81–88 (2007).
    [Crossref] [PubMed]
  35. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004).
    [Crossref] [PubMed]
  36. A. Buades, B. Coll, and J. Morel, “A non-local algorithm for image denoising,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05) (2005), Vol. 2, pp. 60–65.
  37. A. V. Kanaev, W. Hou, S. R. Restaino, S. Matt, and S. Gładysz, “Restoration of images degraded by underwater turbulence using structure tensor oriented image quality (STOIQ) metric,” Opt. Express 23(13), 17077–17090 (2015).
    [Crossref] [PubMed]

2017 (1)

2015 (2)

2014 (2)

2013 (1)

M. Alterman, Y. Y. Schechner, P. Perona, and J. Shamir, “Detecting motion through dynamic refraction,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 245–251 (2013).
[Crossref] [PubMed]

2010 (1)

2009 (1)

M. Aubailly, M. Vorontsov, G. W. Carhart, and M. T. Valley, “Automated video enhancement from a stream of atmospherically- distorted images: the lucky-region fusion approach,” Proc. SPIE 7463, 74630C (2009).
[Crossref]

2008 (1)

M. Tahtali, A. Lambert, and D. Fraser, “Self-tuning Kalman filter estimation of atmospheric warp,” Proc. SPIE 7076, 70760F (2008).
[Crossref]

2007 (1)

M. Chen, W. Lu, Q. Chen, K. J. Ruchala, and G. H. Olivera, “A simple fixed-point approach to invert a deformation field,” Med. Phys. 35(1), 81–88 (2007).
[Crossref] [PubMed]

2006 (2)

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

M. Tahtali, A. J. Lambert, and D. Fraser, “Restoration of nonuniformly warped images using accurate frame by frame shiftmap accumulation,” Proc. SPIE 6316, 631603 (2006).
[Crossref]

2005 (1)

A. Bruhn, J. Weickert, and C. Schnörr, “Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods,” Int. J. Comput. Vis. 61(3), 211–231 (2005).
[Crossref]

2004 (1)

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004).
[Crossref] [PubMed]

2001 (1)

1999 (1)

1998 (1)

Alterman, M.

M. Alterman, Y. Y. Schechner, P. Perona, and J. Shamir, “Detecting motion through dynamic refraction,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 245–251 (2013).
[Crossref] [PubMed]

M. Alterman, Y. Y. Schechner, and Y. Swirski, “Triangulation in random refractive distortions,” in IEEE International Conference on Computational Photography (ICCP) (2013), pp. 1–10.

M. Alterman, Y. Swirski, and Y. Y. Schechner, “STELLA MARIS: stellar marine refractive imaging sensor,” in 2014 IEEE International Conference on Computational Photography (ICCP) (IEEE, 2014), pp. 1–10.
[Crossref]

Anavatti, S. G.

Aubailly, M.

M. Aubailly, M. Vorontsov, G. W. Carhart, and M. T. Valley, “Automated video enhancement from a stream of atmospherically- distorted images: the lucky-region fusion approach,” Proc. SPIE 7463, 74630C (2009).
[Crossref]

Bovik, A. C.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004).
[Crossref] [PubMed]

Bruhn, A.

A. Bruhn, J. Weickert, and C. Schnörr, “Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods,” Int. J. Comput. Vis. 61(3), 211–231 (2005).
[Crossref]

Carhart, G. W.

Carter, P. W.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Chandraker, M.

Z. Li, Z. Murez, D. Kriegman, R. Ramamoorthi, and M. Chandraker, “Learning to see through turbulent water,” in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (2018), pp. 512–520.
[Crossref]

Chen, M.

M. Chen, W. Lu, Q. Chen, K. J. Ruchala, and G. H. Olivera, “A simple fixed-point approach to invert a deformation field,” Med. Phys. 35(1), 81–88 (2007).
[Crossref] [PubMed]

Chen, Q.

M. Chen, W. Lu, Q. Chen, K. J. Ruchala, and G. H. Olivera, “A simple fixed-point approach to invert a deformation field,” Med. Phys. 35(1), 81–88 (2007).
[Crossref] [PubMed]

Corrada-Emmanuel, A.

H. Schultz and A. Corrada-Emmanuel, “System and method for imaging through an irregular water surface,” (December 2009).

Dahme, G.

A. Donate, G. Dahme, and E. Ribeiro, “Classification of textures distorted by waterwaves,” in 18th International Conference on Pattern Recognition (ICPR’06) (2006), Vol. 2, pp. 421–424.
[Crossref]

Donate, A.

A. Donate, G. Dahme, and E. Ribeiro, “Classification of textures distorted by waterwaves,” in 18th International Conference on Pattern Recognition (ICPR’06) (2006), Vol. 2, pp. 421–424.
[Crossref]

Flacco, N. L.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Fraser, D.

Z. Wen, A. Lambert, D. Fraser, and H. Li, “Bispectral analysis and recovery of images distorted by a moving water surface,” Appl. Opt. 49(33), 6376–6384 (2010).
[Crossref] [PubMed]

M. Tahtali, A. Lambert, and D. Fraser, “Self-tuning Kalman filter estimation of atmospheric warp,” Proc. SPIE 7076, 70760F (2008).
[Crossref]

M. Tahtali, A. J. Lambert, and D. Fraser, “Restoration of nonuniformly warped images using accurate frame by frame shiftmap accumulation,” Proc. SPIE 6316, 631603 (2006).
[Crossref]

Gladysz, S.

Halder, K. K.

Hou, W.

Hubbard, B. E.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Jones, N. M.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Kanaev, A. V.

Kriegman, D.

Z. Li, Z. Murez, D. Kriegman, R. Ramamoorthi, and M. Chandraker, “Learning to see through turbulent water,” in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (2018), pp. 512–520.
[Crossref]

Lambert, A.

Z. Wen, A. Lambert, D. Fraser, and H. Li, “Bispectral analysis and recovery of images distorted by a moving water surface,” Appl. Opt. 49(33), 6376–6384 (2010).
[Crossref] [PubMed]

M. Tahtali, A. Lambert, and D. Fraser, “Self-tuning Kalman filter estimation of atmospheric warp,” Proc. SPIE 7076, 70760F (2008).
[Crossref]

Lambert, A. J.

M. Tahtali, A. J. Lambert, and D. Fraser, “Restoration of nonuniformly warped images using accurate frame by frame shiftmap accumulation,” Proc. SPIE 6316, 631603 (2006).
[Crossref]

Li, H.

Li, Z.

Z. Li, Z. Murez, D. Kriegman, R. Ramamoorthi, and M. Chandraker, “Learning to see through turbulent water,” in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (2018), pp. 512–520.
[Crossref]

Lu, W.

M. Chen, W. Lu, Q. Chen, K. J. Ruchala, and G. H. Olivera, “A simple fixed-point approach to invert a deformation field,” Med. Phys. 35(1), 81–88 (2007).
[Crossref] [PubMed]

Matt, S.

Milder, D. M.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Murez, Z.

Z. Li, Z. Murez, D. Kriegman, R. Ramamoorthi, and M. Chandraker, “Learning to see through turbulent water,” in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (2018), pp. 512–520.
[Crossref]

Murshed, M.

Olivera, G. H.

M. Chen, W. Lu, Q. Chen, K. J. Ruchala, and G. H. Olivera, “A simple fixed-point approach to invert a deformation field,” Med. Phys. 35(1), 81–88 (2007).
[Crossref] [PubMed]

Panici, K. R.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Paul, M.

Perona, P.

M. Alterman, Y. Y. Schechner, P. Perona, and J. Shamir, “Detecting motion through dynamic refraction,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 245–251 (2013).
[Crossref] [PubMed]

Platt, B. D.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Potter, R. E.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

R. E. Potter, “Observations from below a rough water surface to determine conditions of or above the surface waves,” (August 22, 1996).

Ramamoorthi, R.

Z. Li, Z. Murez, D. Kriegman, R. Ramamoorthi, and M. Chandraker, “Learning to see through turbulent water,” in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (2018), pp. 512–520.
[Crossref]

Restaino, S. R.

Ribeiro, E.

A. Donate, G. Dahme, and E. Ribeiro, “Classification of textures distorted by waterwaves,” in 18th International Conference on Pattern Recognition (ICPR’06) (2006), Vol. 2, pp. 421–424.
[Crossref]

Ruchala, K. J.

M. Chen, W. Lu, Q. Chen, K. J. Ruchala, and G. H. Olivera, “A simple fixed-point approach to invert a deformation field,” Med. Phys. 35(1), 81–88 (2007).
[Crossref] [PubMed]

Schechner, Y. Y.

M. Alterman, Y. Y. Schechner, P. Perona, and J. Shamir, “Detecting motion through dynamic refraction,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 245–251 (2013).
[Crossref] [PubMed]

M. Alterman, Y. Y. Schechner, and Y. Swirski, “Triangulation in random refractive distortions,” in IEEE International Conference on Computational Photography (ICCP) (2013), pp. 1–10.

M. Alterman, Y. Swirski, and Y. Y. Schechner, “STELLA MARIS: stellar marine refractive imaging sensor,” in 2014 IEEE International Conference on Computational Photography (ICCP) (IEEE, 2014), pp. 1–10.
[Crossref]

Schnörr, C.

A. Bruhn, J. Weickert, and C. Schnörr, “Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods,” Int. J. Comput. Vis. 61(3), 211–231 (2005).
[Crossref]

Schultz, H.

H. Schultz and A. Corrada-Emmanuel, “System and method for imaging through an irregular water surface,” (December 2009).

Shamir, J.

M. Alterman, Y. Y. Schechner, P. Perona, and J. Shamir, “Detecting motion through dynamic refraction,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 245–251 (2013).
[Crossref] [PubMed]

Sheikh, H. R.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004).
[Crossref] [PubMed]

Simoncelli, E. P.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004).
[Crossref] [PubMed]

Swirski, Y.

M. Alterman, Y. Y. Schechner, and Y. Swirski, “Triangulation in random refractive distortions,” in IEEE International Conference on Computational Photography (ICCP) (2013), pp. 1–10.

M. Alterman, Y. Swirski, and Y. Y. Schechner, “STELLA MARIS: stellar marine refractive imaging sensor,” in 2014 IEEE International Conference on Computational Photography (ICCP) (IEEE, 2014), pp. 1–10.
[Crossref]

Tahtali, M.

Tong, K. W.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Twisselmann, D. J.

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Valley, M. T.

M. Aubailly, M. Vorontsov, G. W. Carhart, and M. T. Valley, “Automated video enhancement from a stream of atmospherically- distorted images: the lucky-region fusion approach,” Proc. SPIE 7463, 74630C (2009).
[Crossref]

Vorontsov, M.

M. Aubailly, M. Vorontsov, G. W. Carhart, and M. T. Valley, “Automated video enhancement from a stream of atmospherically- distorted images: the lucky-region fusion approach,” Proc. SPIE 7463, 74630C (2009).
[Crossref]

Vorontsov, M. A.

Wang, Z.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004).
[Crossref] [PubMed]

Weickert, J.

A. Bruhn, J. Weickert, and C. Schnörr, “Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods,” Int. J. Comput. Vis. 61(3), 211–231 (2005).
[Crossref]

Wen, Z.

Appl. Opt. (3)

IEEE Trans. Image Process. (1)

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004).
[Crossref] [PubMed]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

M. Alterman, Y. Y. Schechner, P. Perona, and J. Shamir, “Detecting motion through dynamic refraction,” IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 245–251 (2013).
[Crossref] [PubMed]

Int. J. Comput. Vis. (1)

A. Bruhn, J. Weickert, and C. Schnörr, “Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods,” Int. J. Comput. Vis. 61(3), 211–231 (2005).
[Crossref]

J. Opt. Soc. Am. A (3)

Med. Phys. (1)

M. Chen, W. Lu, Q. Chen, K. J. Ruchala, and G. H. Olivera, “A simple fixed-point approach to invert a deformation field,” Med. Phys. 35(1), 81–88 (2007).
[Crossref] [PubMed]

Opt. Express (2)

Opt. Lett. (1)

Proc. SPIE (3)

M. Tahtali, A. Lambert, and D. Fraser, “Self-tuning Kalman filter estimation of atmospheric warp,” Proc. SPIE 7076, 70760F (2008).
[Crossref]

M. Tahtali, A. J. Lambert, and D. Fraser, “Restoration of nonuniformly warped images using accurate frame by frame shiftmap accumulation,” Proc. SPIE 6316, 631603 (2006).
[Crossref]

M. Aubailly, M. Vorontsov, G. W. Carhart, and M. T. Valley, “Automated video enhancement from a stream of atmospherically- distorted images: the lucky-region fusion approach,” Proc. SPIE 7463, 74630C (2009).
[Crossref]

Waves Random Complex Media (1)

D. M. Milder, P. W. Carter, N. L. Flacco, B. E. Hubbard, N. M. Jones, K. R. Panici, B. D. Platt, R. E. Potter, K. W. Tong, and D. J. Twisselmann, “Reconstruction of through-surface underwater imagery,” Waves Random Complex Media 16(4), 521–530 (2006).
[Crossref]

Other (20)

M. Alterman, Y. Swirski, and Y. Y. Schechner, “STELLA MARIS: stellar marine refractive imaging sensor,” in 2014 IEEE International Conference on Computational Photography (ICCP) (IEEE, 2014), pp. 1–10.
[Crossref]

Y. Y. Schechner and N. Karpel, “Attenuating natural flicker patterns,” in Oceans ’04 MTS/IEEE Techno-Ocean ’04 (IEEE Cat. No.04CH37600) (2004), Vol. 3, pp. 1262–1268.

H. Schultz and A. Corrada-Emmanuel, “System and method for imaging through an irregular water surface,” (December 2009).

P. H. Barsic and C. R. Chinn, “Sea surface slope recovery through passive polarimetric imaging,” in 2012 Oceans (IEEE, 2012), pp. 1–9.

A. Efros, V. Isler, J. Shi, and M. Visontai, “Seeing through water,” in Advances in Neural Information Processing Systems (2005), pp. 393–400.

A. Donate, G. Dahme, and E. Ribeiro, “Classification of textures distorted by waterwaves,” in 18th International Conference on Pattern Recognition (ICPR’06) (2006), Vol. 2, pp. 421–424.
[Crossref]

A. Donate and E. Ribeiro, “Improved reconstruction of images distorted by water waves,” in Advances in Computer Graphics and Computer Vision, Communications in Computer and Information Science (Springer Berlin Heidelberg, 2007), pp. 264–277.

R. E. Potter, “Observations from below a rough water surface to determine conditions of or above the surface waves,” (August 22, 1996).

M. Tahtali, D. Fraser, and A. Lambert, “Restoration of non-uniformly warped images using a typical frame as prototype,” in TENCON 2005 - 2005 IEEE Region 10 Conference (IEEE, 2005), pp. 1–6.

M. Alterman, Y. Y. Schechner, and Y. Swirski, “Triangulation in random refractive distortions,” in IEEE International Conference on Computational Photography (ICCP) (2013), pp. 1–10.

H. Suiter, N. Flacco, P. Carter, K. Tong, R. Ries, and M. Gershenson, “Optics near the Snell angle in a water-to-air change of medium,” in Oceans 2007 (IEEE, 2007), pp. 1–12.

Y. Tian and S. G. Narasimhan, “Seeing through water: image restoration using model-based tracking,” in 2009 IEEE 12th International Conference on Computer Vision (IEEE, 2009), pp. 2303–2310.
[Crossref]

O. Oreifej, G. Shu, T. Pace, and M. Shah, “A two-stage reconstruction approach for seeing through water,” in CVPR 2011 (IEEE, 2011), pp. 1153–1160.

R. Zhang, “Imaging through wavy water surface,” (2018). https://doi.org/10.6084/m9.figshare.6843875.v1 .

M. Aubailly, M. A. Vorontsov, G. W. Carhart, and M. T. Valley, “Video enhancement through automated lucky-region fusion from a stream of atmospherically-distorted images,” in Computational Optical Sensing and Imaging (Optical Society of America, 2009), p. CThC3.

C. Liu, “Beyond pixels: exploring new representations and applications for motion analysis,” Massachusetts Institute of Technology (2009).

A. Buades, B. Coll, and J. Morel, “A non-local algorithm for image denoising,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05) (2005), Vol. 2, pp. 60–65.

M. Tahtali, A. J. Lambert, and D. Fraser, “Progressive restoration of nonuniformly warped images by shiftmap prediction using Kalman filter,” in Adaptive Optics: Analysis and Methods/Computational Optical Sensing and Imaging/Information Photonics/Signal Recovery and Synthesis Topical Meetings on CD-ROM (2007) (Optical Society of America, 2007), p. SMC5.

Z. Li, Z. Murez, D. Kriegman, R. Ramamoorthi, and M. Chandraker, “Learning to see through turbulent water,” in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) (2018), pp. 512–520.
[Crossref]

M. Tahtali and A. Lambert, “Statistical turbulence approach to the covariance matrices in the shiftmap prediction using Kalman filter,” in Frontiers in Optics 2009/Laser Science XXV/Fall 2009 OSA Optics & Photonics Technical Digest (2009) (Optical Society of America, 2009), p. STuC2.

Supplementary Material (3)

NameDescription
» Visualization 1       Image restoration results with water-distorted imageries
» Visualization 2       side-by-side comparison of our image restoration results and original distorted image sequense
» Visualization 3       side-by-side comparison of our image restoration results and original distorted image sequense

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1
Fig. 1 Recursive processing workflow.
Fig. 2
Fig. 2 Centroid point C n being shifted by newly added point P n .
Fig. 3
Fig. 3 Image restoration results with the data from [14]. The first column is undistorted image; the second column is the first distorted frame; the 3th–5th columns are the fusion frames in sequence { I F ( n ) ( r ) }created by our recursive method (see Visualization 1).
Fig. 4
Fig. 4 SSIM and PSNR plot over time of the four image sets. Red line denotes the original image sequence { I ( n ) ( r ) }, blue line denotes the new fusion image sequence { I F ( n ) ( r ) } generated by our method.
Fig. 5
Fig. 5 Submerged camera setup in a water tank.
Fig. 6
Fig. 6 The surrounding scenes viewed directly from the position where the water tank located. (a) The building is about 70m away, this scene is used in the first image sequence in our test; (b) The lamp is about 5m away and the building in the background is about 40m away, this scene is used in the second image sequence in our test.
Fig. 7
Fig. 7 Left: the last frame in original image sequence; Right: the last frame in restored image sequence (see side-by-side comparison of the original and restored image sequence in Visualization 2).
Fig. 8
Fig. 8 The first 12 frames in restored image sequence (see side-by-side comparison of the original and restored image sequence in Visualization 3).

Tables (1)

Tables Icon

Table 1 SSIM and PSNR comparison with [15,19].

Equations (10)

Equations on this page are rendered with MathJax. Learn more.

I ( n ) ( r )=( H* I S )( r+ W n ( r ) )+noise,
M( r )= J( r' )G( rr' ) d 2 r' ,
G σ ( r )=exp{ | r | 2 2 σ 2 }.
I F ( r,t ) t =Kδ( r,t )( I F ( r,t )I( r,t ) ),
δ( r,t )={ M( r,t ) M F ( r,t ), 0, M( r,t )> M F ( r,t ) otherwise.
{ I F ( 0 ) ( r )= I ( 0 ) ( r ) I F ( n ) ( r )=( 1K δ n ) I F ( n1 ) ( r )+K δ n I ( n ) ( r ),n=1,2,3,
{ I F ( 0 ) ( r )= I ( 0 ) ( r ) I F ( n ) ( r )=( 1K δ n ) I F ( n1 ) ( r+ U n ( r ) )+K δ n I ( n ) ( r+ V n ( r ) ),n=1,2,3,
C 0 C 1 = 1 2 C 0 P 1 , C 1 C 2 = 1 3 C 1 P 2 ,
C n1 C n = 1 n+1 C n1 P n .
U n ( r )= 1 n+1 ω n ( r ), V n ( r )= n n+1 ω n * ( r ).

Metrics