Abstract

Digital color cameras using a single detector array with a Bayer color filter array (CFA) require interpolation or demosaicing to estimate missing color information and provide full-color images. However, demosaicing does not specifically address fundamental undersampling and aliasing inherent in typical camera designs. Fast non-uniform interpolation based super-resolution (SR) is an attractive approach to reduce or eliminate aliasing and its relatively low computational load is amenable to real-time applications. The adaptive Wiener filter (AWF) SR algorithm was initially developed for grayscale imaging and has not previously been applied to color SR demosaicing. Here, we develop a novel fast SR method for CFA cameras that is based on the AWF SR algorithm and uses global channel-to-channel statistical models. We apply this new method as a stand-alone algorithm and also as an initialization image for a variational SR algorithm. This paper presents the theoretical development of the color AWF SR approach and applies it in performance comparisons to other SR techniques for both simulated and real data.

©2013 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Fast super-resolution with affine motion using an adaptive Wiener filter and its application to airborne imaging

Russell C. Hardie, Kenneth J. Barnard, and Raul Ordonez
Opt. Express 19(27) 26208-26231 (2011)

Fast super-resolution using an adaptive Wiener filter with robustness to local motion

Russell C. Hardie and Kenneth J. Barnard
Opt. Express 20(19) 21053-21073 (2012)

Super-resolution for imagery from integrated microgrid polarimeters

Russell C. Hardie, Daniel A. LeMaster, and Bradley M. Ratliff
Opt. Express 19(14) 12937-12960 (2011)

More Recommended Articles

References

  • View by:
  • Article Order
  • |
  • Year
  • |
  • Author
  • |
  • Publication
  1. B. E. Bayer, “Color imaging array,” Eastman Kodak Co., U.S. Patent 3,971,065 (Jul 20 1976).
  2. R. Ramanath, W. Snyder, G. Bilbro, and W. Sander, “Demosaicking methods for the Bayer color arrays,” J. Electron. Imaging 11(3), 306–315 (2002).
    [Crossref]
  3. B. K. Gunturk, J. Glotzbach, Y. Altunbasak, R. W. Schafer, and R. M. Mersereau, “Demosaicking: color filter array interpolation,” IEEE Signal Proc. Mag. 22(1), 44–54 (2005).
    [Crossref]
  4. X. Li, B. Gunturk, and L. Zhang, “Image demosaicing: a systematic survey,” Proc. SPIE 6822, 68221J, 68221J-15 (2008).
    [Crossref]
  5. R. D. Fiete, “Image quality and λFN/p for remote sensing systems,” Opt. Eng. 38(7), 1229–1240 (1999).
    [Crossref]
  6. S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003).
    [Crossref]
  7. P. Milanfar, Super-Resolution Imaging (CRC, 2010).
  8. J. C. Gillette, T. M. Stadtmiller, and R. C. Hardie, “Aliasing reduction in staring infrared imagers utilizing subpixel techniques,” Opt. Eng. 34(11), 3130–3137 (1995).
    [Crossref]
  9. M. Alam, M. S. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49(5), 915–923 (2000).
    [Crossref]
  10. S. Lertrattanapanich and N. K. Bose, “High resolution image formation from low resolution frames using Delaunay triangulation,” IEEE Trans. Image Process. 11(12), 1427–1441 (2002).
    [Crossref] [PubMed]
  11. K. Aizawa, T. Komatsu, and T. Saito, “Acquisition of very high resolution images using stereo cameras,” P. Soc. Photo-Opt. Ins. 1605, 318–328 (1991).
  12. A. M. Tekalp, M. K. Ozkan, and M. I. Sezan, “High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration,” Int. Conf. Acoust Spee.3, 169–172 (1992).
    [Crossref]
  13. H. Shekarforoush and R. Chellappa, “Data-driven multichannel superresolution with application to video sequences,” J. Opt. Soc. Am. A 16(3), 481–492 (1999).
    [Crossref]
  14. T. Q. Pham, L. J. Van Vliet, and K. Schutte, “Robust fusion of irregularly sampled data using adaptive normalized convolution,” EURASIP J. Adv. Sig. Pr. 1–12 (2006).
  15. J. Shi, S. E. Reichenbach, and J. D. Howe, “Small-kernel superresolution methods for microscanning imaging systems,” Appl. Opt. 45(6), 1203–1214 (2006).
    [Crossref] [PubMed]
  16. R. C. Hardie, “A fast image super-resolution algorithm using an adaptive Wiener filter,” IEEE Trans. Image Process. 16(12), 2953–2964 (2007).
    [Crossref] [PubMed]
  17. R. C. Hardie, D. A. LeMaster, and B. M. Ratliff, “Super-resolution for imagery from integrated microgrid polarimeters,” Opt. Express 19(14), 12937–12960 (2011).
    [Crossref] [PubMed]
  18. R. C. Hardie, K. J. Barnard, and R. Ordonez, “Fast super-resolution with affine motion using an adaptive Wiener filter and its application to airborne imaging,” Opt. Express 19(27), 26208–26231 (2011).
    [Crossref] [PubMed]
  19. B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, “A computationally efficient super-resolution algorithm for video processing using partition filters,” IEEE Circ. Syst. Vid. 17(5), 621–634 (2007).
    [Crossref]
  20. M. Elad and Y. Hel-Or, “A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur,” IEEE Trans. Image Process. 10(8), 1187–1193 (2001).
    [Crossref] [PubMed]
  21. M. Shimizu, T. Yano, and M. Okutomi, “Super-resolution under image deformation,” Int. C. Patt. Recog. 3, 586–589 (2004).
  22. T. Gotoh and M. Okutomi, “Direct super-resolution and registration using raw CFA images,” Proc. CVPR IEEE2, 600–607 (2004).
    [Crossref]
  23. R. Sasahara, H. Hasegawa, I. Yamada, and K. Sakaniwa, “A color super-resolution with multiple nonsmooth constraints by hybrid steepest descent method,” IEEE Trans. Image Proc. 1, 857–860 (2005).
  24. S. Farsiu, M. Elad, and P. Milanfar, “Multiframe demosaicing and super-resolution of color images,” IEEE Trans. Image Proc. 15(1), 141–159 (2006).
    [Crossref] [PubMed]
  25. M. Gevrekci, B. K. Gunturk, and Y. Altunbasak, “POCS-based restoration of Bayer-sampled image sequences,” IEEE Conf. Acoust. Spee. 1, 753–756 (2007).
    [Crossref]
  26. Y. R. Li and D. Q. Dai, “Color superresolution reconstruction and demosaicing using elastic net and tight frame,” IEEE Circuits-I 55(11), 3500–3512 (2008).
    [Crossref]
  27. M. Trimeche, “Color demosaicing using multi-frame super-resolution,” Eur. Signal Process. Conf. (2008).
  28. D. A. Sorrentino and A. Antoniou, “Improved hybrid demosaicing and color super-resolution implementation using quasi-Newton algorithms,” Can. Con. El. Comp. En. 815–818 (2009).
  29. P. Vandewalle, K. Krichane, D. Alleysson, and S. Süsstrunk, “Joint demosaicing and super-resolution imaging from a set of unregistered aliased images” Proc. SPIE IS&T Elect. Im. 65020A (2007).
  30. T. Heinze, M. von Lowis, and A. Polze, “Joint multi-frame demosaicing and super-resolution with artificial neural networks,” IWSSIP 2012, 540–543 (2012).
  31. M. Shao, K. E. Barner, and R. C. Hardie, “Partition-based interpolation for color filter array demosaicking and super-resolution reconstruction,” Opt. Eng. 44(10), 107003 (2005).
    [Crossref]
  32. M. T. Eismann, Hyperspectral Remote Sensing (SPIE, 2012).
  33. G. D. Boreman, Modulation Transfer Function in Optical and Electro-Optical Systems (SPIE, 2001).
  34. R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37(1), 247–260 (1998).
    [Crossref]
  35. A. Oppenheim and R. Schafer, Discrete-Time Signal Processing (Prentice Hall, 1989).
  36. C. W. Therrien, Discrete Random Signals and Statistical Signal Processing (Prentice Hall PTR, 1992).
  37. A. K. Jain, Fundamentals of Digital Image Processing (Prentice Hall, 1989).
  38. P. Milanfar, “MDSP resolution enhancement software,” Copyright 2009 by University of California. http://users.soe.ucsc.edu/~milanfar/software/superresolution.html .

2012 (1)

T. Heinze, M. von Lowis, and A. Polze, “Joint multi-frame demosaicing and super-resolution with artificial neural networks,” IWSSIP 2012, 540–543 (2012).

2011 (2)

2008 (2)

X. Li, B. Gunturk, and L. Zhang, “Image demosaicing: a systematic survey,” Proc. SPIE 6822, 68221J, 68221J-15 (2008).
[Crossref]

Y. R. Li and D. Q. Dai, “Color superresolution reconstruction and demosaicing using elastic net and tight frame,” IEEE Circuits-I 55(11), 3500–3512 (2008).
[Crossref]

2007 (2)

B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, “A computationally efficient super-resolution algorithm for video processing using partition filters,” IEEE Circ. Syst. Vid. 17(5), 621–634 (2007).
[Crossref]

R. C. Hardie, “A fast image super-resolution algorithm using an adaptive Wiener filter,” IEEE Trans. Image Process. 16(12), 2953–2964 (2007).
[Crossref] [PubMed]

2006 (2)

J. Shi, S. E. Reichenbach, and J. D. Howe, “Small-kernel superresolution methods for microscanning imaging systems,” Appl. Opt. 45(6), 1203–1214 (2006).
[Crossref] [PubMed]

S. Farsiu, M. Elad, and P. Milanfar, “Multiframe demosaicing and super-resolution of color images,” IEEE Trans. Image Proc. 15(1), 141–159 (2006).
[Crossref] [PubMed]

2005 (3)

R. Sasahara, H. Hasegawa, I. Yamada, and K. Sakaniwa, “A color super-resolution with multiple nonsmooth constraints by hybrid steepest descent method,” IEEE Trans. Image Proc. 1, 857–860 (2005).

M. Shao, K. E. Barner, and R. C. Hardie, “Partition-based interpolation for color filter array demosaicking and super-resolution reconstruction,” Opt. Eng. 44(10), 107003 (2005).
[Crossref]

B. K. Gunturk, J. Glotzbach, Y. Altunbasak, R. W. Schafer, and R. M. Mersereau, “Demosaicking: color filter array interpolation,” IEEE Signal Proc. Mag. 22(1), 44–54 (2005).
[Crossref]

2004 (1)

M. Shimizu, T. Yano, and M. Okutomi, “Super-resolution under image deformation,” Int. C. Patt. Recog. 3, 586–589 (2004).

2003 (1)

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003).
[Crossref]

2002 (2)

S. Lertrattanapanich and N. K. Bose, “High resolution image formation from low resolution frames using Delaunay triangulation,” IEEE Trans. Image Process. 11(12), 1427–1441 (2002).
[Crossref] [PubMed]

R. Ramanath, W. Snyder, G. Bilbro, and W. Sander, “Demosaicking methods for the Bayer color arrays,” J. Electron. Imaging 11(3), 306–315 (2002).
[Crossref]

2001 (1)

M. Elad and Y. Hel-Or, “A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur,” IEEE Trans. Image Process. 10(8), 1187–1193 (2001).
[Crossref] [PubMed]

2000 (1)

M. Alam, M. S. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49(5), 915–923 (2000).
[Crossref]

1999 (2)

1998 (1)

R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37(1), 247–260 (1998).
[Crossref]

1995 (1)

J. C. Gillette, T. M. Stadtmiller, and R. C. Hardie, “Aliasing reduction in staring infrared imagers utilizing subpixel techniques,” Opt. Eng. 34(11), 3130–3137 (1995).
[Crossref]

1991 (1)

K. Aizawa, T. Komatsu, and T. Saito, “Acquisition of very high resolution images using stereo cameras,” P. Soc. Photo-Opt. Ins. 1605, 318–328 (1991).

Aizawa, K.

K. Aizawa, T. Komatsu, and T. Saito, “Acquisition of very high resolution images using stereo cameras,” P. Soc. Photo-Opt. Ins. 1605, 318–328 (1991).

Alam, M.

M. Alam, M. S. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49(5), 915–923 (2000).
[Crossref]

Altunbasak, Y.

B. K. Gunturk, J. Glotzbach, Y. Altunbasak, R. W. Schafer, and R. M. Mersereau, “Demosaicking: color filter array interpolation,” IEEE Signal Proc. Mag. 22(1), 44–54 (2005).
[Crossref]

Armstrong, E. E.

R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37(1), 247–260 (1998).
[Crossref]

Barnard, K. J.

R. C. Hardie, K. J. Barnard, and R. Ordonez, “Fast super-resolution with affine motion using an adaptive Wiener filter and its application to airborne imaging,” Opt. Express 19(27), 26208–26231 (2011).
[Crossref] [PubMed]

R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37(1), 247–260 (1998).
[Crossref]

Barner, K. E.

B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, “A computationally efficient super-resolution algorithm for video processing using partition filters,” IEEE Circ. Syst. Vid. 17(5), 621–634 (2007).
[Crossref]

M. Shao, K. E. Barner, and R. C. Hardie, “Partition-based interpolation for color filter array demosaicking and super-resolution reconstruction,” Opt. Eng. 44(10), 107003 (2005).
[Crossref]

Bilbro, G.

R. Ramanath, W. Snyder, G. Bilbro, and W. Sander, “Demosaicking methods for the Bayer color arrays,” J. Electron. Imaging 11(3), 306–315 (2002).
[Crossref]

Bognar, J. G.

R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37(1), 247–260 (1998).
[Crossref]

Bognar, M. S.

M. Alam, M. S. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49(5), 915–923 (2000).
[Crossref]

Bose, N. K.

S. Lertrattanapanich and N. K. Bose, “High resolution image formation from low resolution frames using Delaunay triangulation,” IEEE Trans. Image Process. 11(12), 1427–1441 (2002).
[Crossref] [PubMed]

Chellappa, R.

Dai, D. Q.

Y. R. Li and D. Q. Dai, “Color superresolution reconstruction and demosaicing using elastic net and tight frame,” IEEE Circuits-I 55(11), 3500–3512 (2008).
[Crossref]

Elad, M.

S. Farsiu, M. Elad, and P. Milanfar, “Multiframe demosaicing and super-resolution of color images,” IEEE Trans. Image Proc. 15(1), 141–159 (2006).
[Crossref] [PubMed]

M. Elad and Y. Hel-Or, “A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur,” IEEE Trans. Image Process. 10(8), 1187–1193 (2001).
[Crossref] [PubMed]

Farsiu, S.

S. Farsiu, M. Elad, and P. Milanfar, “Multiframe demosaicing and super-resolution of color images,” IEEE Trans. Image Proc. 15(1), 141–159 (2006).
[Crossref] [PubMed]

Fiete, R. D.

R. D. Fiete, “Image quality and λFN/p for remote sensing systems,” Opt. Eng. 38(7), 1229–1240 (1999).
[Crossref]

Gillette, J. C.

J. C. Gillette, T. M. Stadtmiller, and R. C. Hardie, “Aliasing reduction in staring infrared imagers utilizing subpixel techniques,” Opt. Eng. 34(11), 3130–3137 (1995).
[Crossref]

Glotzbach, J.

B. K. Gunturk, J. Glotzbach, Y. Altunbasak, R. W. Schafer, and R. M. Mersereau, “Demosaicking: color filter array interpolation,” IEEE Signal Proc. Mag. 22(1), 44–54 (2005).
[Crossref]

Gotoh, T.

T. Gotoh and M. Okutomi, “Direct super-resolution and registration using raw CFA images,” Proc. CVPR IEEE2, 600–607 (2004).
[Crossref]

Gunturk, B.

X. Li, B. Gunturk, and L. Zhang, “Image demosaicing: a systematic survey,” Proc. SPIE 6822, 68221J, 68221J-15 (2008).
[Crossref]

Gunturk, B. K.

B. K. Gunturk, J. Glotzbach, Y. Altunbasak, R. W. Schafer, and R. M. Mersereau, “Demosaicking: color filter array interpolation,” IEEE Signal Proc. Mag. 22(1), 44–54 (2005).
[Crossref]

Hardie, R. C.

R. C. Hardie, K. J. Barnard, and R. Ordonez, “Fast super-resolution with affine motion using an adaptive Wiener filter and its application to airborne imaging,” Opt. Express 19(27), 26208–26231 (2011).
[Crossref] [PubMed]

R. C. Hardie, D. A. LeMaster, and B. M. Ratliff, “Super-resolution for imagery from integrated microgrid polarimeters,” Opt. Express 19(14), 12937–12960 (2011).
[Crossref] [PubMed]

B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, “A computationally efficient super-resolution algorithm for video processing using partition filters,” IEEE Circ. Syst. Vid. 17(5), 621–634 (2007).
[Crossref]

R. C. Hardie, “A fast image super-resolution algorithm using an adaptive Wiener filter,” IEEE Trans. Image Process. 16(12), 2953–2964 (2007).
[Crossref] [PubMed]

M. Shao, K. E. Barner, and R. C. Hardie, “Partition-based interpolation for color filter array demosaicking and super-resolution reconstruction,” Opt. Eng. 44(10), 107003 (2005).
[Crossref]

M. Alam, M. S. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49(5), 915–923 (2000).
[Crossref]

R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37(1), 247–260 (1998).
[Crossref]

J. C. Gillette, T. M. Stadtmiller, and R. C. Hardie, “Aliasing reduction in staring infrared imagers utilizing subpixel techniques,” Opt. Eng. 34(11), 3130–3137 (1995).
[Crossref]

Hasegawa, H.

R. Sasahara, H. Hasegawa, I. Yamada, and K. Sakaniwa, “A color super-resolution with multiple nonsmooth constraints by hybrid steepest descent method,” IEEE Trans. Image Proc. 1, 857–860 (2005).

Heinze, T.

T. Heinze, M. von Lowis, and A. Polze, “Joint multi-frame demosaicing and super-resolution with artificial neural networks,” IWSSIP 2012, 540–543 (2012).

Hel-Or, Y.

M. Elad and Y. Hel-Or, “A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur,” IEEE Trans. Image Process. 10(8), 1187–1193 (2001).
[Crossref] [PubMed]

Howe, J. D.

Kang, M. G.

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003).
[Crossref]

Komatsu, T.

K. Aizawa, T. Komatsu, and T. Saito, “Acquisition of very high resolution images using stereo cameras,” P. Soc. Photo-Opt. Ins. 1605, 318–328 (1991).

LeMaster, D. A.

Lertrattanapanich, S.

S. Lertrattanapanich and N. K. Bose, “High resolution image formation from low resolution frames using Delaunay triangulation,” IEEE Trans. Image Process. 11(12), 1427–1441 (2002).
[Crossref] [PubMed]

Li, X.

X. Li, B. Gunturk, and L. Zhang, “Image demosaicing: a systematic survey,” Proc. SPIE 6822, 68221J, 68221J-15 (2008).
[Crossref]

Li, Y. R.

Y. R. Li and D. Q. Dai, “Color superresolution reconstruction and demosaicing using elastic net and tight frame,” IEEE Circuits-I 55(11), 3500–3512 (2008).
[Crossref]

Mersereau, R. M.

B. K. Gunturk, J. Glotzbach, Y. Altunbasak, R. W. Schafer, and R. M. Mersereau, “Demosaicking: color filter array interpolation,” IEEE Signal Proc. Mag. 22(1), 44–54 (2005).
[Crossref]

Milanfar, P.

S. Farsiu, M. Elad, and P. Milanfar, “Multiframe demosaicing and super-resolution of color images,” IEEE Trans. Image Proc. 15(1), 141–159 (2006).
[Crossref] [PubMed]

Narayanan, B.

B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, “A computationally efficient super-resolution algorithm for video processing using partition filters,” IEEE Circ. Syst. Vid. 17(5), 621–634 (2007).
[Crossref]

Okutomi, M.

M. Shimizu, T. Yano, and M. Okutomi, “Super-resolution under image deformation,” Int. C. Patt. Recog. 3, 586–589 (2004).

T. Gotoh and M. Okutomi, “Direct super-resolution and registration using raw CFA images,” Proc. CVPR IEEE2, 600–607 (2004).
[Crossref]

Ordonez, R.

Ozkan, M. K.

A. M. Tekalp, M. K. Ozkan, and M. I. Sezan, “High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration,” Int. Conf. Acoust Spee.3, 169–172 (1992).
[Crossref]

Park, M. K.

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003).
[Crossref]

Park, S. C.

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003).
[Crossref]

Polze, A.

T. Heinze, M. von Lowis, and A. Polze, “Joint multi-frame demosaicing and super-resolution with artificial neural networks,” IWSSIP 2012, 540–543 (2012).

Ramanath, R.

R. Ramanath, W. Snyder, G. Bilbro, and W. Sander, “Demosaicking methods for the Bayer color arrays,” J. Electron. Imaging 11(3), 306–315 (2002).
[Crossref]

Ratliff, B. M.

Reichenbach, S. E.

Saito, T.

K. Aizawa, T. Komatsu, and T. Saito, “Acquisition of very high resolution images using stereo cameras,” P. Soc. Photo-Opt. Ins. 1605, 318–328 (1991).

Sakaniwa, K.

R. Sasahara, H. Hasegawa, I. Yamada, and K. Sakaniwa, “A color super-resolution with multiple nonsmooth constraints by hybrid steepest descent method,” IEEE Trans. Image Proc. 1, 857–860 (2005).

Sander, W.

R. Ramanath, W. Snyder, G. Bilbro, and W. Sander, “Demosaicking methods for the Bayer color arrays,” J. Electron. Imaging 11(3), 306–315 (2002).
[Crossref]

Sasahara, R.

R. Sasahara, H. Hasegawa, I. Yamada, and K. Sakaniwa, “A color super-resolution with multiple nonsmooth constraints by hybrid steepest descent method,” IEEE Trans. Image Proc. 1, 857–860 (2005).

Schafer, R. W.

B. K. Gunturk, J. Glotzbach, Y. Altunbasak, R. W. Schafer, and R. M. Mersereau, “Demosaicking: color filter array interpolation,” IEEE Signal Proc. Mag. 22(1), 44–54 (2005).
[Crossref]

Sezan, M. I.

A. M. Tekalp, M. K. Ozkan, and M. I. Sezan, “High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration,” Int. Conf. Acoust Spee.3, 169–172 (1992).
[Crossref]

Shao, M.

B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, “A computationally efficient super-resolution algorithm for video processing using partition filters,” IEEE Circ. Syst. Vid. 17(5), 621–634 (2007).
[Crossref]

M. Shao, K. E. Barner, and R. C. Hardie, “Partition-based interpolation for color filter array demosaicking and super-resolution reconstruction,” Opt. Eng. 44(10), 107003 (2005).
[Crossref]

Shekarforoush, H.

Shi, J.

Shimizu, M.

M. Shimizu, T. Yano, and M. Okutomi, “Super-resolution under image deformation,” Int. C. Patt. Recog. 3, 586–589 (2004).

Snyder, W.

R. Ramanath, W. Snyder, G. Bilbro, and W. Sander, “Demosaicking methods for the Bayer color arrays,” J. Electron. Imaging 11(3), 306–315 (2002).
[Crossref]

Stadtmiller, T. M.

J. C. Gillette, T. M. Stadtmiller, and R. C. Hardie, “Aliasing reduction in staring infrared imagers utilizing subpixel techniques,” Opt. Eng. 34(11), 3130–3137 (1995).
[Crossref]

Tekalp, A. M.

A. M. Tekalp, M. K. Ozkan, and M. I. Sezan, “High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration,” Int. Conf. Acoust Spee.3, 169–172 (1992).
[Crossref]

Trimeche, M.

M. Trimeche, “Color demosaicing using multi-frame super-resolution,” Eur. Signal Process. Conf. (2008).

von Lowis, M.

T. Heinze, M. von Lowis, and A. Polze, “Joint multi-frame demosaicing and super-resolution with artificial neural networks,” IWSSIP 2012, 540–543 (2012).

Watson, E. A.

R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37(1), 247–260 (1998).
[Crossref]

Yamada, I.

R. Sasahara, H. Hasegawa, I. Yamada, and K. Sakaniwa, “A color super-resolution with multiple nonsmooth constraints by hybrid steepest descent method,” IEEE Trans. Image Proc. 1, 857–860 (2005).

Yano, T.

M. Shimizu, T. Yano, and M. Okutomi, “Super-resolution under image deformation,” Int. C. Patt. Recog. 3, 586–589 (2004).

Yasuda, B. J.

M. Alam, M. S. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49(5), 915–923 (2000).
[Crossref]

Zhang, L.

X. Li, B. Gunturk, and L. Zhang, “Image demosaicing: a systematic survey,” Proc. SPIE 6822, 68221J, 68221J-15 (2008).
[Crossref]

Appl. Opt. (1)

IEEE Circ. Syst. Vid. (1)

B. Narayanan, R. C. Hardie, K. E. Barner, and M. Shao, “A computationally efficient super-resolution algorithm for video processing using partition filters,” IEEE Circ. Syst. Vid. 17(5), 621–634 (2007).
[Crossref]

IEEE Circuits-I (1)

Y. R. Li and D. Q. Dai, “Color superresolution reconstruction and demosaicing using elastic net and tight frame,” IEEE Circuits-I 55(11), 3500–3512 (2008).
[Crossref]

IEEE Signal Proc. Mag. (1)

B. K. Gunturk, J. Glotzbach, Y. Altunbasak, R. W. Schafer, and R. M. Mersereau, “Demosaicking: color filter array interpolation,” IEEE Signal Proc. Mag. 22(1), 44–54 (2005).
[Crossref]

IEEE Signal Process. Mag. (1)

S. C. Park, M. K. Park, and M. G. Kang, “Super-resolution image reconstruction: a technical overview,” IEEE Signal Process. Mag. 20(3), 21–36 (2003).
[Crossref]

IEEE Trans. Image Proc. (2)

R. Sasahara, H. Hasegawa, I. Yamada, and K. Sakaniwa, “A color super-resolution with multiple nonsmooth constraints by hybrid steepest descent method,” IEEE Trans. Image Proc. 1, 857–860 (2005).

S. Farsiu, M. Elad, and P. Milanfar, “Multiframe demosaicing and super-resolution of color images,” IEEE Trans. Image Proc. 15(1), 141–159 (2006).
[Crossref] [PubMed]

IEEE Trans. Image Process. (3)

R. C. Hardie, “A fast image super-resolution algorithm using an adaptive Wiener filter,” IEEE Trans. Image Process. 16(12), 2953–2964 (2007).
[Crossref] [PubMed]

S. Lertrattanapanich and N. K. Bose, “High resolution image formation from low resolution frames using Delaunay triangulation,” IEEE Trans. Image Process. 11(12), 1427–1441 (2002).
[Crossref] [PubMed]

M. Elad and Y. Hel-Or, “A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur,” IEEE Trans. Image Process. 10(8), 1187–1193 (2001).
[Crossref] [PubMed]

IEEE Trans. Instrum. Meas. (1)

M. Alam, M. S. Bognar, R. C. Hardie, and B. J. Yasuda, “Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames,” IEEE Trans. Instrum. Meas. 49(5), 915–923 (2000).
[Crossref]

Int. C. Patt. Recog. (1)

M. Shimizu, T. Yano, and M. Okutomi, “Super-resolution under image deformation,” Int. C. Patt. Recog. 3, 586–589 (2004).

IWSSIP (1)

T. Heinze, M. von Lowis, and A. Polze, “Joint multi-frame demosaicing and super-resolution with artificial neural networks,” IWSSIP 2012, 540–543 (2012).

J. Electron. Imaging (1)

R. Ramanath, W. Snyder, G. Bilbro, and W. Sander, “Demosaicking methods for the Bayer color arrays,” J. Electron. Imaging 11(3), 306–315 (2002).
[Crossref]

J. Opt. Soc. Am. A (1)

Opt. Eng. (4)

R. D. Fiete, “Image quality and λFN/p for remote sensing systems,” Opt. Eng. 38(7), 1229–1240 (1999).
[Crossref]

J. C. Gillette, T. M. Stadtmiller, and R. C. Hardie, “Aliasing reduction in staring infrared imagers utilizing subpixel techniques,” Opt. Eng. 34(11), 3130–3137 (1995).
[Crossref]

M. Shao, K. E. Barner, and R. C. Hardie, “Partition-based interpolation for color filter array demosaicking and super-resolution reconstruction,” Opt. Eng. 44(10), 107003 (2005).
[Crossref]

R. C. Hardie, K. J. Barnard, J. G. Bognar, E. E. Armstrong, and E. A. Watson, “High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system,” Opt. Eng. 37(1), 247–260 (1998).
[Crossref]

Opt. Express (2)

P. Soc. Photo-Opt. Ins. (1)

K. Aizawa, T. Komatsu, and T. Saito, “Acquisition of very high resolution images using stereo cameras,” P. Soc. Photo-Opt. Ins. 1605, 318–328 (1991).

Proc. SPIE (1)

X. Li, B. Gunturk, and L. Zhang, “Image demosaicing: a systematic survey,” Proc. SPIE 6822, 68221J, 68221J-15 (2008).
[Crossref]

Other (15)

B. E. Bayer, “Color imaging array,” Eastman Kodak Co., U.S. Patent 3,971,065 (Jul 20 1976).

P. Milanfar, Super-Resolution Imaging (CRC, 2010).

A. M. Tekalp, M. K. Ozkan, and M. I. Sezan, “High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration,” Int. Conf. Acoust Spee.3, 169–172 (1992).
[Crossref]

T. Q. Pham, L. J. Van Vliet, and K. Schutte, “Robust fusion of irregularly sampled data using adaptive normalized convolution,” EURASIP J. Adv. Sig. Pr. 1–12 (2006).

A. Oppenheim and R. Schafer, Discrete-Time Signal Processing (Prentice Hall, 1989).

C. W. Therrien, Discrete Random Signals and Statistical Signal Processing (Prentice Hall PTR, 1992).

A. K. Jain, Fundamentals of Digital Image Processing (Prentice Hall, 1989).

P. Milanfar, “MDSP resolution enhancement software,” Copyright 2009 by University of California. http://users.soe.ucsc.edu/~milanfar/software/superresolution.html .

M. T. Eismann, Hyperspectral Remote Sensing (SPIE, 2012).

G. D. Boreman, Modulation Transfer Function in Optical and Electro-Optical Systems (SPIE, 2001).

T. Gotoh and M. Okutomi, “Direct super-resolution and registration using raw CFA images,” Proc. CVPR IEEE2, 600–607 (2004).
[Crossref]

M. Trimeche, “Color demosaicing using multi-frame super-resolution,” Eur. Signal Process. Conf. (2008).

D. A. Sorrentino and A. Antoniou, “Improved hybrid demosaicing and color super-resolution implementation using quasi-Newton algorithms,” Can. Con. El. Comp. En. 815–818 (2009).

P. Vandewalle, K. Krichane, D. Alleysson, and S. Süsstrunk, “Joint demosaicing and super-resolution imaging from a set of unregistered aliased images” Proc. SPIE IS&T Elect. Im. 65020A (2007).

M. Gevrekci, B. K. Gunturk, and Y. Altunbasak, “POCS-based restoration of Bayer-sampled image sequences,” IEEE Conf. Acoust. Spee. 1, 753–756 (2007).
[Crossref]

Supplementary Material (1)

» Media 1: MOV (4486 KB)     

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (16)

Fig. 1
Fig. 1 Bayer CFA showing color channel and fundamental detector sampling.

Download Full Size | PPT Slide | PDF

Fig. 2
Fig. 2 Forward observation model relating the desired continuous image to the observed data.

Download Full Size | PPT Slide | PDF

Fig. 3
Fig. 3 Alternate forward observation model combining frame motion and “mosaic” sampling.

Download Full Size | PPT Slide | PDF

Fig. 4
Fig. 4 MTF of the example detector, optical diffraction, and total system.

Download Full Size | PPT Slide | PDF

Fig. 5
Fig. 5 Example observation & estimation windows for 3 LR frames of 4-channel DoFP sensor.

Download Full Size | PPT Slide | PDF

Fig. 6
Fig. 6 Examples of the continuous (a) desired autocorrelation, (b) noise-free observed autocorrelation, and (c) cross-correlation models.

Download Full Size | PPT Slide | PDF

Fig. 7
Fig. 7 Image data used in simulation experiment #1. (a) Full-color (24-bit) high resolution image, (b) sample simulated LR Bayer CFA image, and (c) sample red color channel image.

Download Full Size | PPT Slide | PDF

Fig. 8
Fig. 8 Simulated data results for (a) original HR image, (b) demosaic/interpolation, (c) Shift & Add output, (d) MDSP output, (e) RLS with independent color channel regularization, (f) independent color channel AWF SR, (g) 12-parameter 3-PSF color AWF SR, (h) 6-parameter 1-PSF color AWF SR, (i) 3-parameter 1-PSF color AWF SR.

Download Full Size | PPT Slide | PDF

Fig. 9
Fig. 9 Region of interest from results in Fig. 9. (a) original HR image, (b) demosaic/interpolation, (c) Shift & Add output, (d) MDSP output, (e) RLS with independent color channel regularization, (f) independent color channel AWF SR, (g) 12-parameter 3-PSF color AWF SR, (h) 6-parameter 1-PSF color AWF SR, (i) 3-parameter 1-PSF color AWF SR.

Download Full Size | PPT Slide | PDF

Fig. 10
Fig. 10 Region of interest for simulated results using 4 LR input frames. (a) independent color channel AWF SR, (b) 12-parameter 3-PSF color AWF SR.

Download Full Size | PPT Slide | PDF

Fig. 11
Fig. 11 True shifts, AWF shift estimates, and quantized shift estimates in experiment #2.

Download Full Size | PPT Slide | PDF

Fig. 12
Fig. 12 HR image used in simulation experiment #3 variation SR performance comparison.

Download Full Size | PPT Slide | PDF

Fig. 13
Fig. 13 ROI from Fig. 13. (a) input HR image for simulation, (b) 6-parameter 1-PSF color AWF SR, (c) RLS initialized with interpolated LR input image, (d) RLS initialized with color AWF SR result.

Download Full Size | PPT Slide | PDF

Fig. 14
Fig. 14 Sample input from real data experiment. (a) sample LR Bayer CFA image, (b) red color channel from sample LR Bayer CFA image.

Download Full Size | PPT Slide | PDF

Fig. 15
Fig. 15 Chirp resolution chart ROI. (a) demosaic/interpolation (Media 1 left), (b) Shift & Add output, (c) MDSP output, (d) RLS with independent channel regularization, (e) 6-parameter 1-PSF color AWF SR (Media 1 right), (f) RLS initialized with color AWF SR result.

Download Full Size | PPT Slide | PDF

Fig. 16
Fig. 16 Textbooks ROI from real data experiment. (a) demosaic/interpolation (Media 1 left), (b) Shift & Add output, (c) MDSP output, (d) RLS with independent color channel regularization, (e) 6-parameter 1-PSF color AWF SR (Media 1 right), (f) RLS initialized with color AWF SR result.

Download Full Size | PPT Slide | PDF

Tables (6)

Tables Icon

Table 1 PSF parameters for simulation experiment #1

View Table | View all tables in this article
Tables Icon

Table 2 Model parameters for color AWF methods

View Table | View all tables in this article
Tables Icon

Table 3 Performance results for simulation experiment #1

View Table | View all tables in this article
Tables Icon

Table 4 Performance results for simulation experiment #2, randomly shifted input

View Table | View all tables in this article
Tables Icon

Table 5 Performance results for simulation experiment #2, quantized input shifts

View Table | View all tables in this article
Tables Icon

Table 6 Performance results for simulation experiment #3

View Table | View all tables in this article

Equations (22)

Equations on this page are rendered with MathJax. Learn more.

h d (x,y)= 1 ab rect( x a , y b ),
H d (u,v)=sinc(au,bv)= sin(πau)sin(πbv) π 2 (au)(bv) ,
H o (u,v)={ 2 π [ cos −1 ( ρ ρ c )− ρ ρ c 1− ( ρ ρ c ) 2 ],ρ≤ ρ c 0, otherwise ,
ρ c = 1 λf/# (cyc/mm).
1 δ s ≥2 ρ c ,
K= P W x W y L 2 , ∀i.
d ^ i = W i T g i .
W i = R i −1 P i .
R i =E{ f i f i T }+ σ n 2 I,
P i =E{ f i d i T }.
r f j f k (x,y)= r d j d k (x,y)* h j (x,y)* h k (−x,−y).
r d j f k (x,y)= r d j d k (x,y)* h k (x,y).
r d k f j (x,y)= r d j d k (x,y)* h j (x,y).
r d j d k (x,y)= σ j,k 2 ρ j,k x 2 + y 2 ,
ρ j,k =ρ.
σ j,k 2 ={ σ corr 2 , j=k α σ corr 2 , j≠k .
ρ j,k ={ ρ corr , j=k ρ r,g , j=R, k=G ρ r,b , j=R, k=B ρ g,b , j=G, k=B .
R i =E{ g i g i T }=E{( f i + n i ) ( f i + n i ) T } =E{( f i + n i )( f i T + n i T )} =E{ f i f i T }+E{ n i f i T }+E{ f i n i T }+E{ n i n i T } R i =E{ f i f i T }+ σ n 2 I
r f j f k (x,y)=E{ f j (α,β) f k (α+x,β+y) } =E{ [ d j (α,β)* h j (α,β) ][ d k (α+x,β+y)* h k (α+x,β+y) ] } =E{ [ ∫ −∞ ∞ ∫ −∞ ∞ d j (α−γ,β−η) h j (γ,η)dγdη ][ ∫ −∞ ∞ ∫ −∞ ∞ d k (α+x−θ,β+y−ϕ) h k (θ,ϕ)dθdϕ ] } =E{ [ ∫ −∞ ∞ ∫ −∞ ∞ d j (α−γ,β−η) h j (γ,η)[ ∫ −∞ ∞ ∫ −∞ ∞ d k (α+x−θ,β+y−ϕ) h k (θ,ϕ)dθdϕ ]dγdη ] } = ∫ −∞ ∞ ∫ −∞ ∞ [ ∫ −∞ ∞ ∫ −∞ ∞ E{ d j (α−γ,β−η) d k (α+x−θ,β+y−ϕ) } h k (θ,ϕ)dθdϕ ] h j (γ,η)dγdη = ∫ −∞ ∞ ∫ −∞ ∞ [ ∫ −∞ ∞ ∫ −∞ ∞ r d j d k (x+γ−θ,y+η−ϕ) h k (θ,ϕ)dθdϕ ] h j (γ,η)dγdη (let ξ=θ−γ, ψ=ϕ-η) r f j f k (x,y)= ∫ −∞ ∞ ∫ −∞ ∞ [ ∫ −∞ ∞ ∫ −∞ ∞ r d j d k (x−ξ,y−ψ) h k (γ+ξ,η+ψ)dξdψ ] h j (γ,η)dγdη = ∫ −∞ ∞ ∫ −∞ ∞ r d j d k (x−ξ,y−ψ)[ ∫ −∞ ∞ ∫ −∞ ∞ h j (γ,η) h k (γ+ξ,η+ψ)dγdη ]dξdψ = ∫ −∞ ∞ ∫ −∞ ∞ r d j d k (x−ξ,y−ψ)[ h j (ξ,ψ)∗ h k (−ξ,−ψ) ]dξdψ r f j f k (x,y)= r d j d k (x,y)*[ h j (x,y)* h k (−x,−y) ],
P i =E{ g i d i T } =E{( f i + n i ) d i T } =E{ f i d i T }+E{ n i d i T } P i =E{ f i d i T }
r d j f k (x,y)=E{ d j (α,β) f k (α+x,β+y) } =E{ d j (α,β)[ d k (α+x,β+y)* h k (α+x,β+y) ] } =E{ d j (α,β)[ ∫ −∞ ∞ ∫ −∞ ∞ d k (α+x−γ,β+y−η) h k (γ,η)dγdη ] } = ∫ −∞ ∞ ∫ −∞ ∞ E{ d j (α,β) d k (α+x−γ,β+y−η) h k (γ,η) }dγdη = ∫ −∞ ∞ ∫ −∞ ∞ h k (γ,η)E{ d j (α,β) d k (α+x−γ,β+y−η) }dγdη = ∫ −∞ ∞ ∫ −∞ ∞ h k (γ,η) r d j d k (x−γ,y−η)dγdη r d j f k (x,y)= r d j d k (x,y)* h k (x,y).
r d k f j (x,y)=E{ d k (α,β) f j (α+x,β+y) } =E{ d k (α,β)[ d j (α+x,β+y)* h j (α+x,β+y) ] } =E{ d k (α,β)[ ∫ −∞ ∞ ∫ −∞ ∞ d j (α+x−γ,β+y−η) h j (γ,η)dγdη ] } = ∫ −∞ ∞ ∫ −∞ ∞ E{ d k (α,β) d j (α+x−γ,β+y−η) h j (γ,η) }dγdη = ∫ −∞ ∞ ∫ −∞ ∞ h j (γ,η)E{ d k (α,β) d j (α+x−γ,β+y−η) }dγdη = ∫ −∞ ∞ ∫ −∞ ∞ h j (γ,η) r d k d j (x−γ,y−η)dγdη = r d k d j (x,y)* h j (x,y) r d k f j (x,y)= r d j d k (x,y)* h j (x,y)

Metrics