Accelerating decomposition of light field video for compressive multi-layer display

Xuan Cao; Zheng Geng; Tuotuo Li; Mei Zhang; Zhaoxing Zhang

doi:10.1364/OE.23.034007

Accelerating decomposition of light field video for compressive multi-layer display

Xuan Cao, Zheng Geng, Tuotuo Li, Mei Zhang, and Zhaoxing Zhang

Optics Express
Vol. 23,
Issue 26,
pp. 34007-34022
(2015)
•https://doi.org/10.1364/OE.23.034007

Get Citation
Copy Citation Text
Xuan Cao, Zheng Geng, Tuotuo Li, Mei Zhang, and Zhaoxing Zhang, "Accelerating decomposition of light field video for compressive multi-layer display," Opt. Express 23, 34007-34022 (2015)

Export Citation
- BibTex
- Endnote (RIS)
- HTML
- Plain Text
Get PDF (10140 KB)
Set citation alerts
Save article

Related Topics
Table of Contents Category
- Imaging Systems and Displays
Optics & Photonics Topics
?

The topics in this list come from the OSA Optics and Photonics Topics applied to this article.
Previously assigned OCIS codes
- Spatial light modulators (070.6120)
- Tomographic image processing (100.6950)
- Computational imaging (110.1758)
- Displays (120.2040)

About this Article
History
- Original Manuscript: October 15, 2015
- Revised Manuscript: December 21, 2015
- Manuscript Accepted: December 22, 2015
- Published: December 24, 2015

Accessible

Open Access

Abstract

Compressive light field display based on multi-layer LCDs is becoming a popular solution for 3D display. Decomposing light field into layer images is the most challenging task. Iterative algorithm is an effective solver for this high-dimensional decomposition problem. Existing algorithms, however, iterate from random initial values. As such, significant computation time is required due to the deviation between random initial estimate and target values. Real-time 3D display at video rate is difficult based on existing algorithms. In this paper, we present a new algorithm to provide better initial values and accelerate decomposition of light field video. We utilize internal coherence of single light field frame to transfer the ignorance-to-target to a much lower resolution level. In addition, we explored external coherence for further accelerating light field video and achieved 5.91 times speed improvement. We built a prototype and developed parallel algorithm based on CUDA.

Full Article | PDF Article

OSA Recommended Articles

Hybrid multi-layer displays providing accommodation cues

Dongyeon Kim, Seungjae Lee, Seokil Moon, Jaebum Cho, Youngjin Jo, and Byoungho Lee
Opt. Express 26(13) 17170-17184 (2018)

Demonstration of a low-crosstalk super multi-view light field display with natural depth cues and smooth motion parallax

Peiren Wang, Xinzhu Sang, Xunbo Yu, Xin Gao, Binbin Yan, Boyang Liu, Li Liu, Chao Gao, Yang Le, Yuanhang Li, and Jingyan Du
Opt. Express 27(23) 34442-34453 (2019)

Depth enhancement of multi-layer light field display using polarization dependent internal reflection

Na-Young Jo, Hong-Gi Lim, Sung-Keun Lee, Yong-Soo Kim, and Jae-Hyeung Park
Opt. Express 21(24) 29628-29636 (2013)

References

View by:
Article Order
|
Year
|
Author
|
Publication

J. Geng, “Three-dimensional display technologies,” Adv. Opt. Photonics 5(4), 456–535 (2013).
[Crossref] [PubMed]
A. Jones, I. Mcdowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]
A. Jones, M. Lang, G. Fyffe, X. M. Yu, J. Busch, I. McDowall, M. Bolas, and P. Debevec, “Achieving eye contact in a one-to-many 3D video teleconferencing system,” ACM Trans. Graph. 28(3), 64 (2009).
[Crossref]
J. Geng, “A volumetric 3D display based on a DLP projection engine,” Displays 34(1), 39–48 (2013).
[Crossref]
C. Van Berkel, “Image preparation for 3D-LCD,” Proc. SPIE 3639, 84–91 (1999).
[Crossref]
C. Van Berkel and J. A. Clarke, “Characterisation and optimisation of 3D-LCD module design,” Proc. SPIE 3012, 179–186 (1997).
[Crossref]
Z. X. Zhang, Z. Geng, M. Zhang, and H. Dong, “An interactive multiview 3D display system,” Proc. SPIE 8618, 86180P (2013).
[Crossref]
A. Jones, K. Nagano, J. Liu, J. Busch, X. M. Yu, M. Bolas, and P. Debevec, “Interpolating vertical parallax for an autostereoscopic three-dimensional projector array,” J. Electron. Imaging 23(1), 011005 (2014).
[Crossref]
E. H. Adelson and J. R. Bergen, “The plenoptic function and the elements of early vision,” in Computational Models Visual Processing (MIT Press, 1991), pp. 3–20.
M. Levoy and P. Hanrahan, “Light field rendering,” Proc. SIGGRAPH (1996), pp. 31–42.
G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
[Crossref]
M. Hirsch, G. Wetzstein, and R. Raskar, “A compressive light field projection system,” ACM Trans. Graph. 33(4), 58 (2014).
[Crossref]
G. P. Bell, “Advanced metrics-based design methodology for multilayer 3D displays,” Proc. SPIE 5443, 239–248 (2004).
[Crossref]
G. P. Bell, R. Craig, R. Paxton, G. Wong, and D. Galbraith, “Beyond flat panels: multi-layered displays with real depth,” SID Symposium Digest of Technical Papers 39(1), 352–355 (2008).
[Crossref]
D. Lanman, M. Hirsch, Y. Kim, and R. Raskar, “Content-adaptive parallax barriers: optimizing dual-layer 3D displays using low-rank light field factorization,” ACM Trans. Graph. 29(6), 163 (2010).
[Crossref]
G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
[Crossref]
X. Cao, Z. Geng, M. Zhang, and X. Zhang, “Load-balancing multi-LCD light field display,” Proc. SPIE 9391, 93910F (2015).
[Crossref]
G. Lippmann, “La photograhie integrale,” Comptes Rendus Acad. Sci. 146, 446–451 (1908).
Y. Kim, J. Kim, J. M. Kang, J. H. Jung, H. Choi, and B. Lee, “Point light source integral imaging with improved resolution and viewing angle by the use of electrically movable pinhole array,” Opt. Express 15(26), 18253–18267 (2007).
[Crossref] [PubMed]
T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,” SIAM Rev. 51(3), 455–500 (2009).
[Crossref]
V. Blondel, N. D. Ho, and P. Vandooren, “Weighted nonnegative matrix factorization and face feature extraction,” Image Vis. Comput. 20071–17 (2008).
G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Real-time image generation for compressive light field displays,” J. Phys. Conf. Ser. 415, 012045 (2013).
[Crossref]

2015 (1)

X. Cao, Z. Geng, M. Zhang, and X. Zhang, “Load-balancing multi-LCD light field display,” Proc. SPIE 9391, 93910F (2015).
[Crossref]

2014 (2)

M. Hirsch, G. Wetzstein, and R. Raskar, “A compressive light field projection system,” ACM Trans. Graph. 33(4), 58 (2014).
[Crossref]

A. Jones, K. Nagano, J. Liu, J. Busch, X. M. Yu, M. Bolas, and P. Debevec, “Interpolating vertical parallax for an autostereoscopic three-dimensional projector array,” J. Electron. Imaging 23(1), 011005 (2014).
[Crossref]

2013 (4)

Z. X. Zhang, Z. Geng, M. Zhang, and H. Dong, “An interactive multiview 3D display system,” Proc. SPIE 8618, 86180P (2013).
[Crossref]

J. Geng, “Three-dimensional display technologies,” Adv. Opt. Photonics 5(4), 456–535 (2013).
[Crossref] [PubMed]

J. Geng, “A volumetric 3D display based on a DLP projection engine,” Displays 34(1), 39–48 (2013).
[Crossref]

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Real-time image generation for compressive light field displays,” J. Phys. Conf. Ser. 415, 012045 (2013).
[Crossref]

2012 (1)

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting,” ACM Trans. Graph. 31(4), 80 (2012).
[Crossref]

2011 (1)

G. Wetzstein, D. Lanman, W. Heidrich, and R. Raskar, “Layered 3D: tomographic image synthesis for attenuation-based light field and high dynamic range displays,” ACM Trans. Graph. 30(4), 95 (2011).
[Crossref]

2010 (1)

D. Lanman, M. Hirsch, Y. Kim, and R. Raskar, “Content-adaptive parallax barriers: optimizing dual-layer 3D displays using low-rank light field factorization,” ACM Trans. Graph. 29(6), 163 (2010).
[Crossref]

2009 (2)

A. Jones, M. Lang, G. Fyffe, X. M. Yu, J. Busch, I. McDowall, M. Bolas, and P. Debevec, “Achieving eye contact in a one-to-many 3D video teleconferencing system,” ACM Trans. Graph. 28(3), 64 (2009).
[Crossref]

T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,” SIAM Rev. 51(3), 455–500 (2009).
[Crossref]

2008 (2)

V. Blondel, N. D. Ho, and P. Vandooren, “Weighted nonnegative matrix factorization and face feature extraction,” Image Vis. Comput. 20071–17 (2008).

G. P. Bell, R. Craig, R. Paxton, G. Wong, and D. Galbraith, “Beyond flat panels: multi-layered displays with real depth,” SID Symposium Digest of Technical Papers 39(1), 352–355 (2008).
[Crossref]

2007 (2)

Y. Kim, J. Kim, J. M. Kang, J. H. Jung, H. Choi, and B. Lee, “Point light source integral imaging with improved resolution and viewing angle by the use of electrically movable pinhole array,” Opt. Express 15(26), 18253–18267 (2007).
[Crossref] [PubMed]

A. Jones, I. Mcdowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]

2004 (1)

G. P. Bell, “Advanced metrics-based design methodology for multilayer 3D displays,” Proc. SPIE 5443, 239–248 (2004).
[Crossref]

1999 (1)

C. Van Berkel, “Image preparation for 3D-LCD,” Proc. SPIE 3639, 84–91 (1999).
[Crossref]

1997 (1)

C. Van Berkel and J. A. Clarke, “Characterisation and optimisation of 3D-LCD module design,” Proc. SPIE 3012, 179–186 (1997).
[Crossref]

1908 (1)

G. Lippmann, “La photograhie integrale,” Comptes Rendus Acad. Sci. 146, 446–451 (1908).

Bader, B. W.

T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,” SIAM Rev. 51(3), 455–500 (2009).
[Crossref]

Bell, G. P.

G. P. Bell, “Advanced metrics-based design methodology for multilayer 3D displays,” Proc. SPIE 5443, 239–248 (2004).
[Crossref]

Blondel, V.

V. Blondel, N. D. Ho, and P. Vandooren, “Weighted nonnegative matrix factorization and face feature extraction,” Image Vis. Comput. 20071–17 (2008).

Bolas, M.

A. Jones, I. Mcdowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]

Busch, J.

Cao, X.

X. Cao, Z. Geng, M. Zhang, and X. Zhang, “Load-balancing multi-LCD light field display,” Proc. SPIE 9391, 93910F (2015).
[Crossref]

Choi, H.

Clarke, J. A.

C. Van Berkel and J. A. Clarke, “Characterisation and optimisation of 3D-LCD module design,” Proc. SPIE 3012, 179–186 (1997).
[Crossref]

Craig, R.

Debevec, P.

A. Jones, I. Mcdowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]

Dong, H.

Z. X. Zhang, Z. Geng, M. Zhang, and H. Dong, “An interactive multiview 3D display system,” Proc. SPIE 8618, 86180P (2013).
[Crossref]

Fyffe, G.

Galbraith, D.

Geng, J.

J. Geng, “A volumetric 3D display based on a DLP projection engine,” Displays 34(1), 39–48 (2013).
[Crossref]

J. Geng, “Three-dimensional display technologies,” Adv. Opt. Photonics 5(4), 456–535 (2013).
[Crossref] [PubMed]

Geng, Z.

X. Cao, Z. Geng, M. Zhang, and X. Zhang, “Load-balancing multi-LCD light field display,” Proc. SPIE 9391, 93910F (2015).
[Crossref]

Z. X. Zhang, Z. Geng, M. Zhang, and H. Dong, “An interactive multiview 3D display system,” Proc. SPIE 8618, 86180P (2013).
[Crossref]

Hanrahan, P.

M. Levoy and P. Hanrahan, “Light field rendering,” Proc. SIGGRAPH (1996), pp. 31–42.

Heidrich, W.

Hirsch, M.

M. Hirsch, G. Wetzstein, and R. Raskar, “A compressive light field projection system,” ACM Trans. Graph. 33(4), 58 (2014).
[Crossref]

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Real-time image generation for compressive light field displays,” J. Phys. Conf. Ser. 415, 012045 (2013).
[Crossref]

Ho, N. D.

V. Blondel, N. D. Ho, and P. Vandooren, “Weighted nonnegative matrix factorization and face feature extraction,” Image Vis. Comput. 20071–17 (2008).

Jones, A.

A. Jones, I. Mcdowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]

Jung, J. H.

Kang, J. M.

Kim, J.

Kim, Y.

Kolda, T. G.

T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,” SIAM Rev. 51(3), 455–500 (2009).
[Crossref]

Lang, M.

Lanman, D.

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Real-time image generation for compressive light field displays,” J. Phys. Conf. Ser. 415, 012045 (2013).
[Crossref]

Lee, B.

Levoy, M.

M. Levoy and P. Hanrahan, “Light field rendering,” Proc. SIGGRAPH (1996), pp. 31–42.

Lippmann, G.

G. Lippmann, “La photograhie integrale,” Comptes Rendus Acad. Sci. 146, 446–451 (1908).

Liu, J.

McDowall, I.

A. Jones, I. Mcdowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]

Nagano, K.

Paxton, R.

Raskar, R.

M. Hirsch, G. Wetzstein, and R. Raskar, “A compressive light field projection system,” ACM Trans. Graph. 33(4), 58 (2014).
[Crossref]

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Real-time image generation for compressive light field displays,” J. Phys. Conf. Ser. 415, 012045 (2013).
[Crossref]

Van Berkel, C.

C. Van Berkel, “Image preparation for 3D-LCD,” Proc. SPIE 3639, 84–91 (1999).
[Crossref]

C. Van Berkel and J. A. Clarke, “Characterisation and optimisation of 3D-LCD module design,” Proc. SPIE 3012, 179–186 (1997).
[Crossref]

Vandooren, P.

V. Blondel, N. D. Ho, and P. Vandooren, “Weighted nonnegative matrix factorization and face feature extraction,” Image Vis. Comput. 20071–17 (2008).

Wetzstein, G.

M. Hirsch, G. Wetzstein, and R. Raskar, “A compressive light field projection system,” ACM Trans. Graph. 33(4), 58 (2014).
[Crossref]

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Real-time image generation for compressive light field displays,” J. Phys. Conf. Ser. 415, 012045 (2013).
[Crossref]

Wong, G.

Yamada, H.

A. Jones, I. Mcdowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]

Yu, X. M.

Zhang, M.

X. Cao, Z. Geng, M. Zhang, and X. Zhang, “Load-balancing multi-LCD light field display,” Proc. SPIE 9391, 93910F (2015).
[Crossref]

Z. X. Zhang, Z. Geng, M. Zhang, and H. Dong, “An interactive multiview 3D display system,” Proc. SPIE 8618, 86180P (2013).
[Crossref]

Zhang, X.

X. Cao, Z. Geng, M. Zhang, and X. Zhang, “Load-balancing multi-LCD light field display,” Proc. SPIE 9391, 93910F (2015).
[Crossref]

Zhang, Z. X.

Z. X. Zhang, Z. Geng, M. Zhang, and H. Dong, “An interactive multiview 3D display system,” Proc. SPIE 8618, 86180P (2013).
[Crossref]

ACM Trans. Graph. (6)

A. Jones, I. Mcdowall, H. Yamada, M. Bolas, and P. Debevec, “Rendering for an interactive 360° light field display,” ACM Trans. Graph. 26(3), 40 (2007).
[Crossref]

M. Hirsch, G. Wetzstein, and R. Raskar, “A compressive light field projection system,” ACM Trans. Graph. 33(4), 58 (2014).
[Crossref]

Adv. Opt. Photonics (1)

J. Geng, “Three-dimensional display technologies,” Adv. Opt. Photonics 5(4), 456–535 (2013).
[Crossref] [PubMed]

Comptes Rendus Acad. Sci. (1)

G. Lippmann, “La photograhie integrale,” Comptes Rendus Acad. Sci. 146, 446–451 (1908).

Displays (1)

J. Geng, “A volumetric 3D display based on a DLP projection engine,” Displays 34(1), 39–48 (2013).
[Crossref]

Image Vis. Comput. (1)

V. Blondel, N. D. Ho, and P. Vandooren, “Weighted nonnegative matrix factorization and face feature extraction,” Image Vis. Comput. 20071–17 (2008).

J. Electron. Imaging (1)

J. Phys. Conf. Ser. (1)

G. Wetzstein, D. Lanman, M. Hirsch, and R. Raskar, “Real-time image generation for compressive light field displays,” J. Phys. Conf. Ser. 415, 012045 (2013).
[Crossref]

Opt. Express (1)

Proc. SPIE (5)

X. Cao, Z. Geng, M. Zhang, and X. Zhang, “Load-balancing multi-LCD light field display,” Proc. SPIE 9391, 93910F (2015).
[Crossref]

C. Van Berkel, “Image preparation for 3D-LCD,” Proc. SPIE 3639, 84–91 (1999).
[Crossref]

C. Van Berkel and J. A. Clarke, “Characterisation and optimisation of 3D-LCD module design,” Proc. SPIE 3012, 179–186 (1997).
[Crossref]

Z. X. Zhang, Z. Geng, M. Zhang, and H. Dong, “An interactive multiview 3D display system,” Proc. SPIE 8618, 86180P (2013).
[Crossref]

G. P. Bell, “Advanced metrics-based design methodology for multilayer 3D displays,” Proc. SPIE 5443, 239–248 (2004).
[Crossref]

SIAM Rev. (1)

T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,” SIAM Rev. 51(3), 455–500 (2009).
[Crossref]

SID Symposium Digest of Technical Papers (1)

Other (2)

E. H. Adelson and J. R. Bergen, “The plenoptic function and the elements of early vision,” in Computational Models Visual Processing (MIT Press, 1991), pp. 3–20.

M. Levoy and P. Hanrahan, “Light field rendering,” Proc. SIGGRAPH (1996), pp. 31–42.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.

Click here to see a list of articles that cite this paper

Fig. 1 Schematic diagram of compressive light field display based on multiple LCDs

Download Full Size | PPT Slide | PDF

Fig. 2 Parallax of reconstructed light field by multiple layer images

Download Full Size | PPT Slide | PDF

Fig. 3 Layer images figured out by light field decomposition

Download Full Size | PPT Slide | PDF

Fig. 4 Illustration of light rays attenuation in sub-dimension of light field

Download Full Size | PPT Slide | PDF

Fig. 5 Light field resolution pyramid for utilizing internal coherence

Download Full Size | PPT Slide | PDF

Fig. 6 Light field decomposition with random initial estimate at 1th pyramid level (7x7x48x64)

Download Full Size | PPT Slide | PDF

Fig. 7 Light field decomposition with optimal initialization at 2th pyramid level (7x7x96x128)

Download Full Size | PPT Slide | PDF

Fig. 8 Light field decomposition with optimal initialization at 4th pyramid level (7x7x384x512)

Download Full Size | PPT Slide | PDF

Fig. 9 Decomposition of light field video utilizing both internal and external coherence

Download Full Size | PPT Slide | PDF

Fig. 10 Accelerating decomposition of light field video without quality deterioration.

Download Full Size | PPT Slide | PDF

Fig. 11 Iteration with random estimate and initialization by internal coherence at keyframe (57th frame of “bat”)

Download Full Size | PPT Slide | PDF

Fig. 12 Iteration with random estimate and initialization by external coherence at non-keyframe (58th frame of “bat”)

Download Full Size | PPT Slide | PDF

Fig. 13 Prototype of three layers compressive light field video display

Download Full Size | PPT Slide | PDF

Fig. 14 Photograph of prototype

Download Full Size | PPT Slide | PDF

Tables (2)

Table 1 Accelerating single light field frame by utilizing internal coherence.

View Table | View all tables in this article

Table 2 The algorithm flow chart for light field decomposition in parallel.

View Table | View all tables in this article

Equations (3)

Equations on this page are rendered with MathJax. Learn more.

{\begin{cases} l_{1} = {a_{2}, b_{1}, c_{1}} \\ l_{2} = {a_{1}, b_{1}, c_{2}} \\ l_{3} = {a_{2}, b_{2}, c_{3}} \end{cases}

C \approx s \times I \times (v_{x} \times v_{y} \times r_{x} \times r_{y})

a^{'} = a * \frac{\sum_{j = 1}^{V} l_{j} * b_{j} * c_{j}}{\sum_{j = 1}^{V} {\hat{l}}_{j} * b_{j} * c_{j}}, s . t . 0 \leq a^{'} \leq 1

	LF Res		Dice	Dragon	Happybuddha	Messerschmitt	Reddragon	Average
Random initial values	384*512	Iteration	12	7	6	8	7	8
	384*512	Time(s)	2.683	1.567	1.343	1.788	1.574	1.791
	PSNR(db)		25.861	20.615	20.792	26.097	21.063	22.886

Resolution pyramid utilizing internal coherence	48*64	Iteration	3	3	3	3	3	3
	48*64	Time(s)	0.0523	0.0529	0.0515	0.0533	0.0521	0.0524
	96*128	Iteration	3	3	3	3	3	3
	96*128	Time(s)	0.0641	0.0647	0.0635	0.0607	0.0639	0.0634
	192*256	Iteration	3	3	3	3	3	3
	192*256	Time(s)	0.2079	0.2083	0.2058	0.2034	0.2064	0.2064
	384*512	Iteration	3	3	3	3	3	3
	384*512	Time(s)	0.6788	0.6698	0.6681	0.6677	0.6704	0.6710
	Total time		1.0031	0.9957	0.9889	0.9851	0.9928	0.9931
	PSNR(db)		25.868	20.624	20.802	26.105	21.041	22.888

if (true = = keyframe_(i)) // Index “i” represents i-th light field frame
	{ // keyframe should utilize internal coherence
		For ithLevel = 1:1: (NumLevel-1) // construct a light field resolution pyramid
		{
			if (1 = = ithLevel) // the top level must iterates from random initial values
			{
				A_{(i, ithLevel)} = random([0, 1]) ; // Set all pixels in all LCD layers
				B_{(i, ithLevel)} = random([0, 1]) ; // as random noise ranging from 0 to 1;
				C_{(i, ithLevel)} = random([0, 1]) ;
			}
			For ith = 1:1:NumIteration
			{ // Update pixels in three layers according to E.q. (2)
				A’_{(i, ithLevel)} = Update (A_{(i, ithLevel)}) ; A_{(i, ithLevel)} = A’_{(i, ithLevel)};
				B’_{(i, ithLevel)} = Update (B_{(i, ithLevel)}) ; B_{(i, ithLevel)} = B’_{(i, ithLevel)};
				C’_{(i, ithLevel)} = Update(C_{(i, ithLevel)}) ; C_{(i, ithLevel)} = C’_{(i, ithLevel)};
			}
			// get initial values of next-level by interpolating current layer images
			A_{(i, ithLevel + 1)} = Interpolation(A’_{(i, ithLevel)}) ;
			B_{(i, ithLevel + 1)} = Interpolation(B’_{(i, ithLevel)}) ;
			C_{(i, ithLevel + 1)} = Interpolation(C’_{(i, ithLevel)}) ;
		}
		A_(i) = A_{(i, ithLevel + 1)} ; // Set interpolation of penultimate-level layer images
		B_(i) = B_{(i, ithLevel + 1)} ; // as initial values of full-resolution level
		C_(i) = C_{(i, ithLevel + 1)} ;
	}
	else // if it’s not key-frame, we get the initial values from previous frame directly
	{
		A_(i) = A_(i-1) ; // Set all LCD layers equal to layers image of previous frame
		B_(i) = B_(i-1);
		C_(i) = C_(i-1);
	}
	// Code above figured out the initial layer images of full-resolution light field by internal
	// or external coherence. Following code finish iteration beginning with such initialization.
	For ith = 1:1:NumIteration
	{
		For layer = {A_(i), B_(i), C_(i)} //All pixels in single layer are processed synchronously
		{ // One GPU kernel is responsible for one pixel in current LCD layer
			Sum = 0.0f;
			Sum’ = 0.0f;
			For j = 1:1:V // V is the total amount of views
			{
				l’_j = ab_jc_j ; // record the reconstruction of j-th light ray
				Sum = Sum + l_j * b_j*c_j ; // l_j is the j-th target light ray
				Sum’ = Sum’ + l’_j * b_j*c_j ;
			}
			a’ = a*(Sum/ Sum’) ; // update the pixel a
			a’ = min(0, max(a’, 1)) ; // restricted pixel a’ ranging [0, 1]
			__syncthreads() ; // wait for finishing all kernels
		}
		cudaDeviceSynchronize() ; // wait for finishing updating current layer
	}

	LF Res		Dice	Dragon	Happybuddha	Messerschmitt	Reddragon	Average
Random initial values	384*512	Iteration	12	7	6	8	7	8
	384*512	Time(s)	2.683	1.567	1.343	1.788	1.574	1.791
	PSNR(db)		25.861	20.615	20.792	26.097	21.063	22.886

Resolution pyramid utilizing internal coherence	48*64	Iteration	3	3	3	3	3	3
	48*64	Time(s)	0.0523	0.0529	0.0515	0.0533	0.0521	0.0524
	96*128	Iteration	3	3	3	3	3	3
	96*128	Time(s)	0.0641	0.0647	0.0635	0.0607	0.0639	0.0634
	192*256	Iteration	3	3	3	3	3	3
	192*256	Time(s)	0.2079	0.2083	0.2058	0.2034	0.2064	0.2064
	384*512	Iteration	3	3	3	3	3	3
	384*512	Time(s)	0.6788	0.6698	0.6681	0.6677	0.6704	0.6710
	Total time		1.0031	0.9957	0.9889	0.9851	0.9928	0.9931
	PSNR(db)		25.868	20.624	20.802	26.105	21.041	22.888

if (true = = keyframe_(i)) // Index “i” represents i-th light field frame
	{ // keyframe should utilize internal coherence
		For ithLevel = 1:1: (NumLevel-1) // construct a light field resolution pyramid
		{
			if (1 = = ithLevel) // the top level must iterates from random initial values
			{
				A_{(i, ithLevel)} = random([0, 1]) ; // Set all pixels in all LCD layers
				B_{(i, ithLevel)} = random([0, 1]) ; // as random noise ranging from 0 to 1;
				C_{(i, ithLevel)} = random([0, 1]) ;
			}
			For ith = 1:1:NumIteration
			{ // Update pixels in three layers according to E.q. (2)
				A’_{(i, ithLevel)} = Update (A_{(i, ithLevel)}) ; A_{(i, ithLevel)} = A’_{(i, ithLevel)};
				B’_{(i, ithLevel)} = Update (B_{(i, ithLevel)}) ; B_{(i, ithLevel)} = B’_{(i, ithLevel)};
				C’_{(i, ithLevel)} = Update(C_{(i, ithLevel)}) ; C_{(i, ithLevel)} = C’_{(i, ithLevel)};
			}
			// get initial values of next-level by interpolating current layer images
			A_{(i, ithLevel + 1)} = Interpolation(A’_{(i, ithLevel)}) ;
			B_{(i, ithLevel + 1)} = Interpolation(B’_{(i, ithLevel)}) ;
			C_{(i, ithLevel + 1)} = Interpolation(C’_{(i, ithLevel)}) ;
		}
		A_(i) = A_{(i, ithLevel + 1)} ; // Set interpolation of penultimate-level layer images
		B_(i) = B_{(i, ithLevel + 1)} ; // as initial values of full-resolution level
		C_(i) = C_{(i, ithLevel + 1)} ;
	}
	else // if it’s not key-frame, we get the initial values from previous frame directly
	{
		A_(i) = A_(i-1) ; // Set all LCD layers equal to layers image of previous frame
		B_(i) = B_(i-1);
		C_(i) = C_(i-1);
	}
	// Code above figured out the initial layer images of full-resolution light field by internal
	// or external coherence. Following code finish iteration beginning with such initialization.
	For ith = 1:1:NumIteration
	{
		For layer = {A_(i), B_(i), C_(i)} //All pixels in single layer are processed synchronously
		{ // One GPU kernel is responsible for one pixel in current LCD layer
			Sum = 0.0f;
			Sum’ = 0.0f;
			For j = 1:1:V // V is the total amount of views
			{
				l’_j = ab_jc_j ; // record the reconstruction of j-th light ray
				Sum = Sum + l_j * b_j*c_j ; // l_j is the j-th target light ray
				Sum’ = Sum’ + l’_j * b_j*c_j ;
			}
			a’ = a*(Sum/ Sum’) ; // update the pixel a
			a’ = min(0, max(a’, 1)) ; // restricted pixel a’ ranging [0, 1]
			__syncthreads() ; // wait for finishing all kernels
		}
		cudaDeviceSynchronize() ; // wait for finishing updating current layer
	}

Abstract

References

2015 (1)

2014 (2)

2013 (4)

2012 (1)

2011 (1)

2010 (1)

2009 (2)

2008 (2)

2007 (2)

2004 (1)

1999 (1)

1997 (1)

1908 (1)

Bader, B. W.

Bell, G. P.

Blondel, V.

Bolas, M.

Busch, J.

Cao, X.

Choi, H.

Clarke, J. A.

Craig, R.

Debevec, P.

Dong, H.

Fyffe, G.

Galbraith, D.

Geng, J.

Geng, Z.

Hanrahan, P.

Heidrich, W.

Hirsch, M.

Ho, N. D.

Jones, A.

Jung, J. H.

Kang, J. M.

Kim, J.

Kim, Y.

Kolda, T. G.

Lang, M.

Lanman, D.

Lee, B.

Levoy, M.

Lippmann, G.

Liu, J.

McDowall, I.

Nagano, K.

Paxton, R.

Raskar, R.

Van Berkel, C.

Vandooren, P.

Wetzstein, G.

Wong, G.

Yamada, H.

Yu, X. M.

Zhang, M.

Zhang, X.

Zhang, Z. X.

ACM Trans. Graph. (6)

Adv. Opt. Photonics (1)

Comptes Rendus Acad. Sci. (1)

Displays (1)

Image Vis. Comput. (1)

J. Electron. Imaging (1)

J. Phys. Conf. Ser. (1)

Opt. Express (1)

Proc. SPIE (5)

SIAM Rev. (1)

SID Symposium Digest of Technical Papers (1)

Other (2)

Cited By

Figures (14)

Tables (2)

Equations (3)

Metrics

Login or Create Account