Abstract

Bundle adjustment (BA) is a common estimation algorithm that is widely used in machine vision as the last step in a feature-based three-dimensional (3D) reconstruction algorithm. BA is essentially a non-convex non-linear least-square problem that can simultaneously solve the 3D coordinates of all the feature points describing the scene geometry, as well as the parameters of the camera. The conventional BA takes a parameter either as a fixed value or as an unconstrained variable based on whether the parameter is known or not. In cases where the known parameters are inaccurate but constrained in a range, conventional BA results in an incorrect 3D reconstruction by using these parameters as fixed values. On the other hand, these inaccurate parameters can be treated as unknown variables, but this does not exploit the knowledge of the constraints, and the resulting reconstruction can be erroneous since the BA optimization halts at a dramatically incorrect local minimum due to its non-convexity. In many practical 3D reconstruction applications, unknown variables with range constraints are usually available, such as a measurement with a range of uncertainty or a bounded estimate. Thus to better utilize these pre-known, constrained, but inaccurate parameters, a bound constrained bundle adjustment (BCBA) algorithm is proposed, developed and tested in this study. A scanning fiber endoscope (the camera) is used to capture a sequence of images above a surgery phantom (the object) of known geometry. 3D virtual models are reconstructed based on these images and then compared with the ground truth. The experimental results demonstrate BCBA can achieve a more reliable, rapid, and accurate 3D reconstruction than conventional bundle adjustment.

© 2015 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Depth and thermal sensor fusion to enhance 3D thermographic reconstruction

Yanpeng Cao, Baobei Xu, Zhangyu Ye, Jiangxin Yang, Yanlong Cao, Christel-Loic Tisse, and Xin Li
Opt. Express 26(7) 8179-8193 (2018)

3D reconstruction of cystoscopy videos for comprehensive bladder records

Kristen L. Lurie, Roland Angst, Dimitar V. Zlatev, Joseph C. Liao, and Audrey K. Ellerbee Bowden
Biomed. Opt. Express 8(4) 2106-2123 (2017)

References

  • View by:
  • |
  • |
  • |

  1. G. Bianco, A. Gallo, F. Bruno, and M. Muzzupappa, “A comparison between active and passive techniques for underwater 3d applications,” in International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences3816 (2011), pp. 357–363.
  2. Y. Gong and S. Zhang, “Improving 4-D shape measurement by using projector defocusing,” Proc. SPIE 7790, 77901A (2010).
    [Crossref]
  3. J. Geng, “Structured-light 3D surface imaging: a tutorial,” Adv. Opt. Photon. 3(2), 128–160 (2011).
    [Crossref]
  4. Y. Gong and S. Zhang, “Ultrafast 3-D shape measurement with an off-the-shelf DLP projector,” Opt. Express 18(19), 19743–19754 (2010).
    [Crossref] [PubMed]
  5. Z. Zhang, “Microsoft kinect sensor and its effect,” MultiMedia IEEE 19(2), 4–10 (2012).
    [Crossref]
  6. S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, “A comparison and evaluation of multi-view stereo reconstruction algorithms,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 519–528.
  7. Y. Gong, R. S. Johnston, C. D. Melville, and E. J. Seibel, “Axial-stereo 3D optical metrology of internally machined parts using high-quality imaging from a scanning laser endoscope,” in International Symposium on Optomechatronic Technologies (ISOT), Seattle, USA, November 5–7, (2014).
  8. S. Agarwal, N. Snavely, I. Simon, S. M. Seitz, and R. Szeliski, “Building rome in a day,” in IEEE 12th International Conference on Computer Vision (IEEE, 2011), pp. 105–112.
  9. C. Wu, “Towards linear-time incremental structure from motion,” in International Conference on 3D Vision (IEEE, 2013), pp. 127–134.
  10. D. Crandall, A. Owens, N. Snavely, and D. Huttenlocher, “Discrete-continuous optimization for large-scale structure from motion,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 3001–3008.
  11. B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon, “Bundle adjustment–a modern synthesis,” In Vision Algorithms: Theory and Practice, B. Triggs, A. Zisserman, and R. Szeliski, eds. (Springer Berlin Heidelberg, 2000), pp. 298–372.
    [Crossref]
  12. K. Mitra and R. Chellappa, “A Scalable Projective Bundle Adjustment Algorithm using the L infinity Norm,” in Sixth Indian Conference on Computer Vision, Graphics and Image Processing (IEEE, 2008), pp. 79–86.
  13. C. Engels, H. Stewénius, and D. Nistér, “Bundle adjustment rules,” in Photogrammetric computer vision, (2006).
  14. M. I. A. Lourakis and A. A. Argyros, “SBA: A software package for generic sparse bundle adjustment,” ACM Trans. Math. Software 36(1), 1–30 (2009)
    [Crossref]
  15. C. Wu, S. Agarwal, B. Curless, and S. M. Seitz, “Multicore bundle adjustment,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 3057–3064.
  16. Y. Jeong, D. Nister, D. Steedly, R. Szeliski, and I. S. Kweon, “Pushing the envelope of modern methods for bundle adjustment,” IEEE Trans. Pattern Anal. Mach. Intell. 34(8), 1605–1617 (2012).
    [Crossref] [PubMed]
  17. J. Salvi, X. Armangu, and J. Batlle, “A comparative review of camera calibrating methods with accuracy evaluation,” Pattern Recognit. 35(7), 1617–1635 (2002).
    [Crossref]
  18. K. H. Wong and M. M. Y. Chang, “3D model reconstruction by constrained bundle adjustment,” in Proceedings of the 17th International Conference on Pattern Recognition (IEEE, 2004), pp. 902–905.
  19. C. Albl and T. Pajdla, “Constrained Bundle Adjustment for Panoramic Cameras,” in 18th Computer Vision Winter Workshop, Hernstein, Austria, 4–6 February 2013.
  20. Y. Zhang, K. Hu, and R. Huang, “Bundle adjustment with additional constraints applied to imagery of the Dunhuang wall paintings,” ISPRS J. Photogramm. Remote Sens. 72, 113–120 (2012).
    [Crossref]
  21. R. Szeliski and P. H. Torr, “Geometrically constrained structure from motion: Points on planes,” In 3D Structure from Multiple Images of Large-Scale Environments, R. Koch and L. V. Gool, eds. (Springer Berlin Heidelberg, 1998), pp. 171–186.
    [Crossref]
  22. A. Bartoli and P. Sturm, “Constrained structure and motion from multiple uncalibrated views of a piecewise planar scene,” Int. J. Comput. Vision 52(1), 45–64 (2003).
    [Crossref]
  23. A. Cohen, C. Zach, S. N. Sinha, and M. Pollefeys, “Discovering and exploiting 3D symmetries in structure from motion,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 1514–1521.
  24. T. D. Soper, M. P. Porter, and E. J. Seibel, “Surface mosaics of the bladder reconstructed from endoscopic video for automated surveillance,” IEEE Trans. Biomed. Eng. 59(6), 1670–1680 (2012).
    [Crossref] [PubMed]
  25. Y. Gong, T. D. Soper, V. W. Hou, D. Hu, B. Hannaford, and E. J. Seibel, “Mapping surgical fields by moving a laser-scanning multimodal scope attached to a robot arm,” Proc. SPIE 9036, 90362S (2014).
    [Crossref]
  26. M. I. A. Lourakis and A. A. Argyros, “Is Levenberg-Marquardt the most efficient optimization algorithm for implementing bundle adjustment?,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2005), pp. 1526–1531.
  27. A. Berman and N. Shaked-Monderer, Completely positive matrices (World Scientific, 2003), Chap. 1.
  28. P. H. Calamai and J. J. Mor, “Projected gradient methods for linearly constrained problems,” Math. Program. 39(1), 93–116 (1987).
    [Crossref]
  29. C. M. Lee, C. J. Engelbrech, T. D. Soper, F. Helmchen, and E. J. Seibel, “Scanning fiber endoscopy with highly flexible, 1 mm catheterscopes for wide-field, full-color imaging,” J. Biophotonics 3(5-6), 385–407 (2010).
    [Crossref]
  30. Y. Gong, D. Hu, E. J. Seibel, and B. Hannaford, “Accurate 3D virtual reconstruction of surgical field using calibrated trajectories of an image-guided medical robot,” J. Med. Imag. 1(3), 035002 (2014).
    [Crossref]
  31. Y. Gong, D. Hu, B. Hannaford, and E.J. Seibel, “Toward real-time endoscopically-guided robotic navigation based on a 3D virtual surgical field model,” Proc. SPIE 9415, 94150C (2015).
  32. D. G. Lowe, “Object recognition from local scale-invariant features,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 1999), pp. 1150–1157.
  33. P. J. Besl and N.D. McKay, “A method for registration of 3-D shapes,” IEEE Trans. Pattern Anal. Machine Intell. 14(2), 239–256 (1992).
    [Crossref]
  34. B. Klingner, D. Martin, and J. Roseborough, “Street view motion-from-structure-from-motion,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2013), pp. 953–960.

2015 (1)

Y. Gong, D. Hu, B. Hannaford, and E.J. Seibel, “Toward real-time endoscopically-guided robotic navigation based on a 3D virtual surgical field model,” Proc. SPIE 9415, 94150C (2015).

2014 (2)

Y. Gong, D. Hu, E. J. Seibel, and B. Hannaford, “Accurate 3D virtual reconstruction of surgical field using calibrated trajectories of an image-guided medical robot,” J. Med. Imag. 1(3), 035002 (2014).
[Crossref]

Y. Gong, T. D. Soper, V. W. Hou, D. Hu, B. Hannaford, and E. J. Seibel, “Mapping surgical fields by moving a laser-scanning multimodal scope attached to a robot arm,” Proc. SPIE 9036, 90362S (2014).
[Crossref]

2012 (4)

Y. Zhang, K. Hu, and R. Huang, “Bundle adjustment with additional constraints applied to imagery of the Dunhuang wall paintings,” ISPRS J. Photogramm. Remote Sens. 72, 113–120 (2012).
[Crossref]

Z. Zhang, “Microsoft kinect sensor and its effect,” MultiMedia IEEE 19(2), 4–10 (2012).
[Crossref]

Y. Jeong, D. Nister, D. Steedly, R. Szeliski, and I. S. Kweon, “Pushing the envelope of modern methods for bundle adjustment,” IEEE Trans. Pattern Anal. Mach. Intell. 34(8), 1605–1617 (2012).
[Crossref] [PubMed]

T. D. Soper, M. P. Porter, and E. J. Seibel, “Surface mosaics of the bladder reconstructed from endoscopic video for automated surveillance,” IEEE Trans. Biomed. Eng. 59(6), 1670–1680 (2012).
[Crossref] [PubMed]

2011 (1)

2010 (3)

Y. Gong and S. Zhang, “Improving 4-D shape measurement by using projector defocusing,” Proc. SPIE 7790, 77901A (2010).
[Crossref]

Y. Gong and S. Zhang, “Ultrafast 3-D shape measurement with an off-the-shelf DLP projector,” Opt. Express 18(19), 19743–19754 (2010).
[Crossref] [PubMed]

C. M. Lee, C. J. Engelbrech, T. D. Soper, F. Helmchen, and E. J. Seibel, “Scanning fiber endoscopy with highly flexible, 1 mm catheterscopes for wide-field, full-color imaging,” J. Biophotonics 3(5-6), 385–407 (2010).
[Crossref]

2009 (1)

M. I. A. Lourakis and A. A. Argyros, “SBA: A software package for generic sparse bundle adjustment,” ACM Trans. Math. Software 36(1), 1–30 (2009)
[Crossref]

2003 (1)

A. Bartoli and P. Sturm, “Constrained structure and motion from multiple uncalibrated views of a piecewise planar scene,” Int. J. Comput. Vision 52(1), 45–64 (2003).
[Crossref]

2002 (1)

J. Salvi, X. Armangu, and J. Batlle, “A comparative review of camera calibrating methods with accuracy evaluation,” Pattern Recognit. 35(7), 1617–1635 (2002).
[Crossref]

1992 (1)

P. J. Besl and N.D. McKay, “A method for registration of 3-D shapes,” IEEE Trans. Pattern Anal. Machine Intell. 14(2), 239–256 (1992).
[Crossref]

1987 (1)

P. H. Calamai and J. J. Mor, “Projected gradient methods for linearly constrained problems,” Math. Program. 39(1), 93–116 (1987).
[Crossref]

Agarwal, S.

S. Agarwal, N. Snavely, I. Simon, S. M. Seitz, and R. Szeliski, “Building rome in a day,” in IEEE 12th International Conference on Computer Vision (IEEE, 2011), pp. 105–112.

C. Wu, S. Agarwal, B. Curless, and S. M. Seitz, “Multicore bundle adjustment,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 3057–3064.

Albl, C.

C. Albl and T. Pajdla, “Constrained Bundle Adjustment for Panoramic Cameras,” in 18th Computer Vision Winter Workshop, Hernstein, Austria, 4–6 February 2013.

Argyros, A. A.

M. I. A. Lourakis and A. A. Argyros, “SBA: A software package for generic sparse bundle adjustment,” ACM Trans. Math. Software 36(1), 1–30 (2009)
[Crossref]

M. I. A. Lourakis and A. A. Argyros, “Is Levenberg-Marquardt the most efficient optimization algorithm for implementing bundle adjustment?,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2005), pp. 1526–1531.

Armangu, X.

J. Salvi, X. Armangu, and J. Batlle, “A comparative review of camera calibrating methods with accuracy evaluation,” Pattern Recognit. 35(7), 1617–1635 (2002).
[Crossref]

Bartoli, A.

A. Bartoli and P. Sturm, “Constrained structure and motion from multiple uncalibrated views of a piecewise planar scene,” Int. J. Comput. Vision 52(1), 45–64 (2003).
[Crossref]

Batlle, J.

J. Salvi, X. Armangu, and J. Batlle, “A comparative review of camera calibrating methods with accuracy evaluation,” Pattern Recognit. 35(7), 1617–1635 (2002).
[Crossref]

Berman, A.

A. Berman and N. Shaked-Monderer, Completely positive matrices (World Scientific, 2003), Chap. 1.

Besl, P. J.

P. J. Besl and N.D. McKay, “A method for registration of 3-D shapes,” IEEE Trans. Pattern Anal. Machine Intell. 14(2), 239–256 (1992).
[Crossref]

Bianco, G.

G. Bianco, A. Gallo, F. Bruno, and M. Muzzupappa, “A comparison between active and passive techniques for underwater 3d applications,” in International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences3816 (2011), pp. 357–363.

Bruno, F.

G. Bianco, A. Gallo, F. Bruno, and M. Muzzupappa, “A comparison between active and passive techniques for underwater 3d applications,” in International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences3816 (2011), pp. 357–363.

Calamai, P. H.

P. H. Calamai and J. J. Mor, “Projected gradient methods for linearly constrained problems,” Math. Program. 39(1), 93–116 (1987).
[Crossref]

Chang, M. M. Y.

K. H. Wong and M. M. Y. Chang, “3D model reconstruction by constrained bundle adjustment,” in Proceedings of the 17th International Conference on Pattern Recognition (IEEE, 2004), pp. 902–905.

Chellappa, R.

K. Mitra and R. Chellappa, “A Scalable Projective Bundle Adjustment Algorithm using the L infinity Norm,” in Sixth Indian Conference on Computer Vision, Graphics and Image Processing (IEEE, 2008), pp. 79–86.

Cohen, A.

A. Cohen, C. Zach, S. N. Sinha, and M. Pollefeys, “Discovering and exploiting 3D symmetries in structure from motion,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 1514–1521.

Crandall, D.

D. Crandall, A. Owens, N. Snavely, and D. Huttenlocher, “Discrete-continuous optimization for large-scale structure from motion,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 3001–3008.

Curless, B.

S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, “A comparison and evaluation of multi-view stereo reconstruction algorithms,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 519–528.

C. Wu, S. Agarwal, B. Curless, and S. M. Seitz, “Multicore bundle adjustment,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 3057–3064.

Diebel, J.

S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, “A comparison and evaluation of multi-view stereo reconstruction algorithms,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 519–528.

Engelbrech, C. J.

C. M. Lee, C. J. Engelbrech, T. D. Soper, F. Helmchen, and E. J. Seibel, “Scanning fiber endoscopy with highly flexible, 1 mm catheterscopes for wide-field, full-color imaging,” J. Biophotonics 3(5-6), 385–407 (2010).
[Crossref]

Engels, C.

C. Engels, H. Stewénius, and D. Nistér, “Bundle adjustment rules,” in Photogrammetric computer vision, (2006).

Fitzgibbon, A. W.

B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon, “Bundle adjustment–a modern synthesis,” In Vision Algorithms: Theory and Practice, B. Triggs, A. Zisserman, and R. Szeliski, eds. (Springer Berlin Heidelberg, 2000), pp. 298–372.
[Crossref]

Gallo, A.

G. Bianco, A. Gallo, F. Bruno, and M. Muzzupappa, “A comparison between active and passive techniques for underwater 3d applications,” in International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences3816 (2011), pp. 357–363.

Geng, J.

Gong, Y.

Y. Gong, D. Hu, B. Hannaford, and E.J. Seibel, “Toward real-time endoscopically-guided robotic navigation based on a 3D virtual surgical field model,” Proc. SPIE 9415, 94150C (2015).

Y. Gong, D. Hu, E. J. Seibel, and B. Hannaford, “Accurate 3D virtual reconstruction of surgical field using calibrated trajectories of an image-guided medical robot,” J. Med. Imag. 1(3), 035002 (2014).
[Crossref]

Y. Gong, T. D. Soper, V. W. Hou, D. Hu, B. Hannaford, and E. J. Seibel, “Mapping surgical fields by moving a laser-scanning multimodal scope attached to a robot arm,” Proc. SPIE 9036, 90362S (2014).
[Crossref]

Y. Gong and S. Zhang, “Ultrafast 3-D shape measurement with an off-the-shelf DLP projector,” Opt. Express 18(19), 19743–19754 (2010).
[Crossref] [PubMed]

Y. Gong and S. Zhang, “Improving 4-D shape measurement by using projector defocusing,” Proc. SPIE 7790, 77901A (2010).
[Crossref]

Y. Gong, R. S. Johnston, C. D. Melville, and E. J. Seibel, “Axial-stereo 3D optical metrology of internally machined parts using high-quality imaging from a scanning laser endoscope,” in International Symposium on Optomechatronic Technologies (ISOT), Seattle, USA, November 5–7, (2014).

Hannaford, B.

Y. Gong, D. Hu, B. Hannaford, and E.J. Seibel, “Toward real-time endoscopically-guided robotic navigation based on a 3D virtual surgical field model,” Proc. SPIE 9415, 94150C (2015).

Y. Gong, D. Hu, E. J. Seibel, and B. Hannaford, “Accurate 3D virtual reconstruction of surgical field using calibrated trajectories of an image-guided medical robot,” J. Med. Imag. 1(3), 035002 (2014).
[Crossref]

Y. Gong, T. D. Soper, V. W. Hou, D. Hu, B. Hannaford, and E. J. Seibel, “Mapping surgical fields by moving a laser-scanning multimodal scope attached to a robot arm,” Proc. SPIE 9036, 90362S (2014).
[Crossref]

Hartley, R. I.

B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon, “Bundle adjustment–a modern synthesis,” In Vision Algorithms: Theory and Practice, B. Triggs, A. Zisserman, and R. Szeliski, eds. (Springer Berlin Heidelberg, 2000), pp. 298–372.
[Crossref]

Helmchen, F.

C. M. Lee, C. J. Engelbrech, T. D. Soper, F. Helmchen, and E. J. Seibel, “Scanning fiber endoscopy with highly flexible, 1 mm catheterscopes for wide-field, full-color imaging,” J. Biophotonics 3(5-6), 385–407 (2010).
[Crossref]

Hou, V. W.

Y. Gong, T. D. Soper, V. W. Hou, D. Hu, B. Hannaford, and E. J. Seibel, “Mapping surgical fields by moving a laser-scanning multimodal scope attached to a robot arm,” Proc. SPIE 9036, 90362S (2014).
[Crossref]

Hu, D.

Y. Gong, D. Hu, B. Hannaford, and E.J. Seibel, “Toward real-time endoscopically-guided robotic navigation based on a 3D virtual surgical field model,” Proc. SPIE 9415, 94150C (2015).

Y. Gong, D. Hu, E. J. Seibel, and B. Hannaford, “Accurate 3D virtual reconstruction of surgical field using calibrated trajectories of an image-guided medical robot,” J. Med. Imag. 1(3), 035002 (2014).
[Crossref]

Y. Gong, T. D. Soper, V. W. Hou, D. Hu, B. Hannaford, and E. J. Seibel, “Mapping surgical fields by moving a laser-scanning multimodal scope attached to a robot arm,” Proc. SPIE 9036, 90362S (2014).
[Crossref]

Hu, K.

Y. Zhang, K. Hu, and R. Huang, “Bundle adjustment with additional constraints applied to imagery of the Dunhuang wall paintings,” ISPRS J. Photogramm. Remote Sens. 72, 113–120 (2012).
[Crossref]

Huang, R.

Y. Zhang, K. Hu, and R. Huang, “Bundle adjustment with additional constraints applied to imagery of the Dunhuang wall paintings,” ISPRS J. Photogramm. Remote Sens. 72, 113–120 (2012).
[Crossref]

Huttenlocher, D.

D. Crandall, A. Owens, N. Snavely, and D. Huttenlocher, “Discrete-continuous optimization for large-scale structure from motion,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 3001–3008.

Jeong, Y.

Y. Jeong, D. Nister, D. Steedly, R. Szeliski, and I. S. Kweon, “Pushing the envelope of modern methods for bundle adjustment,” IEEE Trans. Pattern Anal. Mach. Intell. 34(8), 1605–1617 (2012).
[Crossref] [PubMed]

Johnston, R. S.

Y. Gong, R. S. Johnston, C. D. Melville, and E. J. Seibel, “Axial-stereo 3D optical metrology of internally machined parts using high-quality imaging from a scanning laser endoscope,” in International Symposium on Optomechatronic Technologies (ISOT), Seattle, USA, November 5–7, (2014).

Klingner, B.

B. Klingner, D. Martin, and J. Roseborough, “Street view motion-from-structure-from-motion,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2013), pp. 953–960.

Kweon, I. S.

Y. Jeong, D. Nister, D. Steedly, R. Szeliski, and I. S. Kweon, “Pushing the envelope of modern methods for bundle adjustment,” IEEE Trans. Pattern Anal. Mach. Intell. 34(8), 1605–1617 (2012).
[Crossref] [PubMed]

Lee, C. M.

C. M. Lee, C. J. Engelbrech, T. D. Soper, F. Helmchen, and E. J. Seibel, “Scanning fiber endoscopy with highly flexible, 1 mm catheterscopes for wide-field, full-color imaging,” J. Biophotonics 3(5-6), 385–407 (2010).
[Crossref]

Lourakis, M. I. A.

M. I. A. Lourakis and A. A. Argyros, “SBA: A software package for generic sparse bundle adjustment,” ACM Trans. Math. Software 36(1), 1–30 (2009)
[Crossref]

M. I. A. Lourakis and A. A. Argyros, “Is Levenberg-Marquardt the most efficient optimization algorithm for implementing bundle adjustment?,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2005), pp. 1526–1531.

Lowe, D. G.

D. G. Lowe, “Object recognition from local scale-invariant features,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 1999), pp. 1150–1157.

Martin, D.

B. Klingner, D. Martin, and J. Roseborough, “Street view motion-from-structure-from-motion,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2013), pp. 953–960.

McKay, N.D.

P. J. Besl and N.D. McKay, “A method for registration of 3-D shapes,” IEEE Trans. Pattern Anal. Machine Intell. 14(2), 239–256 (1992).
[Crossref]

McLauchlan, P. F.

B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon, “Bundle adjustment–a modern synthesis,” In Vision Algorithms: Theory and Practice, B. Triggs, A. Zisserman, and R. Szeliski, eds. (Springer Berlin Heidelberg, 2000), pp. 298–372.
[Crossref]

Melville, C. D.

Y. Gong, R. S. Johnston, C. D. Melville, and E. J. Seibel, “Axial-stereo 3D optical metrology of internally machined parts using high-quality imaging from a scanning laser endoscope,” in International Symposium on Optomechatronic Technologies (ISOT), Seattle, USA, November 5–7, (2014).

Mitra, K.

K. Mitra and R. Chellappa, “A Scalable Projective Bundle Adjustment Algorithm using the L infinity Norm,” in Sixth Indian Conference on Computer Vision, Graphics and Image Processing (IEEE, 2008), pp. 79–86.

Mor, J. J.

P. H. Calamai and J. J. Mor, “Projected gradient methods for linearly constrained problems,” Math. Program. 39(1), 93–116 (1987).
[Crossref]

Muzzupappa, M.

G. Bianco, A. Gallo, F. Bruno, and M. Muzzupappa, “A comparison between active and passive techniques for underwater 3d applications,” in International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences3816 (2011), pp. 357–363.

Nister, D.

Y. Jeong, D. Nister, D. Steedly, R. Szeliski, and I. S. Kweon, “Pushing the envelope of modern methods for bundle adjustment,” IEEE Trans. Pattern Anal. Mach. Intell. 34(8), 1605–1617 (2012).
[Crossref] [PubMed]

Nistér, D.

C. Engels, H. Stewénius, and D. Nistér, “Bundle adjustment rules,” in Photogrammetric computer vision, (2006).

Owens, A.

D. Crandall, A. Owens, N. Snavely, and D. Huttenlocher, “Discrete-continuous optimization for large-scale structure from motion,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 3001–3008.

Pajdla, T.

C. Albl and T. Pajdla, “Constrained Bundle Adjustment for Panoramic Cameras,” in 18th Computer Vision Winter Workshop, Hernstein, Austria, 4–6 February 2013.

Pollefeys, M.

A. Cohen, C. Zach, S. N. Sinha, and M. Pollefeys, “Discovering and exploiting 3D symmetries in structure from motion,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 1514–1521.

Porter, M. P.

T. D. Soper, M. P. Porter, and E. J. Seibel, “Surface mosaics of the bladder reconstructed from endoscopic video for automated surveillance,” IEEE Trans. Biomed. Eng. 59(6), 1670–1680 (2012).
[Crossref] [PubMed]

Roseborough, J.

B. Klingner, D. Martin, and J. Roseborough, “Street view motion-from-structure-from-motion,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2013), pp. 953–960.

Salvi, J.

J. Salvi, X. Armangu, and J. Batlle, “A comparative review of camera calibrating methods with accuracy evaluation,” Pattern Recognit. 35(7), 1617–1635 (2002).
[Crossref]

Scharstein, D.

S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, “A comparison and evaluation of multi-view stereo reconstruction algorithms,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 519–528.

Seibel, E. J.

Y. Gong, T. D. Soper, V. W. Hou, D. Hu, B. Hannaford, and E. J. Seibel, “Mapping surgical fields by moving a laser-scanning multimodal scope attached to a robot arm,” Proc. SPIE 9036, 90362S (2014).
[Crossref]

Y. Gong, D. Hu, E. J. Seibel, and B. Hannaford, “Accurate 3D virtual reconstruction of surgical field using calibrated trajectories of an image-guided medical robot,” J. Med. Imag. 1(3), 035002 (2014).
[Crossref]

T. D. Soper, M. P. Porter, and E. J. Seibel, “Surface mosaics of the bladder reconstructed from endoscopic video for automated surveillance,” IEEE Trans. Biomed. Eng. 59(6), 1670–1680 (2012).
[Crossref] [PubMed]

C. M. Lee, C. J. Engelbrech, T. D. Soper, F. Helmchen, and E. J. Seibel, “Scanning fiber endoscopy with highly flexible, 1 mm catheterscopes for wide-field, full-color imaging,” J. Biophotonics 3(5-6), 385–407 (2010).
[Crossref]

Y. Gong, R. S. Johnston, C. D. Melville, and E. J. Seibel, “Axial-stereo 3D optical metrology of internally machined parts using high-quality imaging from a scanning laser endoscope,” in International Symposium on Optomechatronic Technologies (ISOT), Seattle, USA, November 5–7, (2014).

Seibel, E.J.

Y. Gong, D. Hu, B. Hannaford, and E.J. Seibel, “Toward real-time endoscopically-guided robotic navigation based on a 3D virtual surgical field model,” Proc. SPIE 9415, 94150C (2015).

Seitz, S. M.

S. Agarwal, N. Snavely, I. Simon, S. M. Seitz, and R. Szeliski, “Building rome in a day,” in IEEE 12th International Conference on Computer Vision (IEEE, 2011), pp. 105–112.

S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, “A comparison and evaluation of multi-view stereo reconstruction algorithms,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 519–528.

C. Wu, S. Agarwal, B. Curless, and S. M. Seitz, “Multicore bundle adjustment,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 3057–3064.

Shaked-Monderer, N.

A. Berman and N. Shaked-Monderer, Completely positive matrices (World Scientific, 2003), Chap. 1.

Simon, I.

S. Agarwal, N. Snavely, I. Simon, S. M. Seitz, and R. Szeliski, “Building rome in a day,” in IEEE 12th International Conference on Computer Vision (IEEE, 2011), pp. 105–112.

Sinha, S. N.

A. Cohen, C. Zach, S. N. Sinha, and M. Pollefeys, “Discovering and exploiting 3D symmetries in structure from motion,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 1514–1521.

Snavely, N.

S. Agarwal, N. Snavely, I. Simon, S. M. Seitz, and R. Szeliski, “Building rome in a day,” in IEEE 12th International Conference on Computer Vision (IEEE, 2011), pp. 105–112.

D. Crandall, A. Owens, N. Snavely, and D. Huttenlocher, “Discrete-continuous optimization for large-scale structure from motion,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 3001–3008.

Soper, T. D.

Y. Gong, T. D. Soper, V. W. Hou, D. Hu, B. Hannaford, and E. J. Seibel, “Mapping surgical fields by moving a laser-scanning multimodal scope attached to a robot arm,” Proc. SPIE 9036, 90362S (2014).
[Crossref]

T. D. Soper, M. P. Porter, and E. J. Seibel, “Surface mosaics of the bladder reconstructed from endoscopic video for automated surveillance,” IEEE Trans. Biomed. Eng. 59(6), 1670–1680 (2012).
[Crossref] [PubMed]

C. M. Lee, C. J. Engelbrech, T. D. Soper, F. Helmchen, and E. J. Seibel, “Scanning fiber endoscopy with highly flexible, 1 mm catheterscopes for wide-field, full-color imaging,” J. Biophotonics 3(5-6), 385–407 (2010).
[Crossref]

Steedly, D.

Y. Jeong, D. Nister, D. Steedly, R. Szeliski, and I. S. Kweon, “Pushing the envelope of modern methods for bundle adjustment,” IEEE Trans. Pattern Anal. Mach. Intell. 34(8), 1605–1617 (2012).
[Crossref] [PubMed]

Stewénius, H.

C. Engels, H. Stewénius, and D. Nistér, “Bundle adjustment rules,” in Photogrammetric computer vision, (2006).

Sturm, P.

A. Bartoli and P. Sturm, “Constrained structure and motion from multiple uncalibrated views of a piecewise planar scene,” Int. J. Comput. Vision 52(1), 45–64 (2003).
[Crossref]

Szeliski, R.

Y. Jeong, D. Nister, D. Steedly, R. Szeliski, and I. S. Kweon, “Pushing the envelope of modern methods for bundle adjustment,” IEEE Trans. Pattern Anal. Mach. Intell. 34(8), 1605–1617 (2012).
[Crossref] [PubMed]

S. Agarwal, N. Snavely, I. Simon, S. M. Seitz, and R. Szeliski, “Building rome in a day,” in IEEE 12th International Conference on Computer Vision (IEEE, 2011), pp. 105–112.

S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, “A comparison and evaluation of multi-view stereo reconstruction algorithms,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 519–528.

R. Szeliski and P. H. Torr, “Geometrically constrained structure from motion: Points on planes,” In 3D Structure from Multiple Images of Large-Scale Environments, R. Koch and L. V. Gool, eds. (Springer Berlin Heidelberg, 1998), pp. 171–186.
[Crossref]

Torr, P. H.

R. Szeliski and P. H. Torr, “Geometrically constrained structure from motion: Points on planes,” In 3D Structure from Multiple Images of Large-Scale Environments, R. Koch and L. V. Gool, eds. (Springer Berlin Heidelberg, 1998), pp. 171–186.
[Crossref]

Triggs, B.

B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon, “Bundle adjustment–a modern synthesis,” In Vision Algorithms: Theory and Practice, B. Triggs, A. Zisserman, and R. Szeliski, eds. (Springer Berlin Heidelberg, 2000), pp. 298–372.
[Crossref]

Wong, K. H.

K. H. Wong and M. M. Y. Chang, “3D model reconstruction by constrained bundle adjustment,” in Proceedings of the 17th International Conference on Pattern Recognition (IEEE, 2004), pp. 902–905.

Wu, C.

C. Wu, S. Agarwal, B. Curless, and S. M. Seitz, “Multicore bundle adjustment,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 3057–3064.

C. Wu, “Towards linear-time incremental structure from motion,” in International Conference on 3D Vision (IEEE, 2013), pp. 127–134.

Zach, C.

A. Cohen, C. Zach, S. N. Sinha, and M. Pollefeys, “Discovering and exploiting 3D symmetries in structure from motion,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 1514–1521.

Zhang, S.

Y. Gong and S. Zhang, “Improving 4-D shape measurement by using projector defocusing,” Proc. SPIE 7790, 77901A (2010).
[Crossref]

Y. Gong and S. Zhang, “Ultrafast 3-D shape measurement with an off-the-shelf DLP projector,” Opt. Express 18(19), 19743–19754 (2010).
[Crossref] [PubMed]

Zhang, Y.

Y. Zhang, K. Hu, and R. Huang, “Bundle adjustment with additional constraints applied to imagery of the Dunhuang wall paintings,” ISPRS J. Photogramm. Remote Sens. 72, 113–120 (2012).
[Crossref]

Zhang, Z.

Z. Zhang, “Microsoft kinect sensor and its effect,” MultiMedia IEEE 19(2), 4–10 (2012).
[Crossref]

ACM Trans. Math. Software (1)

M. I. A. Lourakis and A. A. Argyros, “SBA: A software package for generic sparse bundle adjustment,” ACM Trans. Math. Software 36(1), 1–30 (2009)
[Crossref]

Adv. Opt. Photon. (1)

IEEE Trans. Biomed. Eng. (1)

T. D. Soper, M. P. Porter, and E. J. Seibel, “Surface mosaics of the bladder reconstructed from endoscopic video for automated surveillance,” IEEE Trans. Biomed. Eng. 59(6), 1670–1680 (2012).
[Crossref] [PubMed]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

Y. Jeong, D. Nister, D. Steedly, R. Szeliski, and I. S. Kweon, “Pushing the envelope of modern methods for bundle adjustment,” IEEE Trans. Pattern Anal. Mach. Intell. 34(8), 1605–1617 (2012).
[Crossref] [PubMed]

IEEE Trans. Pattern Anal. Machine Intell. (1)

P. J. Besl and N.D. McKay, “A method for registration of 3-D shapes,” IEEE Trans. Pattern Anal. Machine Intell. 14(2), 239–256 (1992).
[Crossref]

Int. J. Comput. Vision (1)

A. Bartoli and P. Sturm, “Constrained structure and motion from multiple uncalibrated views of a piecewise planar scene,” Int. J. Comput. Vision 52(1), 45–64 (2003).
[Crossref]

ISPRS J. Photogramm. Remote Sens. (1)

Y. Zhang, K. Hu, and R. Huang, “Bundle adjustment with additional constraints applied to imagery of the Dunhuang wall paintings,” ISPRS J. Photogramm. Remote Sens. 72, 113–120 (2012).
[Crossref]

J. Biophotonics (1)

C. M. Lee, C. J. Engelbrech, T. D. Soper, F. Helmchen, and E. J. Seibel, “Scanning fiber endoscopy with highly flexible, 1 mm catheterscopes for wide-field, full-color imaging,” J. Biophotonics 3(5-6), 385–407 (2010).
[Crossref]

J. Med. Imag. (1)

Y. Gong, D. Hu, E. J. Seibel, and B. Hannaford, “Accurate 3D virtual reconstruction of surgical field using calibrated trajectories of an image-guided medical robot,” J. Med. Imag. 1(3), 035002 (2014).
[Crossref]

Math. Program. (1)

P. H. Calamai and J. J. Mor, “Projected gradient methods for linearly constrained problems,” Math. Program. 39(1), 93–116 (1987).
[Crossref]

MultiMedia IEEE (1)

Z. Zhang, “Microsoft kinect sensor and its effect,” MultiMedia IEEE 19(2), 4–10 (2012).
[Crossref]

Opt. Express (1)

Pattern Recognit. (1)

J. Salvi, X. Armangu, and J. Batlle, “A comparative review of camera calibrating methods with accuracy evaluation,” Pattern Recognit. 35(7), 1617–1635 (2002).
[Crossref]

Proc. SPIE (3)

Y. Gong and S. Zhang, “Improving 4-D shape measurement by using projector defocusing,” Proc. SPIE 7790, 77901A (2010).
[Crossref]

Y. Gong, D. Hu, B. Hannaford, and E.J. Seibel, “Toward real-time endoscopically-guided robotic navigation based on a 3D virtual surgical field model,” Proc. SPIE 9415, 94150C (2015).

Y. Gong, T. D. Soper, V. W. Hou, D. Hu, B. Hannaford, and E. J. Seibel, “Mapping surgical fields by moving a laser-scanning multimodal scope attached to a robot arm,” Proc. SPIE 9036, 90362S (2014).
[Crossref]

Other (18)

M. I. A. Lourakis and A. A. Argyros, “Is Levenberg-Marquardt the most efficient optimization algorithm for implementing bundle adjustment?,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2005), pp. 1526–1531.

A. Berman and N. Shaked-Monderer, Completely positive matrices (World Scientific, 2003), Chap. 1.

R. Szeliski and P. H. Torr, “Geometrically constrained structure from motion: Points on planes,” In 3D Structure from Multiple Images of Large-Scale Environments, R. Koch and L. V. Gool, eds. (Springer Berlin Heidelberg, 1998), pp. 171–186.
[Crossref]

C. Wu, S. Agarwal, B. Curless, and S. M. Seitz, “Multicore bundle adjustment,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 3057–3064.

D. G. Lowe, “Object recognition from local scale-invariant features,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 1999), pp. 1150–1157.

A. Cohen, C. Zach, S. N. Sinha, and M. Pollefeys, “Discovering and exploiting 3D symmetries in structure from motion,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 1514–1521.

B. Klingner, D. Martin, and J. Roseborough, “Street view motion-from-structure-from-motion,” in Proceedings of IEEE Conference on Computer Vision (IEEE, 2013), pp. 953–960.

G. Bianco, A. Gallo, F. Bruno, and M. Muzzupappa, “A comparison between active and passive techniques for underwater 3d applications,” in International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences3816 (2011), pp. 357–363.

K. H. Wong and M. M. Y. Chang, “3D model reconstruction by constrained bundle adjustment,” in Proceedings of the 17th International Conference on Pattern Recognition (IEEE, 2004), pp. 902–905.

C. Albl and T. Pajdla, “Constrained Bundle Adjustment for Panoramic Cameras,” in 18th Computer Vision Winter Workshop, Hernstein, Austria, 4–6 February 2013.

S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, “A comparison and evaluation of multi-view stereo reconstruction algorithms,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2006), pp. 519–528.

Y. Gong, R. S. Johnston, C. D. Melville, and E. J. Seibel, “Axial-stereo 3D optical metrology of internally machined parts using high-quality imaging from a scanning laser endoscope,” in International Symposium on Optomechatronic Technologies (ISOT), Seattle, USA, November 5–7, (2014).

S. Agarwal, N. Snavely, I. Simon, S. M. Seitz, and R. Szeliski, “Building rome in a day,” in IEEE 12th International Conference on Computer Vision (IEEE, 2011), pp. 105–112.

C. Wu, “Towards linear-time incremental structure from motion,” in International Conference on 3D Vision (IEEE, 2013), pp. 127–134.

D. Crandall, A. Owens, N. Snavely, and D. Huttenlocher, “Discrete-continuous optimization for large-scale structure from motion,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 3001–3008.

B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon, “Bundle adjustment–a modern synthesis,” In Vision Algorithms: Theory and Practice, B. Triggs, A. Zisserman, and R. Szeliski, eds. (Springer Berlin Heidelberg, 2000), pp. 298–372.
[Crossref]

K. Mitra and R. Chellappa, “A Scalable Projective Bundle Adjustment Algorithm using the L infinity Norm,” in Sixth Indian Conference on Computer Vision, Graphics and Image Processing (IEEE, 2008), pp. 79–86.

C. Engels, H. Stewénius, and D. Nistér, “Bundle adjustment rules,” in Photogrammetric computer vision, (2006).

Supplementary Material (1)

» Media 1: MP4 (4810 KB)     

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (6)

Fig. 1
Fig. 1 Conceptual diagram of an applicable case of BCBA where a sequence of images are captured by inaccurate cameras around a stationary object.
Fig. 2
Fig. 2 The experiment setup. (a) the SFE of 1.6 mm outer diameter with camera frame rate at 30 Hz; (b) the CAD design of the object. It is a spherical dome with maximum radius of 17.5 mm and depth of 10 mm; (c) the 3D printed phantom with near-realistic surgical features; (d) the experiment setup on a micro-positioning stage, which can provide high accurate data of the camera positions with accuracy of 0.01 mm.
Fig. 3
Fig. 3 Matching features in a pair of frames with image overlap were lined up in color. One tenth of the matching features were randomly chosen for better visualization.
Fig. 4
Fig. 4 The procedures of BA and BCBA, respectively. (a)-(d) shows the reconstructed 3D point cloud by BA was shrinking to a flat geometry. (e)-(h) shows only minor adjustment of the 3D points happened in the BCBA optimization process due to the bound constraints.
Fig. 5
Fig. 5 ( Media 1) The comparison of 3D reconstruction results between BA and BCBA. (a)-(c) and (d)-(f) show the reconstructed 3D point clouds, 3D surfaces and the depth maps of BA and BCBA, respectively.
Fig. 6
Fig. 6 The comparison of ICP error analysis between BA and BCBA. (a) and (d) show the 3D alignment of CAD point cloud and reconstructed surfaces for BA and BCBA, respectively; (b) and (e) show the distance maps of the alignment for BA and BCBA, respectively; (c) and (f) show the histograms of the distance of each reconstructed point to CAD model for BA and BCBA, respectively.

Tables (3)

Tables Icon

Table 1 The experiment result comparison of BA and BCBA algorithm.

Tables Icon

Table 2 The comparisons of ICP error and computation time of the BA and BCBA algorithms with different noise.

Tables Icon

Table 3 The comparison of BA versus BCBA with the constraints on FOV and Qz.

Equations (13)

Equations on this page are rendered with MathJax. Learn more.

ω [ u i j v i j 1 ] = K [ R j t j ] [ Q i j 1 ] x i j = u i j ( 1 + k 1 ρ 2 + k 2 ρ 4 ) y i j = v i j ( 1 + k 1 ρ 2 + k 2 ρ 4 ) ρ 2 = u i j 2 + v i j 2
Objective min f ( s ) = 1 2 r ( s ) T r ( s ) Subject to l i s i u i ; i = 1 , 2 , 3 , , N
f ( s + δ ) f ( s ) + g ( s ) T δ + 1 2 δ T H ( s ) δ with g ( s ) d f d s ( s ) H ( s ) d 2 f d s 2 ( s )
H ¯ ( s ) δ = g ( s )
H ¯ ( s ) = [ H ¯ c c H ¯ c p H ¯ p c H ¯ p p ] = [ J c T J c J c T J p J p T J c J p T J p ] .
[ H ¯ c c H ¯ c p H ¯ p c H ¯ p p ] [ δ c δ p ] = [ g c g p ] .
( H ¯ c c H ¯ c p H ¯ p p 1 H ¯ p c ) δ c = ( H ¯ c p H ¯ p p 1 ) g p g c δ p = H ¯ p p 1 ( g p H ¯ p c δ c )
B δ = g ( s ) , with B = H ¯ ( s ) + λ D
pro j ( s ) = min { max { l , s } , u } .
A ( s ) = { i | s i = l i and g i > 0 or s i = u i and g i < 0 }
g ^ i = { g i , i I ( s ) 0 , otherwise
H ^ i j = { H ¯ i j if i I ( s ) and j I ( s ) H ¯ i i if i A ( s ) 0 , otherwise
B ^ δ = g ^ ( s ) , with B ^ = H ^ ( s ) + λ D ^

Metrics