Frontier of Science and Technology

A review of hot topics in image-based virtual lighting

  • WU Hongyu ,
  • JIN Xin
Expand
  • 1. State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, Beijing 100191, China;
    2. Department of Cyber Security, Beijing Electronic Science and Technology Institute, Beijing 100070, China

Received date: 2019-02-01

  Revised date: 2019-05-31

  Online published: 2020-05-11

Abstract

The image-based virtual relighting (IBVR) is to directly modify the lighting effect of an object in the image or to estimate the lighting condition of the image. Unlike the traditional image processing technology, the images are processed in the domain of lighting, retaining the visual characteristics of an object. This paper reviews the important research advances of the IBVR in 2018, as well as its trend of development.

Cite this article

WU Hongyu , JIN Xin . A review of hot topics in image-based virtual lighting[J]. Science & Technology Review, 2020 , 38(6) : 141 -152 . DOI: 10.3981/j.issn.1000-7857.2020.06.020

References

[1] Wikipedia. VES[EB/OL].[2019-07-21]. https://en.wikipedia.org/wiki/Image-based_modeling_and_rendering.
[2] Paul Debevec. VES[EB/OL].[2019-07-21]. http://www.pauldebevec.com/.
[3] Chabert C F, Einarsson P, Jones A, et al. Relighting human locomotion with flowed reflectance fields[C]//ACM SIGGRAPH 2006 Sketches. New York:ACM, 2006:76.
[4] Chen J, Su G, He J, et al. Face image relighting using locally constrained global optimization[C]//European Conference on Computer Vision. Springer, Berlin, Heidelberg, 2010:44-57.
[5] Fuchs M, Blanz V, Lensch H, et al. Adaptive sampling of reflectance fields[J]. ACM Transactions on Graphics, 2007, 26(2):10.
[6] Ren P, Wang J, Gong M, et al. Global illumination with radiance regression functions[J]. ACM Transactions on Graphics, 2013, 32(4):130.
[7] Meka A, Haene C, Pandey R, et al. Deep reflectance fields-high-quality facial reflectance field inference from color gradient illumination[J]. ACM Transactions on Graphics, 2019, 38(4):77.
[8] Ren P, Dong Y, Lin S, et al. Image based relighting using neural networks[J]. ACM Transactions on Graphics, 2015, 34(4):111.
[9] Ng R, Ramamoorthi R, Hanrahan P. All-frequency shadows using non-linear wavelet lighting approximation[J]. ACM Transactions on Graphics. 2003, 22(3):376-381.
[10] Xu Z, Sunkavalli K, Hadap S, et al. Deep image-based relighting from optimal sparse samples[J]. ACM Transactions on Graphics, 2018, 37(4):126.
[11] Barrow H, Tenenbaum J. Recovering intrinsic scene characteristics[J]. Computer Vision System, 1978, 2(3-26):2.
[12] Zhang R, Tsai P S, Cryer J E, et al. Shape-from-shading:A survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1999, 21(8):690-706.
[13] Chai M, Luo L, Sunkavalli K, et al. High-quality hair modeling from a single portrait photo[J]. ACM Transactions on Graphics, 2015, 34(6):204.
[14] Kanamori Y, Endo Y. Relighting humans:Occlusionaware inverse rendering for full body human images[J]. ACM Transactions on Graphics, 2018, 37(6):1-270.
[15] Sun T, Barron J T, Tsai Y T, et al. Single image portrait relighting[J]. ACM Transactions on Graphics, 2019, 38(4):79.
[16] Wu J H, Saito S. Interactive relighting in single low-dynamic range images[J]. ACM Transactions on Graphics, 2017, 36(2):18.
[17] Philip J, Gharbi M, Zhou T, et al. Multi-view relighting using a geometry-aware network[J]. ACM Transactions on Graphics, 2019, 38(4):78.
[18] Shashua A, Riklin-Raviv T. The quotient image:Classbased re-rendering and recognition with varying illuminations[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(2):129-139.
[19] Peers P, Tamura N, Matusik W, et al. Post-production facial performance relighting using reflectance transfer[J]. ACM Transactions on Graphics, 2007, 26(3):52.
[20] Li Q, Yin W, Deng Z. Image-based face illumination transferring using logarithmic total variation models[J]. The Visual Computer, 2010, 26(1):41.
[21] Shu Z, Hadap S, Shechtman E, et al. Portrait lighting transfer using a mass transport approach[J]. ACM Transactions on Graphics, 2018, 37(1):2.
[22] Wikipedia. VES[EB/OL].[2019-07-21]. https://en.wikipedia.org/wiki/Augmented_reality.
[23] Yi R, Zhu C, Tan P, et al. Faces as lighting probes via unsupervised deep highlight extraction[C]//European Conference on Computer Vision. Springer, Berlin, Heidelberg, 2018:317-333.
[24] Hold-Geoffroy Y, Sunkavalli K, Hadap S, et al. Deep outdoor illumination estimation[C]//IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, Washington, DC, 2017:2373-2382.
[25] Gardner M A, Sunkavalli K, Yumer E, et al. Learning to predict indoor illumination from a single image[J]. ACM Transactions on Graphics, 2017, 36(6):176.
[26] SUN360. VES[EB/OL].[2019-07-21]. http://people.csail.mit.edu/jxiao/SUN360/index_high.html.
[27] Barreira J, Bessa M, Barbosa L, et al. A context-aware method for authentically simulating outdoors shadows for mobile augmented reality[J]. IEEE transactions on visualization and computer graphics, 2018, 24(3):1223-1231.
[28] Karsch K, Hedau V, Forsyth D, et al. Rendering synthetic objects into legacy photographs[J]. ACM Transactions on Graphics, 2011, 30(6):157.
[29] Karsch K, Sunkavalli K, Hadap S, et al. Automatic scene inference for 3d object compositing[J]. ACM Transactions on Graphics, 2014, 33(3):32.
[30] Karsch K, Liu C, Kang S B. Depth transfer:Depth extraction from video using non-parametric sampling[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(11):2144-2158.
[31] Liao Z, Karsch K, Forsyth D. An approximate shading model for object relighting[C]//IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, Washington, DC, 2015:5307-5314.
Outlines

/