IEEE Conference on Computational Photography (ICCP) 2022 |
|
Differentiable Appearance Acquisition from a Flash/No-flash RGB-D Pair |
|
|
Hyun Jin Ku |
Hyunho Ha |
Joo Ho Lee |
Dahyun Kang |
James Tompkin |
Min H. Kim |
|
|
|
KAIST |
Sogang University |
Brown University |
|
|
 |
|
High-quality appearance reconstruction from a pair
of exposures. We use a flash/no-flash RGB-D pair from a
smartphone (a) to recover geometry, lighting, and Cook-
Torrance SVBRDF material properties (d). Re-renderings
(c) are close to the original photograph (b) without using
impractical lighting rigs or multiple capture positions. |
|
|
|
|
|
|
|
Abstract |
|
|
|
Reconstructing 3D objects in natural environments requires solving the ill-posed problem of geometry, spatially-varying material, and lighting estimation. As such, many approaches impractically constrain to a dark environment, use controlled lighting rigs, or use few handheld captures but suffer reduced quality. We develop a method that uses just two smartphone exposures captured in ambient lighting to reconstruct appearance more accurately and practically than baseline methods. Our insight is that we can use a flash/no-flash RGB-D pair to pose an inverse rendering problem using point lighting. This allows efficient differentiable rendering to optimize depth and normals from a good initialization and so also the simultaneous optimization of diffuse environment illumination and SVBRDF material. We find that this reduces diffuse albedo error by 25%, specular error by 46%, and normal error by 30% against singleand paired-image baselines that use learning-based techniques. Given that our approach is practical for everyday solid objects, we enable photorealistic relighting for mobile photography and easier content creation for augmented reality.
|
|
|
|
|
|
@InProceedings{MobileSVBRDF:ICCP:2022,
author = {Hyun Jin Ku and Hyunho Ha and Joo Ho Lee and
Dahyun Kang and James Tompkin and Min H. Kim},
title = {Differentiable Appearance Acquisition from a
Flash/No-flash RGB-D Pair},
booktitle = {Proc. IEEE International Conference on
Computational Photography (ICCP) 2022)},
year = {2022},
month = {August},
}
|
|
|
|
|
|
|
Preprint paper:
TBA
|
|
Supplemental
material #1:
TBA |
|
|
|
|
|
Hosted by Visual Computing Laboratory, School of Computing, KAIST.
|
 |
|