VCLab

 

RESEARCH AREAS   PEOPLE   PUBLICATIONS   COURSES   ABOUT US
Home / Publications

indicator

IEEE Conference on Computational Photography (ICCP) 2022

 
Differentiable Appearance Acquisition from a Flash/No-flash RGB-D Pair
 
  Hyun Jin Ku Hyunho Ha Joo Ho Lee Dahyun Kang James Tompkin Min H. Kim  
 
KAIST Sogang University Brown University  
 
  High-quality appearance reconstruction from a pair of exposures. We use a flash/no-flash RGB-D pair from a smartphone (a) to recover geometry, lighting, and Cook- Torrance SVBRDF material properties (d). Re-renderings (c) are close to the original photograph (b) without using impractical lighting rigs or multiple capture positions.  
     
   
  Abstract
   
 

Reconstructing 3D objects in natural environments requires solving the ill-posed problem of geometry, spatially-varying material, and lighting estimation. As such, many approaches impractically constrain to a dark environment, use controlled lighting rigs, or use few handheld captures but suffer reduced quality. We develop a method that uses just two smartphone exposures captured in ambient lighting to reconstruct appearance more accurately and practically than baseline methods. Our insight is that we can use a flash/no-flash RGB-D pair to pose an inverse rendering problem using point lighting. This allows efficient differentiable rendering to optimize depth and normals from a good initialization and so also the simultaneous optimization of diffuse environment illumination and SVBRDF material. We find that this reduces diffuse albedo error by 25%, specular error by 46%, and normal error by 30% against singleand paired-image baselines that use learning-based techniques. Given that our approach is practical for everyday solid objects, we enable photorealistic relighting for mobile photography and easier content creation for augmented reality.

   
  BibTeX
 
@InProceedings{MobileSVBRDF:ICCP:2022,
  author  = {Hyun Jin Ku  and Hyunho Ha and Joo Ho Lee and 
             Dahyun Kang and James Tompkin and Min H. Kim},
  title   = {Differentiable Appearance Acquisition from a 
             Flash/No-flash RGB-D Pair},
  booktitle = {Proc. IEEE International Conference on 
               Computational Photography (ICCP) 2022)},
  year    = {2022},
  month = {August},
  }        
   
   
icon
Preprint paper:
TBA
icon
Supplemental
material #1:
TBA
 

Hosted by Visual Computing Laboratory, School of Computing, KAIST.

KAIST