VCLab

 

RESEARCH AREAS   PEOPLE   PUBLICATIONS   COURSES   ABOUT US
Home / Publications

indicator

Computer Vision and Pattern Recognition (CVPR 2021)

 
High-Quality Stereo Image Restoration from Double Refraction
 
  Hakyeong Kim Andreas Meuleman Daniel S. Jeon Min H. Kim  
 
KAIST
 
 
  (a) an input double-refraction image, (b) an oray image result, (c) an e-ray result, (d) a dense disparity map estimated from our stereo images. Compared to ground truth, PSNRs and SSIMs of the entire o-/e-ray images are 40.80/36.90 dB, SSIM: 0.9854/0.9799, respectively. Refer to the supplemental video for real-time demo.  
     
   
  CVPR 2021 presentation
     
   
  Supplemental video
   
  Abstract
   
 

Single-shot monocular birefractive stereo methods have been used for estimating sparse depth from double refraction over edges. They also obtain an ordinary-ray (o-ray) image concurrently or subsequently through additional post-processing of depth densification and deconvolution. However, when an extraordinary-ray (e-ray) image is restored to acquire stereo images, the existing methods suffer from very severe restoration artifacts due to a low signal-to-noise ratio of input e-ray image or depth/deconvolution errors. In this work, we present a novel stereo image restoration network that can restore stereo images directly from a double-refraction image. First, we built a physically faithful birefractive stereo imaging dataset by simulating the double refraction phenomenon with existing RGB-D datasets. Second, we formulated a joint stereo restoration problem that accounts for not only geometric relation between o-/e-ray images but also joint optimization of restoring both stereo images. We trained our model with our birefractive image dataset in an end-to-end manner. Our model restores high-quality stereo images directly from double refraction in real-time, enabling high-quality stereo video using a monocular camera. Our method also allows us to estimate dense depth maps from stereo images using a conventional stereo method. We evaluate the performance of our method experimentally and synthetically with the ground truth. Results validate that our stereo image restoration network outperforms the existing methods with high accuracy. We demonstrate several image-editing applications using our high-quality stereo images and dense depth maps.

   
  BibTeX
 
@InProceedings{Kim_2021_CVPR,
   author = {Hakyeong Kim and Andreas Meuleman and
      Daniel S. Jeon and Min H. Kim},
   title = {High-Quality Stereo Image Restoration 
      from Double Refraction},
   booktitle = {IEEE Conference on Computer Vision and 
      Pattern Recognition (CVPR)},
   month = {June},
   year = {2021}
}       
   
   
icon
Preprint paper:
PDF (6.3MB)
icon
Supplemental
material #1:
PDF (135KB)
icon
Supplemental
material #2:
PDF (7.1MB)
www GitHub (TBA)
website

Hosted by Visual Computing Laboratory, School of Computing, KAIST.

KAIST