VCLab

 

RESEARCH AREAS   PEOPLE   PUBLICATIONS   COURSES   ABOUT US
Home / Publications

indicator

Computer Vision and Pattern Recognition (CVPR 2020)

 
Single-shot Monocular RGB-D Imaging using Uneven Double Refraction
 
 
  Andreas Meuleman* Seung-Hwan Baek* Felix Heide Min H. Kim  
           
  KAIST Princeton University  
           
    * Equal contribution  
  teaser
  (a) Our prototype, consisting of a linear polarizer and a calcite crystal in front of a conventional camera. (b) and (c) estimated depth map and restored color image from the input. Our algorithm provides both sparse depth and clear RGB image within 34 ms.  
     
     
   
  Supplemental video for a real-time demo
   
  Abstract
   
  Cameras that capture color and depth information have become an essential imaging modality for applications in robotics, autonomous driving, virtual, and augmented reality. Existing RGB-D cameras rely on multiple sensors or active illumination with specialized sensors. In this work, we propose a method for monocular single-shot RGB-D imaging. Instead of learning depth from single-image depth cues, we revisit double-refraction imaging using a birefractive medium, measuring depth as the displacement of differently refracted images superimposed in a single capture. However, existing double-refraction methods are orders of magnitudes too slow to be used in real-time applications, e.g., in robotics, and provide only inaccurate depth due to correspondence ambiguity in double reflection. We resolve this ambiguity optically by leveraging the orthogonality of the two linearly polarized rays in double refraction -- introducing uneven double refraction by adding a linear polarizer to the birefractive medium. Doing so makes it possible to develop a real-time method for reconstructing sparse depth and color simultaneously in real-time. We validate the proposed method, both synthetically and experimentally, and demonstrate 3D object detection and photographic applications.
   
  BibTeX
 
@InProceedings{Meuleman_2020_CVPR,
author = {Andreas Meuleman and Seung-Hwan Baek and Felix Heide and Min H. Kim},
title = {Single-shot Monocular RGB-D Imaging using Uneven Double Refraction},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}
   
   
icon
Preprint paper:
PDF (8.5MB)
icon
Supplemental document:
PDF
www IEEE CVF
website
www GitHub
 
 

Hosted by Visual Computing Laboratory, School of Computing, KAIST.

KAIST