Home / Publications


Computer Vision and Pattern Recognition (CVPR 2023)

Spatio-Focal Bidirectional Disparity Estimation from a Dual-Pixel Image
  Donggun Kim Hyeonjoong Jang Inchul Kim Min H. Kim  
  Captured images from the traditional binocular stereo camera and dual-pixel camera, with a focus-plane depth at the middle object. Upper part of the image is a left image, and lower part is a right image from each camera. Arrows note parallax, which is unidirectional in stereo imaging and bidirectional in dual-pixel imaging.  

Dual-pixel photography is monocular RGB-D photography with an ultra-high resolution, enabling many applications in computational photography. However, there are still several challenges to fully utilizing dual-pixel photography. Unlike the conventional stereo pair, the dual pixel exhibits a bidirectional disparity that includes positive and negative values, depending on the focus plane depth in an image. Furthermore, capturing a wide range of dual-pixel disparity requires a shallow depth of field, resulting in a severely blurred image, degrading depth estimation performance. Recently, several data-driven approaches have been proposed to mitigate these two challenges. However, due to the lack of the ground-truth dataset of the dual-pixel disparity, existing data-driven methods estimate either inverse depth or blurriness map. In this work, we propose a self-supervised learning method that learns bidirectional disparity by utilizing the nature of anisotropic blur kernels in dual-pixel photography. We observe that the dual-pixel left/right images have reflective-symmetric anisotropic kernels, so their sum is equivalent to that of a conventional image. We take a self-supervised training approach with the novel kernel-split symmetry loss accounting for the phenomenon. Our method does not rely on a training dataset of dual-pixel disparity that does not exist yet. Our method can estimate a complete disparity map with respect to the focus-plane depth from a dual-pixel image, outperforming the baseline dual-pixel methods.

  CVPR 2023 presentation
   author = {Donggun Kim and Hyeojoong Jang and Inchul Kim 
            and Min H. Kim},
   title = {Spatio-Focal Bidirectional Disparity Estimation 
            from a Dual-Pixel Image},
   booktitle = {IEEE Conference on Computer Vision and 
      Pattern Recognition (CVPR)},
   month = {June},
   year = {2023}
Preprint paper:
PDF (4.8MB)
material #1:
PDF (8.6MB)
www GitHub

Hosted by Visual Computing Laboratory, School of Computing, KAIST.