VCLab

 

RESEARCH AREAS   PEOPLE   PUBLICATIONS   COURSES   ABOUT US
Home / Publications

indicator

International Conference on Computer Vision (ICCV) 2021

 
Single-shot Hyperspectral-Depth Imaging with Learned Diffractive Optics
 
  Seung-Hwan Baek†*   Hayato Ikoma§ Daniel S. Jeon  
  Yuqi Li Wolfgang Heidrich Gordon Wetzstein§   Min H. Kim  
           
  KAIST § Stanford University KAUST * Princeton University  
 
 
  (a) Our compact single-shot HS-D imaging method uses an optimized DOE that creates (b) a PSF that varies with spectrum and depth. (c)--(e) It encodes spectral-depth information in the captured image, from which we reconstruct a hyperspectral image and a depth map simultaneously.  
     
   
  Abstract
   
 

Imaging depth and spectrum have been extensively studied in isolation from each other for decades. Recently, hyperspectral-depth (HS-D) imaging emerges to capture both information simultaneously by combining two different imaging systems; one for depth, the other for spectrum. While being accurate, this combinational approach induces increased form factor, cost, capture time, and alignment/registration problems. In this work, departing from the combinational principle, we propose a compact single-shot monocular HS-D imaging method. Our method uses a diffractive optical element (DOE), the point spread function of which changes with respect to both depth and spectrum. This enables us to reconstruct spectrum and depth from a single captured image. To this end, we develop a differentiable simulator and a neural-network-based reconstruction that are jointly optimized via automatic differentiation. To facilitate learning the DOE, we present a first HS-D dataset by building a benchtop HS-D imager that acquires high-quality ground truth. We evaluate our method with synthetic and real experiments by building an experimental prototype and achieve state-of-the-art HS-D imaging results.

   
  BibTeX
 
@InProceedings{Baek_2021_ICCV,
author = {Seung-Hwan Baek and Hayato Ikoma and Daniel S. Jeon and Yuqi Li
          and Wolfgang Heidrich and Gordon Wetzstein and Min H. Kim},
title = {Single-shot Hyperspectral-Depth Imaging with Learned Diffractive 
         Optics},
booktitle = {Proc. IEEE International Conference on Computer Vision (ICCV)},
month = {Oct},
year = {2021}
}
   
   
icon
EGSR preprint:
(TBA)
icon
Supplemental
document:
(TBA)
icon
Presentation
slides:
(TBA)
www KAIST dataset
(TBA)

Hosted by Visual Computing Laboratory, School of Computing, KAIST.

KAIST