Home / Publications


Computer Vision and Pattern Recognition (CVPR 2021, Oral)

Real-Time Sphere Sweeping Stereo from Multiview Fisheye Images
  Andreas Meuleman Hyeonjoong Jang Daniel S. Jeon Min H. Kim  
  (Left) Four input fisheye images. (Middle and Right) Our results of omnidirectional panorama and dense distance map (shown as the inverse of distance). It took just 34 ms per frame on this device. Refer to the supplemental video for real-time demo.  
  CVPR 2021 oral presentation
  Supplemental video

A set of cameras with fisheye lenses have been used to capture a wide field of view. The traditional scan-line stereo algorithms based on epipolar geometry are directly inapplicable to this non-pinhole camera setup due to optical characteristics of fisheye lenses; hence, existing complete 360° RGB-D imaging systems have rarely achieved realtime performance yet. In this paper, we introduce an efficient sphere-sweeping stereo that can run directly on multiview fisheye images without requiring additional spherical rectification. Our main contributions are: First, we introduce an adaptive spherical matching method that accounts for each input fisheye camera’s resolving power concerning spherical distortion. Second, we propose a fast inter-scale bilateral cost volume filtering method that refines distance in noisy and textureless regions with optimal complexity of O(n). It enables real-time dense distance estimation while preserving edges. Lastly, the fisheye color and distance images are seamlessly combined into a complete 360° RGB-D image via fast inpainting of the dense distance map. We demonstrate an embedded 360° RGB-D imaging prototype composed of a mobile GPU and four fisheye cameras. Our prototype is capable of capturing complete 360° RGB-D videos with a resolution of two megapixels at 29 fps. Results demonstrate that our real-time method outperforms traditional omnidirectional stereo and learning-based omnidirectional stereo in terms of accuracy and performance.

   author = {Andreas Meuleman and Hyeonjoong Jang and 
      Daniel S. Jeon and Min H. Kim},
   title = {Real-Time Sphere Sweeping Stereo from 
      Multiview Fisheye Images},
   booktitle = {IEEE Conference on Computer Vision and 
      Pattern Recognition (CVPR)},
   month = {June},
   year = {2021}
Preprint paper:
PDF (2.6MB)
material #1:
PDF (13.7MB)
www 360° RGBD image dataset
ZIP (908MB)
www GitHub

Hosted by Visual Computing Laboratory, School of Computing, KAIST.