Computer Vision and Pattern Recognition (CVPR 2020, Oral, Best Paper Finalist)
TextureFusion: High-Quality Texture Acquisition for Real-Time RGB-D Scanning
Joo Ho Lee
Hyunho Ha
Yue Dong
Xin Tong
Min H. Kim
KAIST
Microsoft Research Asia
We compare per-voxel color representation (a) with conventional texture representation without optimization (b). Compared with global optimization only (c), our texture fusion (d) can achieve high-quality color texture in real-time RGB-D scanning.
CVPR 2020 oral presentation
Supplemental video for a real-time demo
Abstract
Real-time RGB-D scanning has become widely used to scan 3D objects progressively with a hand-held sensor. Existing online methods restore color information per voxel so that its quality is often limited by the tradeoff between spatial resolution and performance. They often suffer from blur artifacts in the captured texture. Traditional offline texture mapping methods with non-rigid warping assume that the reconstructed geometry and all views are known, and the optimization takes a long time, which prevents them from real-time applications. In this work, we propose a progressive texture-fusion method specifically designed for real-time RGB-D scanning. To this end, we first devise a novel texture-tile voxel grid, where texture tiles are embedded in the voxel grid of the signed distance function, allowing for high-resolution texture mapping on the low-resolution geometry volume. Instead of using expensive mesh parameterization, we associate vertexes of implicit geometry directly with texture coordinates. Second, we introduce real-time texture warping that applies spatially-varying perspective mapping to input images to efficiently mitigate the mismatch between the intermediate geometry and the current input view. It allows us to enhance the quality of texture over time while updating the geometry in real-time. Results demonstrate that the quality of our real-time texture mapping is highly competitive to that of existing offline texture warping methods. Our method is also capable of being integrated into existing RGB-D scanning frameworks.
BibTeX
@InProceedings{Lee_2020_CVPR,
author = {Joo Ho Lee and Hyunho Ha and Yue Dong and Xin Tong and Min H. Kim},
title = {TextureFusion: High-Quality Texture Acquisition for Real-Time RGB-D Scanning},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}
