Our general-purpose differentiable transient rendering framework allows to compute derivates of complex, multi-bounce transient sequences with
respect to scene parameters, even in the presence of discontinuous light and sensor functions. The figure shows steady-state and transient renders of a
table-top scene with light coming from the left, then being bounced back by two off-camera diffusers. The bottom row shows the transient light transport
scene derivatives with respect to the index of refraction of the red tea. Please refer to Figure 10 for additional results optimizing the index of refraction of the
tea in the teapot, which in turn changes the speed of light.
Supplemental video
Summary presentation video (5 minutes)
Full presentation video (17 minutes)
Abstract
Recent differentiable rendering techniques have become key tools to tackle
many inverse problems in graphics and vision. Existing models, however,
assume steady-state light transport, i.e., infinite speed of light. While this
is a safe assumption for many applications, recent advances in ultrafast
imaging leverage the wealth of information that can be extracted from the
exact time of flight of light. In this context, physically-based transient rendering
allows to efficiently simulate and analyze light transport considering
that the speed of light is indeed finite. In this paper, we introduce a novel
differentiable transient rendering framework, to help bring the potential
of differentiable approaches into the transient regime. To differentiate the
transient path integral we need to take into account that scattering events
at path vertices are no longer independent; instead, tracking the time of
flight of light requires treating such scattering events at path vertices jointly
as a multidimensional, evolving manifold. We thus turn to the generalized
transport theorem, and introduce a novel correlated importance term, which
links the time-integrated contribution of a path to its light throughput, and
allows us to handle discontinuities in the light and sensor functions. Last,
we present results in several challenging scenarios where the time of flight
of light plays an important role such as optimizing indices of refraction,
non-line-of-sight tracking with nonplanar relay walls, and non-line-of-sight
tracking around two corners.
BibTeX
@Article{ShinyoungYi:SIGA:2021,
author = {Yi, Shinyoung and Kim, Donggun and Choi, Kiseok and
Jarabo, Adrian and Gutierrez, Diego and Kim, Min H.},
title = {Differentiable Transient Rendering},
journal = {ACM Transactions on Graphics (Proc. SIGGRAPH Asia 2021)},
year = {2021},
volume = {40},
number = {6},
}
