Differentiable Signed Distance Function Rendering

Delio Vicini     Sébastien Speierer     Wenzel Jakob    
École Polytechnique Fédérale de Lausanne (EPFL)
ACM Transactions on Graphics (Proceedings of SIGGRAPH), 2022

Image-based shape and texture reconstruction of a statue given 32 (synthetic) reference images (a) and known environment illumination. We use differentiable rendering to jointly optimize a signed distance representation of the geometry and albedo texture by minimizing the L 1 loss between the rendered and the reference images. Our method correctly accounts for discontinuities and we therefore do not require ad-hoc object mask or silhouette supervision. We visualize the reconstructed surface (b) and the re-rendered textured object (c). The view and illumination condition in (b) and (c) are different from the ones used during optimization. In (d) we render the ground truth triangle mesh.


Physically-based differentiable rendering has recently emerged as an attractive new technique for solving inverse problems that recover complete 3D scene representations from images. The inversion of shape parameters is of particular interest but also poses severe challenges: shapes are intertwined with visibility, whose discontinuous nature introduces severe bias in computed derivatives unless costly precautions are taken. Shape representations like triangle meshes suffer from additional difficulties, since the continuous optimization of mesh parameters cannot introduce topological changes. One common solution to these difficulties entails representing shapes using signed distance functions (SDFs) and gradually adapting their zero level set during optimization. Previous differentiable rendering of SDFs did not fully account for visibility gradients and required the use of mask or silhouette supervision, or discretization into a triangle mesh. In this article, we show how to extend the commonly used sphere tracing algorithm so that it additionally outputs a reparameterization that provides the means to compute accurate shape parameter derivatives. At a high level, this resembles techniques for differentiable mesh rendering, though we show that the SDF representation admits a particularly efficient reparameterization that outperforms prior work. Our experiments demonstrate the reconstruction of (synthetic) objects without complex regularization or priors, using only a per-pixel RGB loss.



Text Reference

Delio Vicini, Sébastien Speierer, Wenzel Jakob. Differentiable Signed Distance Function Rendering. ACM Transactions on Graphics (Proceedings of SIGGRAPH), 41(4), July 2022.

BibTex Reference

  author  = {Vicini, Delio and Speierer, Sébastien and Jakob, Wenzel},
  title   = {Differentiable Signed Distance Function Rendering},
  journal = {Transactions on Graphics (Proceedings of SIGGRAPH)},
  volume  = {41},
  number  = {4},
  year    = {2022},
  month   = jul,
  pages   = {125:1--125:18},
  doi     = {10.1145/3528223.3530139}