Differentiable Rendering with Perturbed Optimizers - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

Differentiable Rendering with Perturbed Optimizers

Abstract

Reasoning about 3D scenes from their 2D image projections is one of the core problems in computer vision. Solutions to this inverse and ill-posed problem typically involve a search for models that best explain observed image data. Notably, images depend both on the properties of observed scenes and on the process of image formation. Hence, if optimization techniques should be used to explain images, it is crucial to design differentiable functions for the projection of 3D scenes into images, also known as differentiable rendering. Previous approaches to differentiable rendering typically replace non-differentiable operations by smooth approximations, impacting the subsequent 3D estimation. In this paper, we take a more general approach and study differentiable renderers through the prism of randomized optimization and the related notion of perturbed optimizers. In particular, our work highlights the link between some well-known differentiable renderer formulations and randomly smoothed optimizers, and introduces differentiable perturbed renderers. We also propose a variance reduction mechanism to alleviate the computational burden inherent to perturbed optimizers and introduce an adaptive scheme to automatically adjust the smoothing parameters of the rendering process. We apply our method to 3D scene reconstruction and demonstrate its advantages on the tasks of 6D pose estimation and 3D mesh reconstruction. By providing informative gradients that can be used as a strong supervisory signal, we demonstrate the benefits of perturbed renderers to obtain more accurate solutions when compared to the state-of-the-art alternatives using smooth gradient approximations.
Fichier principal
Vignette du fichier
lelidec2021diffrender.pdf (3.89 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03378451 , version 1 (14-10-2021)

Identifiers

  • HAL Id : hal-03378451 , version 1

Cite

Quentin Le Lidec, Ivan Laptev, Cordelia Schmid, Justin Carpentier. Differentiable Rendering with Perturbed Optimizers. NeurIPS 2021 - Thirty-fifth Conference on Neural Information Processing Systems, Dec 2021, Sydney / Virtual, Australia. ⟨hal-03378451⟩
111 View
118 Download

Share

Gmail Facebook X LinkedIn More