Point-Based Neural Rendering with Per-View Optimization - Inria - Institut national de recherche en sciences et technologies du numérique
Journal Articles Computer Graphics Forum Year : 2021

Point-Based Neural Rendering with Per-View Optimization

Abstract

There has recently been great interest in neural rendering methods. Some approaches use 3D geometry reconstructed with Multi-View Stereo (MVS) but cannot recover from the errors of this process, while others directly learn a volumetric neural representation, but suffer from expensive training and inference. We introduce a general approach that is initialized with MVS, but allows further optimization of scene properties in the space of input views, including depth and reprojected features, resulting in improved novel view synthesis. A key element of our approach is a differentiable point-based splatting pipeline, based on our bi-directional Elliptical Weighted Average solution. To further improve quality and efficiency of our point-based method, we introduce a probabilistic depth test and efficient camera selection. We use these elements together in our neural renderer, allowing us to achieve a good compromise between quality and speed. Our pipeline can be applied to multi-view harmonization and stylization in addition to novel view synthesis.
Fichier principal
Vignette du fichier
KopanasPointBasedNeuralRenderingPerViewOptimization.pdf (12.01 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03268140 , version 1 (22-06-2021)
hal-03268140 , version 2 (24-06-2021)
hal-03268140 , version 3 (28-06-2021)

Identifiers

Cite

Georgios Kopanas, Julien Philip, Thomas Leimkühler, George Drettakis. Point-Based Neural Rendering with Per-View Optimization. Computer Graphics Forum, 2021, 40 (4), ⟨10.1111/cgf.14339⟩. ⟨hal-03268140v3⟩
181 View
307 Download

Altmetric

Share

More