NeRFshop: Interactive Editing of Neural Radiance Fields - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Proceedings of the ACM on Computer Graphics and Interactive Techniques Year : 2023

NeRFshop: Interactive Editing of Neural Radiance Fields

Abstract

Neural Radiance Fields (NeRFs) have revolutionized novel view synthesis for captured scenes, with recent methods allowing interactive free-viewpoint navigation and fast training for scene reconstruction. However, the implicit representations used by these methods — often including neural networks and complex encodings— make them difficult to edit. Some initial methods have been proposed, but they suffer from limited editing capabilities and/or from a lack of interactivity, and are thus unsuitable for interactive editing of captured scenes. We tackle both limitations and introduce NeRFshop, a novel end-to-end method that allows users to interactively select and deform objects through cage-based transformations. NeRFshop provides fine scribblebased user control for the selection of regions or objects to edit, semi-automatic cage creation, and interactive volumetric manipulation of scene content thanks to our GPU-friendly two-level interpolation scheme. Further, we introduce a preliminary approach that reduces potential resulting artifacts of these transformations with a volumetric membrane interpolation technique inspired by Poisson image editing and provide a process that “distills” the edits into a standalone NeRF representation.
Fichier principal
Vignette du fichier
nerfshop.pdf (5.74 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04027538 , version 1 (13-03-2023)

Identifiers

Cite

Clément Jambon, Bernhard Kerbl, Georgios Kopanas, Stavros Diolatzis, Thomas Leimkühler, et al.. NeRFshop: Interactive Editing of Neural Radiance Fields. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 2023, 6 (1), ⟨10.1145/3585499⟩. ⟨hal-04027538⟩
307 View
729 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More