JAWS: Just A Wild Shot for Cinematic Transfer in Neural Radiance Fields - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

JAWS: Just A Wild Shot for Cinematic Transfer in Neural Radiance Fields

Xi Wang
  • Function : Author
  • PersonId : 1035171
Jinglei Shi
  • Function : Author
  • PersonId : 1236486
Marc Christie

Abstract

This paper presents JAWS, an optimization-driven approach that achieves the robust transfer of visual cinematic features from a reference in-the-wild video clip to a newly generated clip. To this end, we rely on an implicit-neuralrepresentation (INR) in a way to compute a clip that shares the same cinematic features as the reference clip. We propose a general formulation of a camera optimization problem in an INR that computes extrinsic and intrinsic camera parameters as well as timing. By leveraging the differentiability of neural representations, we can back-propagate our designed cinematic losses measured on proxy estimators through a NeRF network to the proposed cinematic parameters directly. We also introduce specific enhancements such as guidance maps to improve the overall quality and efficiency. Results display the capacity of our system to replicate well known camera sequences from movies, adapting the framing, camera parameters and timing of the generated video clip to maximize the similarity with the reference clip.
Fichier principal
Vignette du fichier
main.pdf (81.97 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04046701 , version 1 (26-03-2023)

Licence

Identifiers

Cite

Xi Wang, Robin Courant, Jinglei Shi, Eric Marchand, Marc Christie. JAWS: Just A Wild Shot for Cinematic Transfer in Neural Radiance Fields. CVPR 2023 - IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun 2023, Vancouver, Canada. pp.16933-16942, ⟨10.1109/ICRA48891.2023.10160963⟩. ⟨hal-04046701⟩
98 View
13 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More