Joint 3D Shape and Motion Estimation from Rolling Shutter Light-Field Images - Inria - Institut national de recherche en sciences et technologies du numérique
Communication Dans Un Congrès Année : 2023

Joint 3D Shape and Motion Estimation from Rolling Shutter Light-Field Images

Résumé

In this paper, we propose an approach to address the problem of 3D reconstruction of scenes from a single image captured by a light-field camera equipped with a rolling shutter sensor. Our method leverages the 3D information cues present in the light-field and the motion information provided by the rolling shutter effect. We present a generic model for the imaging process of this sensor and a two-stage algorithm that minimizes the re-projection error while considering the position and motion of the camera in a motion-shape bundle adjustment estimation strategy. Thereby, we provide an instantaneous 3D shape-and-pose-and-velocity sensing paradigm. To the best of our knowledge, this is the first study to leverage this type of sensor for this purpose. We also present a new benchmark dataset composed of different light-fields showing rolling shutter effects, which can be used as a common base to improve the evaluation and tracking the progress in the field. We demonstrate the effectiveness and advantages of our approach through several experiments conducted for different scenes and types of motions. The source code and dataset are publicly available at: https://github.com/ICB-Vision-AI/RSLF

Dates et versions

hal-04269428 , version 1 (03-11-2023)

Licence

Identifiants

Citer

Hermes Mcgriff, Renato Martins, Nicolas Andreff, Cédric Demonceaux. Joint 3D Shape and Motion Estimation from Rolling Shutter Light-Field Images. IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Jan 2024, Hawaii, United States. ⟨hal-04269428⟩
171 Consultations
0 Téléchargements

Altmetric

Partager

More