Volume Sweeping: Learning Photoconsistency for Multi-View Shape Reconstruction - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles International Journal of Computer Vision Year : 2021

Volume Sweeping: Learning Photoconsistency for Multi-View Shape Reconstruction

Abstract

We propose a full study and methodology for multi-view stereo reconstruction with performance capture data. Multi-view 3D reconstruction has largely been studied with general, high resolution and high texture content inputs, where classic low-level feature extraction and matching are generally successful. However in performance capture scenarios, texture content is limited by wider angle shots resulting in smaller subject projection areas, and intrinsically low image content of casual clothing. We present a dedicated pipeline, based on a per-camera depth map sweeping strategy, analyzing in particular how recent deep network advances allow to replace classic multi-view photoconsistency functions with one that is learned. We show that learning based on a volumetric receptive field around a 3D depth candidate improves over using per-view 2D windows, giving the photoconsistency inference more visibility over local 3D correlations in viewpoint color aggrega-tion. Despite being trained on a standard dataset of scanned static objects, the proposed method is shown to generalize and significantly outperform existing approaches on performance capture data, while achieving competitive results on recent benchmarks.
Fichier principal
Vignette du fichier
IJCV2019submission.pdf (14.74 Mo) Télécharger le fichier
Vignette du fichier
Teaser.png (687.45 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Format : Figure, Image

Dates and versions

hal-02975479 , version 1 (22-10-2020)

Identifiers

Cite

Vincent Leroy, Jean-Sébastien Franco, Edmond Boyer. Volume Sweeping: Learning Photoconsistency for Multi-View Shape Reconstruction. International Journal of Computer Vision, 2021, 129, pp.284-299. ⟨10.1007/s11263-020-01377-0⟩. ⟨hal-02975479⟩
774 View
309 Download

Altmetric

Share

Gmail Facebook X LinkedIn More