Evaluation of hybrid deep learning and optimization method for 3D human pose and shape reconstruction in simulated depth images - Inria - Institut national de recherche en sciences et technologies du numérique
Article Dans Une Revue Computers and Graphics Année : 2023

Evaluation of hybrid deep learning and optimization method for 3D human pose and shape reconstruction in simulated depth images

Résumé

In this paper, we address the problem of capturing both the shape and the pose of a human character using a single depth sensor. Some previous works proposed to fi ta parametric generic human template into the depth image, while others developed deep learning (DL) approaches to fi nd the correspondence between depth pixels and ver- tices of the template. We designed a hybrid approach, combining the advantages of both methods, and conducted extensive experiments on the SURREAL, DFAUST datasets and a subset of AMASS. Results show that this hybrid approach en- ables us to enhance pose and shape estimation compared to using DL or model fi tting separately. We also evaluated the ability of the DL-based dense correspondence method to segment also the background - not only the body parts. We also evaluated 4 di ff er- ent methods to perform the model fi tting based on a dense correspondence, where the number of available 3D points di ff ers from the number of corresponding template ver- tices. These two results enabled us to better understand how to combine DL and model fi tting, and the potential limits of this approach to deal with real depth images. Future works could explore the potential of taking temporal information into account, which has proven to increase the accuracy of pose and shape reconstruction based on a unique depth or RGB image.
Fichier principal
Vignette du fichier
1-s2.0-S0097849323001346-main.pdf (9.22 Mo) Télécharger le fichier
Origine Publication financée par une institution

Dates et versions

hal-04159384 , version 1 (11-07-2023)

Licence

Identifiants

Citer

Xiaofang Wang, Stéphanie Prévost, Adnane Boukhayma, Eric Desjardin, Céline Loscos, et al.. Evaluation of hybrid deep learning and optimization method for 3D human pose and shape reconstruction in simulated depth images. Computers and Graphics, 2023, 115, pp.158-166. ⟨10.1016/j.cag.2023.07.005⟩. ⟨hal-04159384⟩
113 Consultations
134 Téléchargements

Altmetric

Partager

More