Evaluation of hybrid deep learning and optimization method for 3D human pose and shape reconstruction in simulated depth images
Résumé
In this paper, we address the problem of capturing both the shape and the pose of a
human character using a single depth sensor. Some previous works proposed to fi ta
parametric generic human template into the depth image, while others developed deep
learning (DL) approaches to fi nd the correspondence between depth pixels and ver-
tices of the template. We designed a hybrid approach, combining the advantages of
both methods, and conducted extensive experiments on the SURREAL, DFAUST
datasets and a subset of AMASS. Results show that this hybrid approach en-
ables us to enhance pose and shape estimation compared to using DL or model fi tting
separately. We also evaluated the ability of the DL-based dense correspondence method
to segment also the background - not only the body parts. We also evaluated 4 di ff er-
ent methods to perform the model fi tting based on a dense correspondence, where the
number of available 3D points di ff ers from the number of corresponding template ver-
tices. These two results enabled us to better understand how to combine DL and model
fi tting, and the potential limits of this approach to deal with real depth images. Future
works could explore the potential of taking temporal information into account, which
has proven to increase the accuracy of pose and shape reconstruction based on a unique
depth or RGB image.
Origine | Publication financée par une institution |
---|