UnderPressure: Deep Learning for Foot Contact Detection, Ground Reaction Force Estimation and Footskate Cleanup - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Computer Graphics Forum Year : 2022

UnderPressure: Deep Learning for Foot Contact Detection, Ground Reaction Force Estimation and Footskate Cleanup

Lucas Mourot
Ludovic Hoyet
François Le Clerc
Pierre Hellier

Abstract

Human motion synthesis and editing are essential to many applications like video games, virtual reality, and film post- production. However, they often introduce artefacts in motion capture data, which can be detrimental to the perceived realism. In particular, footskating is a frequent and disturbing artefact, which requires knowledge of foot contacts to be cleaned up. Current approaches to obtain foot contact labels rely either on unreliable threshold-based heuristics or on tedious manual annotation. In this article, we address automatic foot contact label detection from motion capture data with a deep learning based method. To this end, we first publicly release UNDERPRESSURE, a novel motion capture database labelled with pressure insoles data serving as reliable knowledge of foot contact with the ground. Then, we design and train a deep neural network to estimate ground reaction forces exerted on the feet from motion data and then derive accurate foot contact labels. The evaluation of our model shows that we significantly outperform heuristic approaches based on height and velocity thresholds and that our approach is much more robust when applied on motion sequences suffering from perturbations like noise or footskate. We further propose a fully automatic workflow for footskate cleanup: foot contact labels are first derived from estimated ground reaction forces. Then, footskate is removed by solving foot constraints through an optimisation-based inverse kinematics (IK) approach that ensures consistency with the estimated ground reaction forces. Beyond footskate cleanup, both the database and the method we propose could help to improve many approaches based on foot contact labels or ground reaction forces, including inverse dynamics problems like motion reconstruction and learning of deep motion models in motion synthesis or character animation. Our implementation, pre-trained model as well as links to database can be found at github.com/InterDigitalInc/UnderPressure.
Fichier principal
Vignette du fichier
SCA_2022_UnderPressure.pdf (2.62 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03865772 , version 1 (22-11-2022)

Identifiers

Cite

Lucas Mourot, Ludovic Hoyet, François Le Clerc, Pierre Hellier. UnderPressure: Deep Learning for Foot Contact Detection, Ground Reaction Force Estimation and Footskate Cleanup. Computer Graphics Forum, 2022, ⟨10.1111/cgf.14635⟩. ⟨hal-03865772⟩
95 View
83 Download

Altmetric

Share

Gmail Facebook X LinkedIn More