Realistic Face Animation for Audiovisual Speech Applications: A Densification Approach Driven by Sparse Stereo Meshes - Inria - Institut national de recherche en sciences et technologies du numérique
Communication Dans Un Congrès Année : 2009

Realistic Face Animation for Audiovisual Speech Applications: A Densification Approach Driven by Sparse Stereo Meshes

Résumé

Being able to produce realistic facial animation is crucial for many speech applications in language learning technologies. Reaching realism needs to acquire and to animate dense 3D models of the face which are often acquired with 3D scanners. However, acquiring the dy- namics of the speech from 3D scans is difficult as the acquisition time generally allows only sustained sounds to be recorded. On the contrary, acquiring the speech dynamics on a sparse set of points is easy using a stereovision recording a talker with markers painted on his/her face. In this paper, we propose an approach to animate a very realistic dense talking head which makes use of a reduced set of 3D dense meshes ac- quired for sustained sounds as well as the speech dynamics learned on a talker painted with white markers. The contributions of the paper are twofold: We first propose an appropriate principal component anal- ysis (PCA) with missing data techniques in order to compute the basic modes of the speech dynamics despite possible unobservable points in the sparse meshes obtained by the stereovision system. We then propose a method for densifying the modes, that is a method for computing the dense modes for spatial animation from the sparse modes learned by the stereovision system. Examples prove the effectiveness of the approach and the high realism obtained with our method.
Fichier principal
Vignette du fichier
mirage2009Final.pdf (259.38 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

inria-00429338 , version 1 (11-05-2010)

Identifiants

Citer

Marie-Odile Berger, Jonathan Ponroy, Brigitte Wrobel-Dautcourt. Realistic Face Animation for Audiovisual Speech Applications: A Densification Approach Driven by Sparse Stereo Meshes. Computer Vision/Computer Graphics CollaborationTechniques 4th International Conference, MIRAGE 2009, INRIA Rocquencourt, May 2009, Rocquencourt, France. pp.297-307, ⟨10.1007/978-3-642-01811-4_27⟩. ⟨inria-00429338⟩
111 Consultations
200 Téléchargements

Altmetric

Partager

More