Study on Automatic 3D Facial Caricaturization: From Rules to Deep Learning - Inria - Institut national de recherche en sciences et technologies du numérique
Article Dans Une Revue Frontiers in Virtual Reality Année : 2022

Study on Automatic 3D Facial Caricaturization: From Rules to Deep Learning

Résumé

Facial caricature is the art of drawing faces in an exaggerated way to convey emotions such as humor or sarcasm. Automatic caricaturization has been explored both in the 2D and 3D domain. In this paper, we propose two novel approaches to automatically caricaturize input facial scans, filling gaps in the literature in terms of user-control, caricature style transfer, and exploring the use of deep learning for 3D mesh caricaturization. The first approach is a gradient-based differential deformation approach with data driven stylization. It is a combination of two deformation processes: facial curvature and proportions exaggeration. The second approach is a GAN for unpaired face-scan-to-3D-caricature translation. We leverage existing facial and caricature datasets, along with recent domain-to-domain translation methods and 3D convolutional operators, to learn to caricaturize 3D facial scans in an unsupervised way. To evaluate and compare these two novel approaches with the state of the art, we conducted the first user study of facial mesh caricaturization techniques, with 49 participants. It highlights the subjectivity of the caricature perception and the complementarity of the methods. Finally, we provide insights for automatically generating caricaturized 3D facial mesh.
Fichier principal
Vignette du fichier
Olivier2022Study.pdf (4.53 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03763591 , version 1 (29-08-2022)

Licence

Identifiants

Citer

Nicolas Olivier, Glenn Kerbiriou, Ferran Argelaguet Sanz, Quentin Avril, Fabien Danieau, et al.. Study on Automatic 3D Facial Caricaturization: From Rules to Deep Learning. Frontiers in Virtual Reality, 2022, 2, pp.1-15. ⟨10.3389/frvir.2021.785104⟩. ⟨hal-03763591⟩
137 Consultations
112 Téléchargements

Altmetric

Partager

More