Study on Automatic 3D Facial Caricaturization: From Rules to Deep Learning - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Frontiers in Virtual Reality Year : 2022

Study on Automatic 3D Facial Caricaturization: From Rules to Deep Learning

Abstract

Facial caricature is the art of drawing faces in an exaggerated way to convey emotions such as humor or sarcasm. Automatic caricaturization has been explored both in the 2D and 3D domain. In this paper, we propose two novel approaches to automatically caricaturize input facial scans, filling gaps in the literature in terms of user-control, caricature style transfer, and exploring the use of deep learning for 3D mesh caricaturization. The first approach is a gradient-based differential deformation approach with data driven stylization. It is a combination of two deformation processes: facial curvature and proportions exaggeration. The second approach is a GAN for unpaired face-scan-to-3D-caricature translation. We leverage existing facial and caricature datasets, along with recent domain-to-domain translation methods and 3D convolutional operators, to learn to caricaturize 3D facial scans in an unsupervised way. To evaluate and compare these two novel approaches with the state of the art, we conducted the first user study of facial mesh caricaturization techniques, with 49 participants. It highlights the subjectivity of the caricature perception and the complementarity of the methods. Finally, we provide insights for automatically generating caricaturized 3D facial mesh.
Fichier principal
Vignette du fichier
Olivier2022Study.pdf (4.53 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03763591 , version 1 (29-08-2022)

Licence

Identifiers

Cite

Nicolas Olivier, Glenn Kerbiriou, Ferran Argelaguet Sanz, Quentin Avril, Fabien Danieau, et al.. Study on Automatic 3D Facial Caricaturization: From Rules to Deep Learning. Frontiers in Virtual Reality, 2022, 2, pp.1-15. ⟨10.3389/frvir.2021.785104⟩. ⟨hal-03763591⟩
115 View
77 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More