Conference Papers Year : 2023

Another Point of View on Visual Speech Recognition

Abstract

Standard Visual Speech Recognition (VSR) systems directly process images as input features without any apriori link between raw pixel data and facial traits. Pixel information is smartly sieved when facial landmarks are extracted from pictures and repurposed as graph nodes. Their evolution through time is thus modeled by a Graph Convolutional Network. However, with graph-based VSR being in its infancy, the selection of points and their correlation are still ill-defined and often bound to aprioristic knowledge and handcrafted techniques. In this paper, we investigate the graph approach for VSR and its ability to learn the correlation between points beyond the mouth region. We also study the different contributions that each facial region brings to the system accuracy, proving that more scattered but better connected graphs can be both computationally light and accurate.
Fichier principal
Vignette du fichier
pouthier23_interspeech.pdf (858.97 Ko) Télécharger le fichier
Origin Publisher files allowed on an open archive

Dates and versions

hal-04250885 , version 1 (20-10-2023)

Identifiers

Cite

Baptiste Pouthier, Laurent Pilati, Giacomo Valenti, Charles Bouveyron, Frédéric Precioso. Another Point of View on Visual Speech Recognition. 24th INTERSPEECH Conference, Aug 2023, Dublin, Ireland. pp.4089-4093, ⟨10.21437/Interspeech.2023-969⟩. ⟨hal-04250885⟩
132 View
97 Download

Altmetric

Share

More