Towards the prediction of the vocal tract shape from the sequence of phonemes to be articulated - Inria - Institut national de recherche en sciences et technologies du numérique
Communication Dans Un Congrès Année : 2021

Towards the prediction of the vocal tract shape from the sequence of phonemes to be articulated

Résumé

In this work, we address the prediction of speech articulators' temporal geometric position from the sequence of phonemes to be articulated. We start from a set of real-time MRI sequences uttered by a female French speaker. The contours of five articulators were tracked automatically in each of the frames in the MRI video. Then, we explore the capacity of a bidirectional GRU to correctly predict each articulator's shape and position given the sequence of phonemes and their duration. We propose a 5-fold cross-validation experiment to evaluate the generalization capacity of the model. In a second experiment, we evaluate our model's data efficiency by reducing training data. We evaluate the point-to-point Euclidean distance and the Pearson's correlations along time between the predicted and the target shapes. We also evaluate produced shapes of the critical articulators of specific phonemes. We show that our model can achieve good results with minimal data, producing very realistic vocal tract shapes.
Fichier principal
Vignette du fichier
INTERSPEECH_2021_Phoneme_to_Articulatory.pdf (449.35 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03360113 , version 1 (30-09-2021)

Identifiants

Citer

Vinicius Ribeiro, Karyna Isaieva, Justine Leclère, Pierre-André Vuissoz, Yves Laprie. Towards the prediction of the vocal tract shape from the sequence of phonemes to be articulated. INTERSPEECH 2021, Aug 2021, Brno, Czech Republic. ⟨10.21437/Interspeech.2021-184⟩. ⟨hal-03360113⟩
191 Consultations
318 Téléchargements

Altmetric

Partager

More