XLS-R fine-tuning on noisy word boundaries for unsupervised speech segmentation into words - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

XLS-R fine-tuning on noisy word boundaries for unsupervised speech segmentation into words

Résumé

Due to the absence of explicit word boundaries in the speech stream, the task of segmenting spoken sentences into word units without text supervision is particularly challenging. In this work, we leverage the most recent selfsupervised speech models that have proved to quickly adapt to new tasks through fine-tuning, even in low resource conditions. Taking inspiration from semi-supervised learning, we finetune an XLS-R model to predict word boundaries themselves produced by top-tier speech segmentation systems: DPDP, VG-HuBERT, GradSeg and DP-Parse. Once XLS-R is finetuned, it is used to infer new word boundary labels that are used in turn for another finetuning step. Our method consistently improves the performance of each system and sets a new state-of-the-art that is, on average 130% higher than the previous one as measured by the F1 score on correctly discovered word tokens on five corpora featuring different languages. Finally, our system can segment speech from languages unseen during fine-tuning in a zero-shot fashion 1 .
Fichier principal
Vignette du fichier
111_xls_r_fine_tuning_on_noisy_wor.pdf (254.56 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Licence : CC BY NC SA - Paternité - Pas d'utilisation commerciale - Partage selon les Conditions Initiales

Dates et versions

hal-04398496 , version 1 (16-01-2024)

Licence

Paternité - Pas d'utilisation commerciale - Partage selon les Conditions Initiales

Identifiants

  • HAL Id : hal-04398496 , version 1

Citer

Robin Algayres, Pablo Diego-Simon, Benoît Sagot, Emmanuel Dupoux. XLS-R fine-tuning on noisy word boundaries for unsupervised speech segmentation into words. EMNLP 2023 - Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Dec 2023, Singapore, Singapore. ⟨hal-04398496⟩
52 Consultations
15 Téléchargements

Partager

Gmail Facebook X LinkedIn More