End-to-end Joint Rich and Normalized ASR with a limited amount of rich training data - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

End-to-end Joint Rich and Normalized ASR with a limited amount of rich training data

Résumé

Joint rich and normalized automatic speech recognition (ASR), that produces transcriptions both with and without punctuation and capitalization, remains a challenge. End-to-end (E2E) ASR models offer both convenience and the ability to perform such joint transcription of speech. Training such models requires paired speech and rich text data, which is not widely available. In this paper, we compare two different approaches to train a stateless Transducer-based E2E joint rich and normalized ASR system, ready for streaming applications, with a limited amount of rich labeled data. The first approach uses a language model to generate pseudo-rich transcriptions of normalized training data. The second approach uses a single decoder conditioned on the type of the output. The first approach leads to E2E rich ASR which perform better on out-of-domain data, with up to 9% relative reduction in errors. The second approach demonstrates the feasibility of an E2E joint rich and normalized ASR system using as low as 5% rich training data with moderate (2.42% absolute) increase in errors.
Fichier principal
Vignette du fichier
Template.pdf (223.87 Ko) Télécharger le fichier
format.ps (109.2 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04304642 , version 1 (24-11-2023)

Licence

Paternité

Identifiants

Citer

Can Cui, Imran Ahamad Sheikh, Mostafa Sadeghi, Emmanuel Vincent. End-to-end Joint Rich and Normalized ASR with a limited amount of rich training data. 2023. ⟨hal-04304642⟩
26 Consultations
7 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More