End-to-end Joint Rich and Normalized ASR with a limited amount of rich training data - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Preprints, Working Papers, ... Year : 2023

End-to-end Joint Rich and Normalized ASR with a limited amount of rich training data

Abstract

Joint rich and normalized automatic speech recognition (ASR), that produces transcriptions both with and without punctuation and capitalization, remains a challenge. End-to-end (E2E) ASR models offer both convenience and the ability to perform such joint transcription of speech. Training such models requires paired speech and rich text data, which is not widely available. In this paper, we compare two different approaches to train a stateless Transducer-based E2E joint rich and normalized ASR system, ready for streaming applications, with a limited amount of rich labeled data. The first approach uses a language model to generate pseudo-rich transcriptions of normalized training data. The second approach uses a single decoder conditioned on the type of the output. The first approach leads to E2E rich ASR which perform better on out-of-domain data, with up to 9% relative reduction in errors. The second approach demonstrates the feasibility of an E2E joint rich and normalized ASR system using as low as 5% rich training data with moderate (2.42% absolute) increase in errors.
Fichier principal
Vignette du fichier
Template.pdf (223.87 Ko) Télécharger le fichier
format.ps (109.2 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04304642 , version 1 (24-11-2023)

Licence

Identifiers

Cite

Can Cui, Imran Ahamad Sheikh, Mostafa Sadeghi, Emmanuel Vincent. End-to-end Joint Rich and Normalized ASR with a limited amount of rich training data. 2023. ⟨hal-04304642⟩
34 View
12 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More