DNN-based semantic rescoring models for speech recognition - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

DNN-based semantic rescoring models for speech recognition


In this work, we address the problem of improving an automatic speech recognition (ASR) system. We want to efficiently model long-term semantic relations between words and introduce this information through a semantic model. We propose neural network (NN) semantic models for rescoring the N-best hypothesis list. These models use two types of representations as part of DNN input features: static word embeddings (from word2vec) and dynamic contextual embeddings (from BERT). Semantic information is computed thanks to these representations and used in the hypothesis pair comparison mode. We perform experiments on the publicly available dataset TED-LIUM. Clean speech and speech mixed with real noise are experimented, according to our industrial project context. The proposed BERT-based rescoring approach gives a significant improvement of the word error rate (WER) over the ASR system without rescoring semantic models under all experimented conditions and with n-gram and recurrent NN language model (Long Short-Term model, LSTM).
Fichier principal
Vignette du fichier
TSD2021_W2vec_BERT_sem_v12_camera_ready.pdf (643.14 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03239211 , version 1 (27-05-2021)


  • HAL Id : hal-03239211 , version 1


Irina Illina, Dominique Fohr. DNN-based semantic rescoring models for speech recognition. TSD 2021 - 24th International Conference on Text, Speech and Dialogue, Sep 2021, Olomouc, Czech Republic. ⟨hal-03239211⟩
82 View
188 Download


Gmail Mastodon Facebook X LinkedIn More