Hierarchical-Task Reservoir for Online Semantic Analysis from Continuous Speech - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles IEEE Transactions on Neural Networks and Learning Systems Year : 2021

Hierarchical-Task Reservoir for Online Semantic Analysis from Continuous Speech

Luca Pedrelli
  • Function : Author
Xavier Hinaut

Abstract

In this paper, we propose a novel architecture called Hierarchical-Task Reservoir (HTR) suitable for real-time applications for which different levels of abstraction are available. We apply it to semantic role labeling based on continuous speech recognition. Taking inspiration from the brain, that demonstrates hierarchies of representations from perceptive to integrative areas, we consider a hierarchy of four sub-tasks with increasing levels of abstraction (phone, word, part-of-speech and semantic role tags). These tasks are progressively learned by the layers of the HTR architecture. Interestingly, quantitative and qualitative results show that the hierarchical-task approach provides an advantage to improve the prediction. In particular, the qualitative results show that a shallow or a hierarchical reservoir, considered as baselines, do not produce estimations as good as the HTR model would. Moreover, we show that it is possible to further improve the accuracy of the model by designing skip connections and by considering word embedding in the internal representations. Overall, the HTR outperformed the other stateof-the-art reservoir-based approaches and it resulted in extremely efficient w.r.t. typical RNNs in deep learning (e.g. LSTMs). The HTR architecture is proposed as a step toward the modeling of online and hierarchical processes at work in the brain during language comprehension.
Fichier principal
Vignette du fichier
PedrelliHinaut2020_preprint_HAL-v3.pdf (1.24 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03031413 , version 1 (30-11-2020)
hal-03031413 , version 2 (20-04-2021)
hal-03031413 , version 3 (19-01-2022)

Identifiers

Cite

Luca Pedrelli, Xavier Hinaut. Hierarchical-Task Reservoir for Online Semantic Analysis from Continuous Speech. IEEE Transactions on Neural Networks and Learning Systems, 2021, ⟨10.1109/TNNLS.2021.3095140⟩. ⟨hal-03031413v3⟩
167 View
338 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More