Knowledge extraction from the learning of sequences in a long short term memory (LSTM) architecture - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Article Dans Une Revue Knowledge-Based Systems Année : 2021

Knowledge extraction from the learning of sequences in a long short term memory (LSTM) architecture

Ikram Chraibi Kaadoud
Nicolas P. Rougier
Frédéric Alexandre

Résumé

Transparency and trust in machine learning algorithms have been deemed to be fundamental and yet, from a practical point of view, they remain difficult to implement. Particularly, explainability and interpretability are certainly among the most difficult capabilities to be addressed and imply to be able to understand a decision in terms of simple cues and rules. In this article, we address this specific problem in the context of sequence learning by recurrent neuronal models (and more specifically Long Short Term Memory model). We introduce a general method to extract knowledge from the latent space based on the clustering of the internal states. From these hidden states, we explain how to build and validate an automaton that corresponds to the underlying (unknown) grammar, and allows to predict if a given sequence is valid or not. Finally, we show that it is possible for such complex recurrent model, to extract the knowledge that is implicitly encoded in the sequences and we report a high rate of recognition of the sequences extracted from the original grammar. This method is illustrated on artificial grammars (Reber grammar variants) as well as on a real use-case in the electrical domain, whose underlying grammar is unknown.
Fichier principal
Vignette du fichier
Knowledge_Based_systems (4).pdf (2.68 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03437920 , version 1 (20-11-2021)

Identifiants

Citer

Ikram Chraibi Kaadoud, Nicolas P. Rougier, Frédéric Alexandre. Knowledge extraction from the learning of sequences in a long short term memory (LSTM) architecture. Knowledge-Based Systems, 2021, 235, pp.18. ⟨10.1016/j.knosys.2021.107657⟩. ⟨hal-03437920⟩
76 Consultations
174 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More