S-TRIGGER: Continual State Representation Learning via Self-Triggered Generative Replay - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

S-TRIGGER: Continual State Representation Learning via Self-Triggered Generative Replay

Abstract

We consider the problem of building a state repre-sentation model for control, in a continual learning setting. As the environment changes, the aim is to efficiently compress the sensory state information without losing past knowledge, and then use Reinforcement Learning on the resulting features for efficient policy learning. To this end, we propose S-TRIGGER, a general method for Continual State Representation Learning applicable to Variational Auto-Encoders and its many variants. The method is based on Generative Replay, i.e. the use of generated samples to maintain past knowledge. It comes along with a statistically sound method for environment change detection, which self-triggers the Generative Replay. Our experiments on VAEs show that S-TRIGGER learns state representations that allows fast and high-performing Reinforcement Learning, while avoiding catastrophic forgetting. The resulting system has a bounded size and is capable of autonomously learning new information without using past data.

Dates and versions

hal-03377783 , version 1 (14-10-2021)

Identifiers

Cite

Hugo Caselles-Dupré, Michael Garcia-Ortiz, David Filliat. S-TRIGGER: Continual State Representation Learning via Self-Triggered Generative Replay. IJCNN 2021 - International Joint Conference on Neural Networks, Jul 2021, Shenzhen / Virtual, China. pp.1-7, ⟨10.1109/IJCNN52387.2021.9533683⟩. ⟨hal-03377783⟩
61 View
0 Download

Altmetric

Share

Gmail Facebook X LinkedIn More