Self-Attentional Credit Assignment for Transfer in Reinforcement Learning - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year :

Self-Attentional Credit Assignment for Transfer in Reinforcement Learning

Johan Ferret
  • Function : Author
  • PersonId : 1092607
Matthieu Geist
Olivier Pietquin

Abstract

The ability to transfer knowledge to novel environments and tasks is a sensible desiderata for general learning agents. Despite the apparent promises, transfer in RL is still an open and little exploited research area. In this paper, we take a brand-new perspective about transfer: we suggest that the ability to assign credit unveils structural invariants in the tasks that can be transferred to make RL more sample efficient. Our main contribution is Secret, a novel approach to transfer learning for RL that uses a backward-view credit assignment mechanism based on a self-attentive architecture. Two aspects are key to its generality: it learns to assign credit as a separate offline supervised process and exclusively modifies the reward function. Consequently, it can be supplemented by transfer methods that do not modify the reward function and it can be plugged on top of any RL algorithm.
Fichier principal
Vignette du fichier
Credit_Alignment_HAL (1).pdf (6.03 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03159832 , version 1 (09-03-2021)

Identifiers

  • HAL Id : hal-03159832 , version 1

Cite

Johan Ferret, Raphaël Marinier, Matthieu Geist, Olivier Pietquin. Self-Attentional Credit Assignment for Transfer in Reinforcement Learning. IJCAI 2020 - 29th International Joint Conference on Artificial Intelligence, Jul 2020, Yokohama / Virtual, Japan. ⟨hal-03159832⟩
55 View
99 Download

Share

Gmail Facebook Twitter LinkedIn More