Momentum in Reinforcement Learning - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

Momentum in Reinforcement Learning

Olivier Pietquin
Matthieu Geist

Résumé

We adapt the optimization's concept of momentum to reinforcement learning. Seeing the state-action value functions as an analog to the gradients in optimization, we interpret momentum as an average of consecutive q-functions. We derive Momentum Value Iteration (MoVI), a variation of Value iteration that incorporates this momentum idea. Our analysis shows that this allows MoVI to average errors over successive iterations. We show that the proposed approach can be readily extended to deep learning. Specifically,we propose a simple improvement on DQN based on MoVI, and experiment it on Atari games.
Fichier principal
Vignette du fichier
vieillard20a-supp.pdf (5.38 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03137343 , version 1 (10-02-2021)

Identifiants

  • HAL Id : hal-03137343 , version 1

Citer

Nino Vieillard, Bruno Scherrer, Olivier Pietquin, Matthieu Geist. Momentum in Reinforcement Learning. AISTATS 2020 - 23rd International Conference on Artificial Intelligence and Statistics, Aug 2020, Palermo / Virtual, Italy. ⟨hal-03137343⟩
54 Consultations
94 Téléchargements

Partager

Gmail Facebook X LinkedIn More