A Unified View of TD Algorithms; Introducing Full-Gradient TD and Equi-Gradient Descent TD - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2007

A Unified View of TD Algorithms; Introducing Full-Gradient TD and Equi-Gradient Descent TD

Résumé

This paper addresses the issue of policy evaluation in Markov Decision Processes, using linear function approximation. It provides a unified view of algorithms such as TD(lambda), LSTD(lambda), iLSTD, residual-gradient TD. It is asserted that they all consist in minimizing a gradient function and differ by the form of this function and their means of minimizing it. Two new schemes are introduced in that framework: Full-gradient TD which uses a generalization of the principle introduced in iLSTD, and EGD TD, which reduces the gradient by successive equi-gradient descents. These three algorithms form a new intermediate family with the interesting property of making much better use of the samples than TD while keeping a gradient descent scheme, which is useful for complexity issues and optimistic policy iteration.
Fichier principal
Vignette du fichier
unified.pdf (136.29 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

inria-00116936 , version 1 (28-11-2006)
inria-00116936 , version 2 (29-11-2006)

Identifiants

Citer

Manuel Loth, Philippe Preux, Manuel Davy. A Unified View of TD Algorithms; Introducing Full-Gradient TD and Equi-Gradient Descent TD. European Symposium on Artificial Neural Networks, Apr 2007, Bruges, Belgium, Belgium. ⟨inria-00116936v2⟩
255 Consultations
337 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More