Sparse Temporal Difference Learning using LASSO - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2007

Sparse Temporal Difference Learning using LASSO

Manuel Loth
  • Function : Author
  • PersonId : 836853
Philippe Preux


We consider the problem of on-line value function estimation in reinforcement learning. We concentrate on the function approximator to use. To try to break the curse of dimensionality, we focus on non parametric function approximators. We propose to fit the use of kernels into the temporal difference algorithms by using regression via the LASSO. We introduce the equi-gradient descent algorithm (EGD) which is a direct adaptation of the one recently introduced in the LARS algorithm family for solving the LASSO. We advocate our choice of the EGD as a judicious algorithm for these tasks. We present the EGD algorithm in details as well as some experimental results. We insist on the qualities of the EGD for reinforcement learning.
Fichier principal
Vignette du fichier
lassoTd.pdf (154.15 Ko) Télécharger le fichier
Origin : Publisher files allowed on an open archive

Dates and versions

inria-00117075 , version 1 (30-11-2006)


  • HAL Id : inria-00117075 , version 1


Manuel Loth, Manuel Davy, Philippe Preux. Sparse Temporal Difference Learning using LASSO. IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning, Apr 2007, Hawaï, USA, United States. ⟨inria-00117075⟩
173 View
931 Download


Gmail Facebook Twitter LinkedIn More