Online Learning with Noise: A Kernel-Based Policy-Gradient Approach - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2010

Online Learning with Noise: A Kernel-Based Policy-Gradient Approach

Alain Dutech

Abstract

Various forms of noise are present in the brain. The role of noise in a exploration/exploitation trade-off is cast into the framework of reinforcement learning for a complex task of motor learning. A neuro-controler using a linear transformation of the input to which is added a gaussian noise is modelized as a stochastic controler that can be learned online in ''direct policy-gradient'' scheme. The reward signal is related to sensor information, thus no direct or indirect model of the system to control is needed. The task chosen (reaching with a multi-joint arm) is redundant and non-linear. The controler inputs are then projected to a feature space of higher dimension using a topographic coding based on gaussian kernels. We show that through a consistent noise level it possible to explore the environnment so as to find good control solution that can be exploited. Besides, the controler is able to adapt continuously to changes in the system dynamics. The general framework of this work will allow to study various noises and their effect, especially since it is quite compatible with more complexe types of stochastic neuro-controler, as demonstrated by other works on binary or spiking networks.
Fichier principal
Vignette du fichier
dauce_noisePolicyGradient_NeuroComp10.pdf (186.9 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

inria-00517006 , version 1 (13-09-2010)

Identifiers

  • HAL Id : inria-00517006 , version 1

Cite

Emmanuel Daucé, Alain Dutech. Online Learning with Noise: A Kernel-Based Policy-Gradient Approach. Conférence Française de Neurosciences Computationnelles - NeuroComp 2010, Oct 2010, Lyon, France. ⟨inria-00517006⟩
187 View
80 Download

Share

Gmail Facebook X LinkedIn More