Non-Markovian Reinforcement Learning for Reactive Grid scheduling - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2011

Non-Markovian Reinforcement Learning for Reactive Grid scheduling


Two recurrent questions often appear when solving numerous real world policy search problems. First, the variables defining the so called Markov Decision Process are often continuous, that leads to the necessity for discretization of the considered state/action space or the use of a regression model, often non-linear, to approach the Q-function nee- ded in the reinforcement learning paradigm. Second, the markovian hypothesis is made which is often strongly discutable and can lead to unacceptably suboptimal resulting policies. In this paper, the job scheduling problem in grid infrastructure is modeled as a continuous action-state space, multi-objective reinforcement learning problem, under realistic assumptions ; the high level goals of users, administrators, and shareholders are captured through simple utility functions. So, formalizing the problem as a par- tially observable Markov decision process (POMDP), we detail the algorithm of fitted Q-function learning using an Echo State Network. The experiment, conducted on simu- lation of real grid activity will demonstrate the significative gain of the method against native scheduling infrastructure and a classic feed forward back-propagated neural net- work (FFNN) for Q function learning in the most difficult cases.
Fichier principal
Vignette du fichier
esn-cap11.pdf (360.42 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

inria-00586504 , version 1 (16-04-2011)


  • HAL Id : inria-00586504 , version 1


Julien Perez, Balázs Kégl, Cecile Germain-Renaud. Non-Markovian Reinforcement Learning for Reactive Grid scheduling. Conférence Francophone d'Apprentissage, May 2011, Chambéry, France. ⟨inria-00586504⟩
309 View
136 Download


Gmail Facebook X LinkedIn More