Reinforcement Learning of POMDPs using Spectral Methods - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2016

Reinforcement Learning of POMDPs using Spectral Methods

Résumé

We propose a new reinforcement learning algorithm for partially observable Markov decision processes (POMDP) based on spectral decomposition methods. While spectral methods have been previously employed for consistent learning of (passive) latent variable models such as hidden Markov models, POMDPs are more challenging since the learner interacts with the environment and possibly changes the future observations in the process. We devise a learning algorithm running through episodes, in each episode we employ spectral techniques to learn the POMDP parameters from a trajectory generated by a fixed policy. At the end of the episode, an optimization oracle returns the optimal memoryless planning policy which maximizes the expected reward based on the estimated POMDP model. We prove an order-optimal regret bound w.r.t. the optimal memoryless policy and efficient scaling with respect to the dimensionality of observation and action spaces.
Fichier principal
Vignette du fichier
master.pdf (1001.49 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01322207 , version 1 (26-05-2016)

Identifiants

  • HAL Id : hal-01322207 , version 1

Citer

Kamyar Azizzadenesheli, Alessandro Lazaric, Animashree Anandkumar. Reinforcement Learning of POMDPs using Spectral Methods. Proceedings of the 29th Annual Conference on Learning Theory (COLT2016), Jun 2016, New York City, United States. ⟨hal-01322207⟩
189 Consultations
138 Téléchargements

Partager

Gmail Facebook X LinkedIn More