Episodic reinforcement learning in finite MDPs: Minimax lower bounds revisited - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Episodic reinforcement learning in finite MDPs: Minimax lower bounds revisited

Omar Darwiche Domingues
  • Fonction : Auteur
  • PersonId : 1105616
Emilie Kaufmann
Michal Valko

Résumé

In this paper, we propose new problem-independent lower bounds on the sample complexity and regret in episodic MDPs, with a particular focus on the non-stationary case in which the transition kernel is allowed to change in each stage of the episode. Our main contribution is a lower bound of Ω((H 3 SA/ε 2) log(1/δ)) on the sample complexity of an (ε, δ)-PAC algorithm for best policy identification in a non-stationary MDP, relying on a construction of "hard MDPs" which is different from the ones previously used in the literature. Using this same class of MDPs, we also provide a rigorous proof of the Ω(√ H 3 SAT) regret bound for non-stationary MDPs. Finally, we discuss connections to PAC-MDP lower bounds.
Fichier principal
Vignette du fichier
domingues2021episodic.pdf (365.69 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03289004 , version 1 (16-07-2021)

Identifiants

  • HAL Id : hal-03289004 , version 1

Citer

Omar Darwiche Domingues, Pierre Ménard, Emilie Kaufmann, Michal Valko. Episodic reinforcement learning in finite MDPs: Minimax lower bounds revisited. Algorithmic Learning Theory, Mar 2021, Paris / Virtual, France. ⟨hal-03289004⟩
84 Consultations
259 Téléchargements

Partager

Gmail Facebook X LinkedIn More