The Regret of Exploration and the Control of Bad Episodes in Reinforcement Learning - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

The Regret of Exploration and the Control of Bad Episodes in Reinforcement Learning

Résumé

The first contribution of this paper is the introduction of a new performance measure of a RL algorithm that is more discriminating than the regret, that we call the regret of exploration that measures the asymptotic cost of exploration. The second contribution is a new performance test (PT) to end episodes in RL optimistic algorithms. This test is based on the performance of the current policy with respect to the best policy over the current confidence set. This is in contrast with all existing RL algorithms whose episode lengths are only based on the number of visits to the states. This modification does not harm the regret and brings an additional property. We show that while all current episodic RL algorithms have a linear regret of exploration, our method has a O(log T) regret of exploration for non-degenerate deterministic MDPs.
Fichier principal
Vignette du fichier
boone23a.pdf (794.13 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04161584 , version 1 (13-07-2023)

Licence

Paternité

Identifiants

  • HAL Id : hal-04161584 , version 1

Citer

Victor Boone, Bruno Gaujal. The Regret of Exploration and the Control of Bad Episodes in Reinforcement Learning. ICML 2023 - 40th International Conference on Machine Learning, Jul 2023, Hawaii-Honolulu, United States. pp.2824-2856. ⟨hal-04161584⟩
39 Consultations
44 Téléchargements

Partager

Gmail Facebook X LinkedIn More