The Regret of Exploration and the Control of Bad Episodes in Reinforcement Learning - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

The Regret of Exploration and the Control of Bad Episodes in Reinforcement Learning

Abstract

The first contribution of this paper is the introduction of a new performance measure of a RL algorithm that is more discriminating than the regret, that we call the regret of exploration that measures the asymptotic cost of exploration. The second contribution is a new performance test (PT) to end episodes in RL optimistic algorithms. This test is based on the performance of the current policy with respect to the best policy over the current confidence set. This is in contrast with all existing RL algorithms whose episode lengths are only based on the number of visits to the states. This modification does not harm the regret and brings an additional property. We show that while all current episodic RL algorithms have a linear regret of exploration, our method has a O(log T) regret of exploration for non-degenerate deterministic MDPs.
Fichier principal
Vignette du fichier
boone23a.pdf (794.13 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04161584 , version 1 (13-07-2023)

Licence

Identifiers

  • HAL Id : hal-04161584 , version 1

Cite

Victor Boone, Bruno Gaujal. The Regret of Exploration and the Control of Bad Episodes in Reinforcement Learning. ICML 2023 - 40th International Conference on Machine Learning, Jul 2023, Hawaii-Honolulu, United States. pp.2824-2856. ⟨hal-04161584⟩
40 View
48 Download

Share

Gmail Mastodon Facebook X LinkedIn More