A kernel-based approach to non-stationary reinforcement learning in metric spaces - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

A kernel-based approach to non-stationary reinforcement learning in metric spaces

Omar D Domingues
  • Function : Author
Emilie Kaufmann
Michal Valko

Abstract

In this work, we propose KeRNS: an algorithm for episodic reinforcement learning in nonstationary Markov Decision Processes (MDPs) whose state-action set is endowed with a metric. Using a non-parametric model of the MDP built with time-dependent kernels, we prove a regret bound that scales with the covering dimension of the state-action space and the total variation of the MDP with time, which quantifies its level of non-stationarity. Our method generalizes previous approaches based on sliding windows and exponential discounting used to handle changing environments. We further propose a practical implementation of KeRNS, we analyze its regret and validate it experimentally.
Fichier principal
Vignette du fichier
domingues2021kernel-based-non-stationary.pdf (12.85 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03289026 , version 1 (16-07-2021)

Identifiers

  • HAL Id : hal-03289026 , version 1

Cite

Omar D Domingues, Pierre Ménard, Matteo Pirotta, Emilie Kaufmann, Michal Valko. A kernel-based approach to non-stationary reinforcement learning in metric spaces. International Conference on Artificial Intelligence and Statistics, Apr 2021, San Diego / Virtual, United States. ⟨hal-03289026⟩
96 View
80 Download

Share

Gmail Facebook X LinkedIn More