Efficient Change-Point Detection for Tackling Piecewise-Stationary Bandits - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Journal of Machine Learning Research Year : 2022

Efficient Change-Point Detection for Tackling Piecewise-Stationary Bandits

Lilian Besson
Emilie Kaufmann
Odalric-Ambrym Maillard
Julien Seznec
  • Function : Author
  • PersonId : 1084851

Abstract

We introduce GLR-klUCB, a novel algorithm for the piecewise iid non-stationary bandit problem with bounded rewards. This algorithm combines an efficient bandit algorithm, kl-UCB, with an efficient, parameter-free, changepoint detector, the Bernoulli Generalized Likelihood Ratio Test, for which we provide new theoretical guarantees of independent interest. Unlike previous non-stationary bandit algorithms using a change-point detector, GLR-klUCB does not need to be calibrated based on prior knowledge on the arms' means. We prove that this algorithm can attain a $O(\sqrt{TA \Upsilon_T\log(T)})$ regret in $T$ rounds on some ``easy'' instances, where A is the number of arms and $\Upsilon_T$ the number of change-points, without prior knowledge of $\Upsilon_T$. In contrast with recently proposed algorithms that are agnostic to $\Upsilon_T$, we perform a numerical study showing that GLR-klUCB is also very efficient in practice, beyond easy instances.
Fichier principal
Vignette du fichier
BKMS22 (1).pdf (668.02 Ko) Télécharger le fichier
Origin Publisher files allowed on an open archive

Dates and versions

hal-02006471 , version 1 (04-02-2019)
hal-02006471 , version 2 (08-12-2020)
hal-02006471 , version 3 (01-08-2022)

Licence

Identifiers

Cite

Lilian Besson, Emilie Kaufmann, Odalric-Ambrym Maillard, Julien Seznec. Efficient Change-Point Detection for Tackling Piecewise-Stationary Bandits. Journal of Machine Learning Research, 2022. ⟨hal-02006471v3⟩
642 View
952 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More