Article Dans Une Revue Journal of Machine Learning Research Année : 2022

Efficient Change-Point Detection for Tackling Piecewise-Stationary Bandits

Résumé

We introduce GLR-klUCB, a novel algorithm for the piecewise iid non-stationary bandit problem with bounded rewards. This algorithm combines an efficient bandit algorithm, kl-UCB, with an efficient, parameter-free, changepoint detector, the Bernoulli Generalized Likelihood Ratio Test, for which we provide new theoretical guarantees of independent interest. Unlike previous non-stationary bandit algorithms using a change-point detector, GLR-klUCB does not need to be calibrated based on prior knowledge on the arms' means. We prove that this algorithm can attain a $O(\sqrt{TA \Upsilon_T\log(T)})$ regret in $T$ rounds on some ``easy'' instances, where A is the number of arms and $\Upsilon_T$ the number of change-points, without prior knowledge of $\Upsilon_T$. In contrast with recently proposed algorithms that are agnostic to $\Upsilon_T$, we perform a numerical study showing that GLR-klUCB is also very efficient in practice, beyond easy instances.

Domaines

Fichier principal
Vignette du fichier
BKMS22 (1).pdf (668.02 Ko) Télécharger le fichier
Origine Fichiers éditeurs autorisés sur une archive ouverte
Licence

Dates et versions

hal-02006471 , version 1 (04-02-2019)
hal-02006471 , version 2 (08-12-2020)
hal-02006471 , version 3 (01-08-2022)

Licence

Identifiants

Citer

Lilian Besson, Emilie Kaufmann, Odalric-Ambrym Maillard, Julien Seznec. Efficient Change-Point Detection for Tackling Piecewise-Stationary Bandits. Journal of Machine Learning Research, 2022. ⟨hal-02006471v3⟩
1244 Consultations
1696 Téléchargements

Altmetric

Partager

  • More