From Optimality to Robustness: Dirichlet Sampling Strategies in Stochastic Bandits - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

From Optimality to Robustness: Dirichlet Sampling Strategies in Stochastic Bandits

Abstract

The stochastic multi-arm bandit problem has been extensively studied under standard assumptions on the arm's distribution (e.g bounded with known support, exponential family, etc). These assumptions are suitable for many real-world problems but sometimes they require knowledge (on tails for instance) that may not be precisely accessible to the practitioner, raising the question of the robustness of bandit algorithms to model misspecification. In this paper we study a generic Dirichlet Sampling (DS) algorithm, based on pairwise comparisons of empirical indices computed with re-sampling of the arms' observations and a data-dependent exploration bonus. We show that different variants of this strategy achieve provably optimal regret guarantees when the distributions are bounded and logarithmic regret for semi-bounded distributions with a mild quantile condition. We also show that a simple tuning achieve robustness with respect to a large class of unbounded distributions, at the cost of slightly worse than logarithmic asymptotic regret. We finally provide numerical experiments showing the merits of DS in a decision-making problem on synthetic agriculture data.
Fichier principal
Vignette du fichier
main.pdf (875.44 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03421252 , version 1 (09-11-2021)
hal-03421252 , version 2 (17-11-2021)

Identifiers

Cite

Dorian Baudry, Patrick Saux, Odalric-Ambrym Maillard. From Optimality to Robustness: Dirichlet Sampling Strategies in Stochastic Bandits. NeurIPS 2021 - 35th International Conference on Neural Information Processing Systems, Dec 2021, Sydney, Australia. ⟨hal-03421252v2⟩
100 View
313 Download

Altmetric

Share

Gmail Facebook X LinkedIn More