Exponential Asymptotic Optimality of Whittle Index Policy. - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Article Dans Une Revue Queueing Systems Année : 2023

Exponential Asymptotic Optimality of Whittle Index Policy.

Résumé

We evaluate the performance of Whittle index policy for restless Markovian bandit. It is shown in Weber and Weiss (J Appl Probab 27(3):637–648, 1990) that if the bandit is indexable and the associated deterministic system has a global attractor fixed point, then the Whittle index policy is asymptotically optimal in the regime where the arm population grows proportionally with the number of activation arms. In this paper, we show that, under the same conditions, this convergence rate is exponential in the arm population, unless the fixed point is singular (to be defined later), which almost never happens in practice. Our result holds for the continuous-time model of Weber and Weiss (1990) and for a discrete-time model in which all bandits make synchronous transitions. Our proof is based on the nature of the deterministic equation governing the stochastic system: We show that it is a piecewise affine continuous dynamical system inside the simplex of the empirical measure of the arms. Using simulations and numerical solvers, we also investigate the singular cases, as well as how the level of singularity influences the (exponential) convergence rate. We illustrate our theorem on a Markovian fading channel model.
Fichier principal
Vignette du fichier
optimality_whittle.pdf (1.03 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03041176 , version 1 (11-12-2020)
hal-03041176 , version 2 (19-07-2023)

Licence

Paternité

Identifiants

Citer

Nicolas Gast, Bruno Gaujal, Chen Yan. Exponential Asymptotic Optimality of Whittle Index Policy.. Queueing Systems, 2023, 104, pp.1-44. ⟨10.1007/s11134-023-09875-x⟩. ⟨hal-03041176v2⟩
109 Consultations
182 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More