Exponential Asymptotic Optimality of Whittle Index Policy. - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Queueing Systems Year : 2023

Exponential Asymptotic Optimality of Whittle Index Policy.

Abstract

We evaluate the performance of Whittle index policy for restless Markovian bandit. It is shown in Weber and Weiss (J Appl Probab 27(3):637–648, 1990) that if the bandit is indexable and the associated deterministic system has a global attractor fixed point, then the Whittle index policy is asymptotically optimal in the regime where the arm population grows proportionally with the number of activation arms. In this paper, we show that, under the same conditions, this convergence rate is exponential in the arm population, unless the fixed point is singular (to be defined later), which almost never happens in practice. Our result holds for the continuous-time model of Weber and Weiss (1990) and for a discrete-time model in which all bandits make synchronous transitions. Our proof is based on the nature of the deterministic equation governing the stochastic system: We show that it is a piecewise affine continuous dynamical system inside the simplex of the empirical measure of the arms. Using simulations and numerical solvers, we also investigate the singular cases, as well as how the level of singularity influences the (exponential) convergence rate. We illustrate our theorem on a Markovian fading channel model.
Fichier principal
Vignette du fichier
optimality_whittle.pdf (1.03 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03041176 , version 1 (11-12-2020)
hal-03041176 , version 2 (19-07-2023)

Licence

Attribution

Identifiers

Cite

Nicolas Gast, Bruno Gaujal, Chen Yan. Exponential Asymptotic Optimality of Whittle Index Policy.. Queueing Systems, 2023, 104, pp.1-44. ⟨10.1007/s11134-023-09875-x⟩. ⟨hal-03041176v2⟩
99 View
170 Download

Altmetric

Share

Gmail Facebook X LinkedIn More