Testing Indexability and Computing Whittle and Gittins Index in Subcubic Time - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Mathematical Methods of Operations Research Year : 2023

Testing Indexability and Computing Whittle and Gittins Index in Subcubic Time

Abstract

Whittle index is a generalization of Gittins index that provides very efficient allocation rules for restless multi-armed bandits. In this work, we develop an algorithm to test the indexability and compute the Whittle indices of any finite-state restless bandit arm. This algorithm works in the discounted and non-discounted cases, and can compute Gittins index. Our algorithm builds on three tools: (1) a careful characterization of Whittle index that allows one to compute recursively the kth smallest index from the (k − 1)th smallest, and to test indexability, (2) the use of the Sherman-Morrison formula to make this recursive computation efficient, and (3) a sporadic use of the fastest matrix inversion and multiplication methods to obtain a subcubic complexity. We show that an efficient use of the Sherman-Morrison formula leads to an algorithm that computes Whittle index in (2/3)n^3 + o(n^3 ) arithmetic operations, where n is the number of states of the arm. The careful use of fast matrix multiplication leads to the first subcubic algorithm to compute Whittle or Gittins index: By using the current fastest matrix multiplication, the theoretical complexity of our algorithm is O(n^2.5286 ). We also develop an efficient implementation of our algorithm that can compute indices of Markov chains with several thousands of states in less than a few seconds.
Fichier principal
Vignette du fichier
sn-main.pdf (891.92 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03602458 , version 1 (09-03-2022)
hal-03602458 , version 2 (29-04-2022)
hal-03602458 , version 3 (08-12-2022)
hal-03602458 , version 4 (17-04-2023)
hal-03602458 , version 5 (20-06-2023)

Licence

Identifiers

Cite

Nicolas Gast, Bruno Gaujal, Kimang Khun. Testing Indexability and Computing Whittle and Gittins Index in Subcubic Time. Mathematical Methods of Operations Research, 2023, ⟨10.1007/s00186-023-00821-4⟩. ⟨hal-03602458v5⟩
201 View
340 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More