Denumerable constrained Markov decision problems and finite approximations - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Rapport (Rapport De Recherche) Année : 1991

Denumerable constrained Markov decision problems and finite approximations

Résumé

The purpose of this paper is two fold. First to establish the Theory of discounted constrained Markov Decision Process with a countable state and action spaces with general multi-chain structure. Second, to introduce finite approximation methods. We define the occupation measures and obtain properties of the set of all achievable occupation measures under the different admissible policies. We establish the optimality of stationary policies for the constrained control problem, and a LP with a countable number of decision variables through wich optimal stationary policies are computed. Since for such a LP one cannot expect to find an optimal solution in a finite number of operations, we present two schemes for finite approximations and establish the convergence of optimal values and policies for both the discounted and the expected average cost, with unbounded cost. Sometimes it turns to be easier to solve the problem with infinite state space than the problem with finite yet large state space. Based on the optimal policy for the problem with infinite state space, we construct policies which are almost optimal for the problem with truncated state space. This method is applied to obtain an e-optimal policy for a problem of optimal priority assignment under constraints for a system of K finite queues.
Fichier principal
Vignette du fichier
RR-1568.pdf (346.59 Ko) Télécharger le fichier

Dates et versions

inria-00074993 , version 1 (24-05-2006)

Identifiants

  • HAL Id : inria-00074993 , version 1

Citer

Eitan Altman. Denumerable constrained Markov decision problems and finite approximations. [Research Report] RR-1568, INRIA. 1991, pp.33. ⟨inria-00074993⟩
124 Consultations
256 Téléchargements

Partager

Gmail Facebook X LinkedIn More