Optimally Solving Dec-POMDPs as Continuous-State MDPs - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Journal of Artificial Intelligence Research Year : 2016

Optimally Solving Dec-POMDPs as Continuous-State MDPs

Abstract

Decentralized partially observable Markov decision processes (Dec-POMDPs) provide a general model for decision-making under uncertainty in decentralized settings, but are difficult to solve optimally (NEXP-Complete). As a new way of solving these problems, we introduce the idea of transforming a Dec-POMDP into a continuous-state deterministic MDP with a piecewise-linear and convex value function. This approach makes use of the fact that planning can be accomplished in a centralized offline manner, while execution can still be decentralized. This new Dec-POMDP formulation , which we call an occupancy MDP, allows powerful POMDP and continuous-state MDP methods to be used for the first time. To provide scalability, we refine this approach by combining heuristic search and compact representations that exploit the structure present in multi-agent domains, without losing the ability to converge to an optimal solution. In particular, we introduce a feature-based heuristic search value iteration (FB-HSVI) algorithm that relies on feature-based compact representations, point-based updates and efficient action selection. A theoretical analysis demonstrates that FB-HSVI terminates in finite time with an optimal solution. We include an extensive empirical analysis using well-known benchmarks, thereby demonstrating that our approach provides significant scalability improvements compared to the state of the art.
Fichier principal
Vignette du fichier
dibangoye16a.pdf (758.17 Ko) Télécharger le fichier
Origin : Publisher files allowed on an open archive
Loading...

Dates and versions

hal-01279444 , version 1 (01-03-2016)

Licence

Public Domain

Identifiers

Cite

Jilles Steeve Dibangoye, Christopher Amato, Olivier Buffet, François Charpillet. Optimally Solving Dec-POMDPs as Continuous-State MDPs. Journal of Artificial Intelligence Research, 2016, 55, pp.443-497. ⟨10.1613/jair.4623⟩. ⟨hal-01279444⟩
782 View
340 Download

Altmetric

Share

Gmail Facebook X LinkedIn More