Monte-Carlo Search for an Equilibrium in Dec-POMDPs - Inria - Institut national de recherche en sciences et technologies du numérique
Conference Papers Year : 2023

Monte-Carlo Search for an Equilibrium in Dec-POMDPs

Abstract

Decentralized partially observable Markov decision processes (Dec-POMDPs) formalize the problem of designing individual controllers for a group of collaborative agents under stochastic dynamics and partial observability. Seeking a global optimum is difficult (NEXP complete), but seeking a Nash equilibrium -- each agent policy being a best response to the other agents -- is more accessible, and allowed addressing infinite-horizon problems with solutions in the form of finite state controllers. In this paper, we show that this approach can be adapted to cases where only a generative model (a simulator) of the Dec-POMDP is available. This requires relying on a simulation-based POMDP solver to construct an agent's FSC node by node. A related process is used to heuristically derive initial FSCs. Experiment with benchmarks shows that MC-JESP is competitive with exisiting Dec-POMDP solvers, even better than many offline methods using explicit models.
Fichier principal
Vignette du fichier
MCJESP.pdf (582.71 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04191493 , version 1 (30-08-2023)

Licence

Identifiers

Cite

Yang You, Vincent Thomas, Francis Colas, Olivier Buffet. Monte-Carlo Search for an Equilibrium in Dec-POMDPs. The 39th Conference on Uncertainty in Artificial Intelligence (UAI), Jul 2023, Pittsburgh, PA, United States. ⟨10.48550/arXiv.2305.11811⟩. ⟨hal-04191493⟩
47 View
61 Download

Altmetric

Share

More