Cooperation through communication in decentralized Markov games - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2004

Cooperation through communication in decentralized Markov games

Raghav Aras
  • Function : Author
  • PersonId : 830439
Alain Dutech


In this paper, we present a comunication-integrated reinforcement-learning algorithm for a general-sum Markov game or MG played by independent, cooperative agents. The algorithm assumes that agents can communicate but do not know the purpose (the semantic) of doing so. We model agents that have different tasks, some of which may be commonly beneficial. The objective of the agents is to determine which are the commonly beneficial tasks, and learn a sequence of actions that achieves the common tasks. In other words, the agents play a multi-stage coordination game, of which they know niether the stage-wise payoff matrix nor the stage transition matrix. Our principal interest is in imposing realistic conditions of learning on the agents. Towards this end, we assume that they operate in a strictly imperfect monitoring setting wherein they do not observe one another's actions or rewards. A learning algorithm for a Markov game under this stricter condition of learning has not been proposed yet to our knowledge. We describe this Markov game with individual reward functions as a new formalism, decentralized Markov game or Dec-MG, a formalism borrowed from Dec-MDP (Markov decison process). For the communicatory aspect of the learning conditions, we propose a series of communication frameworks graduated in terms of facilitation of information exchange amongst the agents. We present results of testing our algorithm in a toy problem MG called a total guessing game.
Fichier principal
Vignette du fichier
aras_aista04.pdf (134.19 Ko) Télécharger le fichier

Dates and versions

inria-00000210 , version 1 (13-09-2005)


  • HAL Id : inria-00000210 , version 1


Raghav Aras, Alain Dutech, François Charpillet. Cooperation through communication in decentralized Markov games. International Conference on Advances in Intelligent Systems - Theory and Applications - AISTA'2004, Nov 2004, Luxembourg-Kirchberg/Luxembourg. ⟨inria-00000210⟩
260 View
154 Download


Gmail Facebook Twitter LinkedIn More