A Universal 2-state n-action Adaptive Management Solver - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

A Universal 2-state n-action Adaptive Management Solver

Luz V. Pascal
  • Function : Author
Marianne Akian
  • Function : Author
Sam Nicol
  • Function : Author


In poor data and urgent decision-making applications, managers need to make decisions without complete knowledge of the system dynamics. In biodiversity conservation, adaptive management (AM) is the principal tool for decision-making under uncertainty. AM can be solved using simplified Mixed Observable Markov Decision Processes called hidden model MDPs (hmMDPs) when the unknown dynamics are assumed stationary. hmMDPs provide optimal policies to AM problems by augmenting the MDP state space with an unobservable state variable representing a finite set of predefined models. A drawback in formalising an AM problem is that experts are often solicited to provide this predefined set of models by specifying the transition matrices. Expert elicitation is a challenging and time-consuming process that is prone to biases, and a key assumption of hmMDPs is that the true transition matrix will be included in the candidate model set. We propose an original approach to build a hmMDP with a universal set of predefined models that is capable of solving any 2-state n-action AM problem. Our approach uses properties of the transition matrices to build the model set and is independent of expert input, removing the potential for expert error in the optimal solution. We provide analytical formulations to derive the minimum set of models to include into an hmMDP to solve any AM problems with 2 states and n actions. We assess our universal AM algorithm on two species conservation case studies from Australia and randomly generated problems.
No file

Dates and versions

hal-03505465 , version 1 (30-12-2021)


  • HAL Id : hal-03505465 , version 1


Luz V. Pascal, Marianne Akian, Sam Nicol, Iadine Chades. A Universal 2-state n-action Adaptive Management Solver. Proceedings of the AAAI Conference on Artificial Intelligence, Feb 2021, virtual conference, United States. pp.14884-14892. ⟨hal-03505465⟩
26 View
0 Download


Gmail Mastodon Facebook X LinkedIn More