Verifiable Conditions for the Irreducibility and Aperiodicity of Markov Chains by Analyzing Underlying Deterministic Models - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Bernoulli Year : 2018

Verifiable Conditions for the Irreducibility and Aperiodicity of Markov Chains by Analyzing Underlying Deterministic Models

Alexandre Chotard
  • Function : Author
  • PersonId : 926027
Anne Auger
  • Function : Author
  • PersonId : 751513
  • IdHAL : anne-auger

Abstract

We consider Markov chains that obey the following general non-linear state space model: $\Phi_{k+1} = F(\Phi_k, \alpha(\Phi_k, U_{k+1}))$ where the function $F$ is $C^1$ while $\alpha$ is typically discontinuous and $\{U_k: k \in \mathbb{Z}_{>0} \}$ is an independent and identically distributed process. We assume that for all $x$, the random variable $\alpha(x, U_1)$ admits a density $p_x$ such that $(x, w) \mapsto p_x(w)$ is lower semi-continuous. We generalize and extend previous results that connect properties of the underlying deterministic control model to provide conditions for the chain to be $\varphi$-irreducible and aperiodic. By building on those results, we show that if a rank condition on the controllability matrix is satisfied for all $x$, there is equivalence between the existence of a globally attracting state for the control model and $\varphi$-irreducibility of the Markov chain. Additionally, under the same rank condition on the controllability matrix, we prove that there is equivalence between the existence of a steadily attracting state and the $\varphi$-irreducibility and aperiodicity of the chain. The notion of steadily attracting state is new. We additionally derive practical conditions by showing that the rank condition on the controllability matrix needs to be verified only at a globally attracting state (resp.\ steadily attracting state) for the chain to be a $\varphi$-irreducible T-chain (resp.\ $\varphi$-irreducible aperiodic T-chain). Those results hold under considerably weaker assumptions on the model than previous ones that would require $(x,u) \mapsto F(x,\alpha(x,u))$ to be $C^\infty$ (while it can be discontinuous here). Additionally the establishment of a \emph{necessary and sufficient} condition for the $\varphi$-irreducibility and aperiodicity without a structural assumption on the control set is novel---even for Markov chains where $(x,u) \mapsto F(x,\alpha(x,u))$ is $C^\infty$. We illustrate that the conditions are easy to verify on a non-trivial and non-artificial example of Markov chain arising in the context of adaptive stochastic search algorithms to optimize continuous functions in a black-box scenario.
Fichier principal
Vignette du fichier
bernoulli-revision1.pdf (653.63 Ko) Télécharger le fichier
Origin Files produced by the author(s)
Loading...

Dates and versions

hal-01222222 , version 1 (29-10-2015)
hal-01222222 , version 2 (23-11-2017)

Identifiers

Cite

Alexandre Chotard, Anne Auger. Verifiable Conditions for the Irreducibility and Aperiodicity of Markov Chains by Analyzing Underlying Deterministic Models. Bernoulli, 2018, 25 (1), pp.112-147. ⟨10.3150/17-BEJ970⟩. ⟨hal-01222222v2⟩
459 View
316 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More