Global implicit function theorems and the online Expectation-Maximisation algorithm - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Australian and New Zealand Journal of Statistics Year : 2022

Global implicit function theorems and the online Expectation-Maximisation algorithm

Abstract

The expectation-maximisation (EM) algorithm is an important tool for statistical computation. Due to the changing nature of data, online and mini-batch variants of EM and EM-like algorithms have become increasingly popular. The consistency of the estimator sequences that are produced by these EM variants often rely on an assumption regarding the continuous differentiability of a parameter update function. In many cases, the parameter update function is often not in closed form and may only be defined implicitly, which makes the verification of the continuous differentiability property difficult. We demonstrate how a global implicit function theorem can be used to verify such properties in the cases of finite mixtures of distributions in the exponential family and more generally when the component specific distribution admits a data augmentation scheme in the exponential family. We demonstrate the use of such a theorem in the case of mixtures of beta distributions, gamma distributions, fully-visible Boltzmann machines and Student distributions. Via numerical simulations, we provide empirical evidence towards the consistency of the online EM algorithm parameter estimates in such cases.
Fichier principal
Vignette du fichier
oemrev14hal.pdf (23.4 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03110213 , version 1 (14-01-2021)
hal-03110213 , version 2 (12-11-2021)

Identifiers

Cite

Hien Duy Nguyen, Florence Forbes. Global implicit function theorems and the online Expectation-Maximisation algorithm. Australian and New Zealand Journal of Statistics, 2022, 64 (2), pp.255-281. ⟨10.1111/anzs.12356⟩. ⟨hal-03110213v2⟩
158 View
127 Download

Altmetric

Share

Gmail Facebook X LinkedIn More