The perturbed prox-preconditioned spider algorithm: non-asymptotic convergence bounds - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

The perturbed prox-preconditioned spider algorithm: non-asymptotic convergence bounds

Abstract

A novel algorithm named Perturbed Prox-Preconditioned SPIDER (3P-SPIDER) is introduced. It is a stochastic variancereduced proximal-gradient type algorithm built on Stochastic Path Integral Differential EstimatoR (SPIDER), an algorithm known to achieve near-optimal first-order oracle inequality for nonconvex and nonsmooth optimization. Compared to the vanilla prox-SPIDER, 3P-SPIDER uses preconditioned gradient estimators. Preconditioning can either be applied "explicitly" to a gradient estimator or be introduced "implicitly" as in applications to the EM algorithm. 3P-SPIDER also assumes that the preconditioned gradients may (possibly) be not known in closed analytical form and therefore must be approximated which adds an additional degree of perturbation. Studying the convergence in expectation, we show that 3P-SPIDER achieves a near-optimal oracle inequality O(n^(1/2) /epsilon) where n is the number of observations and epsilon the target precision even when the gradient is estimated by Monte Carlo methods. We illustrate the algorithm on an application to the minimization of a penalized empirical loss.
Fichier principal
Vignette du fichier
FMtheory_HAL.pdf (164.83 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03183775 , version 1 (28-03-2021)
hal-03183775 , version 2 (24-05-2021)

Identifiers

  • HAL Id : hal-03183775 , version 2

Cite

Gersende Fort, E Moulines. The perturbed prox-preconditioned spider algorithm: non-asymptotic convergence bounds. SSP 2021 - IEEE Statistical Signal Processing Workshop, Jul 2021, Rio de Janeiro, Brazil. ⟨hal-03183775v2⟩
147 View
114 Download

Share

Gmail Mastodon Facebook X LinkedIn More