On the Convergence of Single-Call Stochastic Extra-Gradient Methods - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2019

On the Convergence of Single-Call Stochastic Extra-Gradient Methods

Résumé

Owing to their stability and convergence speed, extragradient methods have become a staple for solving large-scale saddle-point problems in machine learning. The basic premise of these algorithms is the use of an extrapolation step before performing an update; thanks to this exploration step, extragradient methods overcome many of the non-convergence issues that plague gradient descent/ascent schemes. On the other hand, as we show in this paper, running vanilla extragradient with stochastic gradients may jeopardize its convergence, even in simple bilinear models. To overcome this failure, we investigate a double stepsize extragradient algorithm where the exploration step evolves at a more aggressive timescale compared to the update step. We show that this modification allows the method to converge even with stochastic gradients, and we derive sharp convergence rates under an error bound condition.
Fichier principal
Vignette du fichier
SingleCall-NIPS.pdf (2.94 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02403555 , version 1 (10-12-2019)

Identifiants

Citer

Yu-Guan Hsieh, Franck Iutzeler, Jérôme Malick, Panayotis Mertikopoulos. On the Convergence of Single-Call Stochastic Extra-Gradient Methods. Advances in Neural Information Processing Systems, Dec 2019, Vancouver, Canada. ⟨hal-02403555⟩
149 Consultations
201 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More