Minimization by Incremental Stochastic Surrogate Optimization for Large Scale Nonconvex Problems - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Minimization by Incremental Stochastic Surrogate Optimization for Large Scale Nonconvex Problems

Belhal Karimi
  • Fonction : Auteur
  • PersonId : 1211377
Eric Moulines
Ping Li
  • Fonction : Auteur
  • PersonId : 1211379

Résumé

Many constrained, nonconvex and nonsmooth optimization problems can be tackled using the Majorization-Minimization (MM) method which alternates between constructing a surrogate function which upper bounds the objective function, and then minimizing this surrogate. For problems which minimize a finite sum of functions, a stochastic version of the MM method selects a batch of functions at random at each iteration and optimizes the accumulated surrogate. However, in many cases of interest such as variational inference for latent variable models, the surrogate functions are expressed as an expectation. In this contribution, we propose a doubly stochastic MM method based on Monte Carlo approximation of these stochastic surrogates. We establish asymptotic and non-asymptotic convergence of our scheme in a constrained, nonconvex, nonsmooth optimization setting. We apply our new framework for inference of logistic regression model with missing data and for variational inference of Bayesian variants of LeNet-5 and Resnet-18 on benchmark datasets.
Fichier principal
Vignette du fichier
karimi22a.pdf (557.57 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03921370 , version 1 (03-01-2023)

Identifiants

  • HAL Id : hal-03921370 , version 1

Citer

Belhal Karimi, Hoi-To Wai, Eric Moulines, Ping Li. Minimization by Incremental Stochastic Surrogate Optimization for Large Scale Nonconvex Problems. ALT 2022 - The 33rd International Conference on Algorithmic Learning Theory, Mar 2022, Paris, France. ⟨hal-03921370⟩
15 Consultations
16 Téléchargements

Partager

Gmail Facebook X LinkedIn More