Preprints, Working Papers, ... Year : 2025

A multilevel stochastic regularized first-order method with application to training

Filippo Marini
  • Function : Author
  • PersonId : 1489050
Margherita Porcelli
  • Function : Author
  • PersonId : 1489051

Abstract

In this paper, we propose a new multilevel stochastic framework for the solution of optimization problems. The proposed approach uses random regularized first-order models that exploit an available hierarchical description of the problem, being either in the classical variable space or in the function space, meaning that different levels of accuracy for the objective function are available. The converge analysis of the method is conducted and its numerical behavior is tested on the solution of finite-sum minimization problems. Indeed, the multilevel framework is tailored to the solution of such problems resulting in fact in a nontrivial variance reduction technique with adaptive step-size that outperforms standard approaches when solving nonconvex problems. Differently from classical deterministic multilevel methods, our stochastic method does not require the finest approximation to coincide with the original objective function. This allows to avoid the evaluation of the full sum in finite-sum minimization problems, opening at the solution of classification problems with large data sets.
Fichier principal
Vignette du fichier
A_variance_reduction_perspective_on_multilevel_methods__an_automatic_stepsize_choice_for_SVRG__1_ (10).pdf (740.93 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04884049 , version 1 (13-01-2025)

Licence

Identifiers

  • HAL Id : hal-04884049 , version 1

Cite

Filippo Marini, Margherita Porcelli, Elisa Riccietti. A multilevel stochastic regularized first-order method with application to training. 2025. ⟨hal-04884049⟩
0 View
0 Download

Share

More