Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization

Abstract

Viewing optimization methods as numerical integrators for ordinary differential equations (ODEs) provides a thought-provoking modern framework for studying accelerated first-order optimizers. In this literature, acceleration is often supposed to be linked to the quality of the integrator (accuracy, energy preservation, symplecticity). In this work, we propose a novel ordinary differential equation that questions this connection: both the explicit and the semi-implicit (a.k.a symplectic) Euler discretizations on this ODE lead to an accelerated algorithm for convex programming. Although semi-implicit methods are well-known in numerical analysis to enjoy many desirable features for the integration of physical systems, our findings show that these properties do not necessarily relate to acceleration.
Fichier principal
Vignette du fichier
zhang21m.pdf (2.06 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03454377 , version 1 (29-11-2021)

Identifiers

  • HAL Id : hal-03454377 , version 1

Cite

Peiyuan Zhang, Antonio Orvieto, Hadi Daneshmand, Thomas Hofmann, Roy Smith. Revisiting the Role of Euler Numerical Integration on Acceleration and Stability in Convex Optimization. AISTATS 2021 - 24th International Conference on Artifi-cial Intelligence and Statistics, Apr 2021, Virtual, Unknown Region. ⟨hal-03454377⟩
21 View
50 Download

Share

Gmail Mastodon Facebook X LinkedIn More