Dual Extrapolation for Sparse Generalized Linear Models - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Journal of Machine Learning Research Year : 2020

Dual Extrapolation for Sparse Generalized Linear Models

Extrapolation Duale pour les Modèles Linéaires Généralisés Parcimonieux

Abstract

Generalized Linear Models (GLM) form a wide class of regression and classification models, where prediction is a function of a linear combination of the input variables. For statistical inference in high dimension, sparsity inducing regularizations have proven to be useful while offering statistical guarantees. However, solving the resulting optimization problems can be challenging: even for popular iterative algorithms such as coordinate descent, one needs to loop over a large number of variables. To mitigate this, techniques known as screening rules and working sets diminish the size of the optimization problem at hand, either by progressively removing variables, or by solving a growing sequence of smaller problems. For both techniques, significant variables are identified thanks to convex duality arguments. In this paper, we show that the dual iterates of a GLM exhibit a Vector AutoRegressive (VAR) behavior after sign identification, when the primal problem is solved with proximal gradient descent or cyclic coordinate descent. Exploiting this regularity, one can construct dual points that offer tighter certificates of optimality, enhancing the performance of screening rules and helping to design competitive working set algorithms.
Fichier principal
Vignette du fichier
main.pdf (2.84 Mo) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-02263500 , version 1 (05-08-2019)

Identifiers

  • HAL Id : hal-02263500 , version 1

Cite

Mathurin Massias, Samuel Vaiter, Alexandre Gramfort, Joseph Salmon. Dual Extrapolation for Sparse Generalized Linear Models. Journal of Machine Learning Research, 2020, 21 (234), pp.1-33. ⟨hal-02263500⟩
223 View
146 Download

Share

Gmail Facebook X LinkedIn More