Aggregated hold out for sparse linear regression with a robust loss function - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Electronic Journal of Statistics Year : 2022

Aggregated hold out for sparse linear regression with a robust loss function

Abstract

Sparse linear regression methods generally have a free hyperparameter which controls the amount of sparsity, and is subject to a bias-variance tradeoff. This article considers the use of Aggregated hold-out to aggregate over values of this hyperparameter, in the context of linear regression with the Huber loss function. Aggregated hold-out (Agghoo) is a procedure which averages estimators selected by hold-out (cross-validation with a single split). In the theoretical part of the article, it is proved that Agghoo satisfies a non-asymptotic oracle inequality when it is applied to sparse estimators which are parametrized by their zero-norm. In particular , this includes a variant of the Lasso introduced by Zou, Hastié and Tibshirani. Simulations are used to compare Agghoo with cross-validation. They show that Agghoo performs better than CV when the intrinsic dimension is high and when there are confounders correlated with the predictive covariates.
Fichier principal
Vignette du fichier
sparse_huber_hal_v3.pdf (600.61 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-02485694 , version 1 (26-02-2020)
hal-02485694 , version 2 (04-05-2021)
hal-02485694 , version 3 (28-11-2022)

Identifiers

Cite

Guillaume Maillard. Aggregated hold out for sparse linear regression with a robust loss function. Electronic Journal of Statistics , 2022, 16 (1), pp.935 -- 997. ⟨10.1214/21-EJS1952⟩. ⟨hal-02485694v3⟩
83 View
89 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More