Robust high dimensional learning for Lipschitz and convex losses - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Article Dans Une Revue Journal of Machine Learning Research Année : 2020

Robust high dimensional learning for Lipschitz and convex losses

Résumé

We establish risk bounds for Regularized Empirical Risk Minimizers (RERM) when the loss is Lipschitz and convex and the regularization function is a norm. In a first part, we obtain these results in the i.i.d. setup under subgaussian assumptions on the design. In a second part, a more general framework where the design might have heavier tails and data may be corrupted by outliers both in the design and the response variables is considered. In this situation, RERM performs poorly in general. We analyse an alternative procedure based on median-of-means principles and called "minmax MOM". We show optimal subgaussian deviation rates for these estimators in the relaxed setting. The main results are meta-theorems allowing a wide-range of applications to various problems in learning theory. To show a non-exhaustive sample of these potential applications, it is applied to classification problems with logistic loss functions regularized by LASSO and SLOPE, to regression problems with Huber loss regularized by Group LASSO and Total Variation. Another advantage of the minmax MOM formulation is that it suggests a systematic way to slightly modify descent based algorithms used in high-dimensional statistics to make them robust to outliers Lecué and Lerasle (2017b). We illustrate this principle in a Simulations section where a " minmax MOM" version of classical proximal descent algorithms are turned into robust to outliers algorithms.
Fichier principal
Vignette du fichier
19-585.pdf (607.47 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-03874496 , version 1 (28-11-2022)

Licence

Paternité

Identifiants

Citer

Geoffrey Chinot, Matthieu Lerasle, Guillaume Lecué. Robust high dimensional learning for Lipschitz and convex losses. Journal of Machine Learning Research, 2020, 21, pp.1-47. ⟨10.48550/arXiv.1905.04281⟩. ⟨hal-03874496⟩
15 Consultations
10 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More