Analyzing the discrepancy principle for kernelized spectral filter learning algorithms - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Journal of Machine Learning Research Year : 2021

Analyzing the discrepancy principle for kernelized spectral filter learning algorithms

Analyse d'une règle d'arrêt prématuré basée sur le principe de discrépance pour les algorithmes à filtrage spectral

Abstract

We investigate the construction of early stopping rules in the non-parametric regression problem where iterative learning algorithms are used and the optimal iteration number is unknown. More precisely, we study the discrepancy principle, as well as modifications based on smoothed residuals, for kernelized spectral filter learning algorithms including gradient descent. Our main theoretical bounds are oracle inequalities established for the empirical estimation error (fixed design), and for the prediction error (random design). From these finite-sample bounds it follows that the classical discrepancy principle is statistically adaptive for slow rates occurring in the hard learning scenario, while the smoothed discrepancy principles are adaptive over ranges of faster rates (resp. higher smoothness parameters). Our approach relies on deviation inequalities for the stopping rules in the fixed design setting, combined with change-of-norm arguments to deal with the random design setting.
Fichier principal
Vignette du fichier
20-358.pdf (561.25 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Licence : Copyright

Dates and versions

hal-02548917 , version 1 (21-04-2020)
hal-02548917 , version 2 (07-03-2023)

Identifiers

Cite

Alain Celisse, Martin Wahl. Analyzing the discrepancy principle for kernelized spectral filter learning algorithms. Journal of Machine Learning Research, 2021. ⟨hal-02548917v2⟩
160 View
176 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More