PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingales - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Article Dans Une Revue Transactions on Machine Learning Research Journal Année : 2023

PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingales

Résumé

While PAC-Bayes is now an established learning framework for light-tailed losses (\emph{e.g.}, subgaussian or subexponential), its extension to the case of heavy-tailed losses remains largely uncharted and has attracted a growing interest in recent years. We contribute PAC-Bayes generalisation bounds for heavy-tailed losses under the sole assumption of bounded variance of the loss function. Under that assumption, we extend previous results from \citet{kuzborskij2019efron}. Our key technical contribution is exploiting an extention of Markov's inequality for supermartingales. Our proof technique unifies and extends different PAC-Bayesian frameworks by providing bounds for unbounded martingales as well as bounds for batch and online learning with heavy-tailed losses.
Fichier principal
Vignette du fichier
main.pdf (378.69 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03815101 , version 1 (14-10-2022)
hal-03815101 , version 2 (27-04-2023)
hal-03815101 , version 3 (27-04-2023)

Identifiants

Citer

Maxime Haddouche, Benjamin Guedj. PAC-Bayes Generalisation Bounds for Heavy-Tailed Losses through Supermartingales. Transactions on Machine Learning Research Journal, 2023. ⟨hal-03815101v3⟩
54 Consultations
43 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More