Comparing Comparators in Generalization Bounds - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Preprints, Working Papers, ... Year : 2023

Comparing Comparators in Generalization Bounds

Abstract

We derive generic information-theoretic and PAC-Bayesian generalization bounds involving an arbitrary convex comparator function, which measures the discrepancy between the training and population loss. The bounds hold under the assumption that the cumulant-generating function (CGF) of the comparator is upper-bounded by the corresponding CGF within a family of bounding distributions. We show that the tightest possible bound is obtained with the comparator being the convex conjugate of the CGF of the bounding distribution, also known as the Cramér function. This conclusion applies more broadly to generalization bounds with a similar structure. This confirms the near-optimality of known bounds for bounded and sub-Gaussian losses and leads to novel bounds under other bounding distributions.
Fichier principal
Vignette du fichier
2310.10534.pdf (1.01 Mo) Télécharger le fichier
Origin : Files produced by the author(s)
Licence : CC BY NC SA - Attribution - NonCommercial - ShareAlike

Dates and versions

hal-04259101 , version 1 (25-10-2023)

Licence

Attribution

Identifiers

Cite

Fredrik Hellström, Benjamin Guedj. Comparing Comparators in Generalization Bounds. 2023. ⟨hal-04259101⟩
8 View
10 Download

Altmetric

Share

Gmail Facebook X LinkedIn More