Tighter PAC-Bayes Generalisation Bounds by Leveraging Example Difficulty - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2022

Tighter PAC-Bayes Generalisation Bounds by Leveraging Example Difficulty

Abstract

We introduce a modified version of the excess risk, which can be used to obtain tighter, fast-rate PAC-Bayesian generalisation bounds. This modified excess risk leverages information about the relative hardness of data examples to reduce the variance of its empirical counterpart, tightening the bound. We combine this with a new bound for $[-1, 1]$-valued (and potentially non-independent) signed losses, which is more favourable when they empirically have low variance around $0$. The primary new technical tool is a novel result for sequences of interdependent random vectors which may be of independent interest. We empirically evaluate these new bounds on a number of real-world datasets.

Dates and versions

hal-03953307 , version 1 (23-01-2023)

Identifiers

Cite

Felix Biggs, Benjamin Guedj. Tighter PAC-Bayes Generalisation Bounds by Leveraging Example Difficulty. AISTATS 2023 - 26th International Conference on Artificial Intelligence and Statistics, Apr 2023, Valencia, Spain. ⟨hal-03953307⟩
44 View
0 Download

Altmetric

Share

Gmail Facebook X LinkedIn More