Federated Learning with Nonvacuous Generalisation Bounds - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

Federated Learning with Nonvacuous Generalisation Bounds

Résumé

We introduce a novel strategy to train randomised predictors in federated learning, where each node of the network aims at preserving its privacy by releasing a local predictor but keeping secret its training dataset with respect to the other nodes. We then build a global randomised predictor which inherits the properties of the local private predictors in the sense of a PAC-Bayesian generalisation bound. We consider the synchronous case where all nodes share the same training objective (derived from a generalisation bound), and the asynchronous case where each node may have its own personalised training objective. We show through a series of numerical experiments that our approach achieves a comparable predictive performance to that of the batch approach where all datasets are shared across nodes. Moreover the predictors are supported by numerically nonvacuous generalisation bounds while preserving privacy for each node. We explicitly compute the increment on predictive performance and generalisation bounds between batch and federated settings, highlighting the price to pay to preserve privacy.
Fichier principal
Vignette du fichier
2310.11203.pdf (590.47 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Licence : CC BY NC SA - Paternité - Pas d'utilisation commerciale - Partage selon les Conditions Initiales

Dates et versions

hal-04260486 , version 1 (26-10-2023)

Licence

Paternité

Identifiants

Citer

Pierre Jobic, Maxime Haddouche, Benjamin Guedj. Federated Learning with Nonvacuous Generalisation Bounds. 2023. ⟨hal-04260486⟩
36 Consultations
3 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More