Differentially Private Federated Learning on Heterogeneous Data - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Differentially Private Federated Learning on Heterogeneous Data

Résumé

Federated Learning (FL) is a paradigm for large-scale distributed learning which faces two key challenges: (i) training efficiently from highly heterogeneous user data, and (ii) protecting the privacy of participating users. In this work, we propose a novel FL approach (DP-SCAFFOLD) to tackle these two challenges together by incorporating Differential Privacy (DP) constraints into the popular SCAFFOLD algorithm. We focus on the challenging setting where users communicate with a "honest-but-curious" server without any trusted intermediary, which requires to ensure privacy not only towards a third party observing the final model but also towards the server itself. Using advanced results from DP theory and optimization, we establish the convergence of our algorithm for convex and non-convex objectives. Our paper clearly highlights the trade-off between utility and privacy and demonstrates the superiority of DP-SCAFFOLD over the state-ofthe-art algorithm DP-FedAvg when the number of local updates and the level of heterogeneity grows. Our numerical results confirm our analysis and show that DP-SCAFFOLD provides significant gains in practice.
Fichier principal
Vignette du fichier
noble22a.pdf (4.47 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03905078 , version 1 (17-12-2022)

Identifiants

Citer

Maxence Noble, Aurélien Bellet, Aymeric Dieuleveut. Differentially Private Federated Learning on Heterogeneous Data. Proceedings of The 25th International Conference on Artificial Intelligence and Statistics (AISTATS), 2022, Virtual, Spain. ⟨hal-03905078⟩
31 Consultations
12 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More