Differential Privacy Guarantees for Stochastic Gradient Langevin Dynamics - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Preprints, Working Papers, ... Year : 2022

Differential Privacy Guarantees for Stochastic Gradient Langevin Dynamics

Abstract

We analyse the privacy leakage of noisy stochastic gradient descent by modeling Rényi divergence dynamics with Langevin diffusions. Inspired by recent work on non-stochastic algorithms, we derive similar desirable properties in the stochastic setting. In particular, we prove that the privacy loss converges exponentially fast for smooth and strongly convex objectives under constant step size, which is a significant improvement over previous DP-SGD analyses. We also extend our analysis to arbitrary sequences of varying step sizes and derive new utility bounds. Last, we propose an implementation and our experiments show the practical utility of our approach compared to classical DP-SGD libraries.
Fichier principal
Vignette du fichier
_ICML__Langevin_SGD___ICML_Template.pdf (361.29 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03547726 , version 1 (28-01-2022)
hal-03547726 , version 2 (05-02-2022)

Identifiers

  • HAL Id : hal-03547726 , version 2

Cite

Théo Ryffel, Francis Bach, David Pointcheval. Differential Privacy Guarantees for Stochastic Gradient Langevin Dynamics. 2022. ⟨hal-03547726v2⟩
96 View
162 Download

Share

Gmail Facebook X LinkedIn More