Communication Dans Un Congrès Année : 2026

Decentralized Machine Learning with Centralized Performance Guarantees via Gibbs Algorithms

Résumé

In this paper, it is shown, for the first time, that centralized performance is achievable in decentralized learning without sharing the local datasets. Specifically, when clients adopt an empirical risk minimization with relative-entropy regularization (ERM-RER) learning framework and a forward-backward communication between clients is established, it suffices to share the locally obtained Gibbs measures to achieve the same performance as that of a centralized ERM-RER with access to all the datasets. The core idea is that the Gibbs measure produced by client~$k$ is used, as reference measure, by client~$k+1$. This effectively establishes a principled way to encode prior information through a reference measure. In particular, achieving centralized performance in the decentralized setting requires a specific scaling of the regularization factors with the local sample sizes. Overall, this result opens the door to novel decentralized learning paradigms that shift the collaboration strategy from sharing data to sharing the local inductive bias via the reference measures over the set of models.

Fichier principal
Vignette du fichier
BERMUDEZ_ISIT2026_Gibbs.pdf (133.5 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence
HAL

Est complété par hal-05471674 Rapport Yaiza Bermudez, Samir M. Perlaza, Iñaki Esnaola. Decentralized Machine Learning with Centralized Performance Guarantees via Gibbs Algorithms. RR-9608, Inria. 2026. ⟨hal-05471674⟩

Dates et versions

hal-05589435 , version 1 (28-04-2026)

Licence

Identifiants

  • HAL Id : hal-05589435 , version 1

Citer

Yaiza Bermudez, Samir M. Perlaza, Iñaki Esnaola. Decentralized Machine Learning with Centralized Performance Guarantees via Gibbs Algorithms. ISIT 2026 - IEEE International Symposium on Information Theory, Jun 2026, Guangzhou, China. ⟨hal-05589435⟩
11 Consultations
32 Téléchargements

Partager

  • More