Throughput-Optimal Topology Design for Cross-Silo Federated Learning - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

Throughput-Optimal Topology Design for Cross-Silo Federated Learning

Othmane Marfoq
Chuan Xu
Giovanni Neglia
Richard Vidal
  • Fonction : Auteur
  • PersonId : 1081827

Résumé

Federated learning usually employs a client-server architecture where an orchestrator iteratively aggregates model updates from remote clients and pushes them back a refined model. This approach may be inefficient in cross-silo settings, as close-by data silos with high-speed access links may exchange information faster than with the orchestrator, and the orchestrator may become a communication bottleneck. In this paper we define the problem of topology design for cross-silo federated learning using the theory of max-plus linear systems to compute the system throughput---number of communication rounds per time unit. We also propose practical algorithms that, under the knowledge of measurable network characteristics, find a topology with the largest throughput or with provable throughput guarantees. In realistic Internet networks with 10 Gbps access links for silos, our algorithms speed up training by a factor 9 and 1.5 in comparison to the master-slave architecture and to state-of-the-art MATCHA, respectively. Speedups are even larger with slower access links.
Fichier principal
Vignette du fichier
NIPS2020.pdf (12.18 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-03007834 , version 1 (17-11-2020)
hal-03007834 , version 2 (17-11-2020)

Identifiants

Citer

Othmane Marfoq, Chuan Xu, Giovanni Neglia, Richard Vidal. Throughput-Optimal Topology Design for Cross-Silo Federated Learning. NeurIPS 2020 - 34th Conference on Neural Information Processing Systems, Dec 2020, Vancouver / Online, Canada. ⟨hal-03007834v2⟩
171 Consultations
186 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More