Throughput-Optimal Topology Design for Cross-Silo Federated Learning - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2020

Throughput-Optimal Topology Design for Cross-Silo Federated Learning

Othmane Marfoq
Chuan Xu
Giovanni Neglia
Richard Vidal
  • Fonction : Auteur

Résumé

Federated learning usually employs a client-server architecture where an orchestrator iteratively aggregates model updates from remote clients and pushes them back a refined model. This approach may be inefficient in cross-silo settings, as close-by data silos with high-speed access links may exchange information faster than with the orchestrator, and the orchestrator may become a communication bottleneck. In this paper we define the problem of topology design for cross-silo federated learning using the theory of max-plus linear systems to compute the system throughput---number of communication rounds per time unit. We also propose practical algorithms that, under the knowledge of measurable network characteristics, find a topology with the largest throughput or with provable throughput guarantees. In realistic Internet networks with 10 Gbps access links for silos, our algorithms speed up training by a factor 9 and 1.5 in comparison to the master-slave architecture and to state-of-the-art MATCHA, respectively. Speedups are even larger with slower access links.
Fichier principal
Vignette du fichier
2010.12229.pdf (11.63 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-03007834 , version 1 (17-11-2020)
hal-03007834 , version 2 (17-11-2020)

Identifiants

Citer

Othmane Marfoq, Chuan Xu, Giovanni Neglia, Richard Vidal. Throughput-Optimal Topology Design for Cross-Silo Federated Learning. 2020. ⟨hal-03007834v1⟩
171 Consultations
186 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More