Gossiping GANs - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2018

Gossiping GANs

Résumé

A recently celebrated kind of deep neural networks is Generative Adversarial Networks. GANs are generators of samples from a distribution that has been learned; they are up to now centrally trained from local data on a single location. We question the performance of training GANs using a spread dataset over a set of distributed machines, using a gossip approach shown to work on standard neural networks [1]. This performance is compared to the federated learning distributed method, that has the drawback of sending model data to a server. We also propose a gossip variant, where GAN components are gossiped independently. Experiments are conducted with Tensorflow with up to 100 emulated machines, on the canonical MNIST dataset. The position of this paper is to provide a first evidence that gossip performances for GAN training are close to the ones of federated learning, while operating in a fully decentralized setup. Second, to highlight that for GANs, the distribution of data on machines is critical (i.e., i.i.d. or not). Third, to illustrate that the gossip variant, despite proposing data diversity to the learning phase, brings only marginal improvements over the classic gossip approach.
Fichier non déposé

Dates et versions

hal-01923742 , version 1 (15-11-2018)

Identifiants

Citer

Corentin Hardy, Erwan Le Merrer, Bruno Sericola. Gossiping GANs. DIDL 2018 - Second Workshop on Distributed Infrastructures for Deep Learning, Dec 2018, Rennes, France. ⟨10.1145/3286490.3286563⟩. ⟨hal-01923742⟩
143 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More