Bootstrap Your Own Latent: A new approach to self-supervised learning - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

Bootstrap Your Own Latent: A new approach to self-supervised learning

Jean-Bastien Grill
  • Fonction : Auteur
  • PersonId : 972490
Florian Strub
Florent Altché
  • Fonction : Auteur
  • PersonId : 1027053
Corentin Tallec
  • Fonction : Auteur
Pierre H Richemond
  • Fonction : Auteur
  • PersonId : 1073235
Carl Doersch
  • Fonction : Auteur
Bilal Piot
  • Fonction : Auteur
Rémi Munos
  • Fonction : Auteur
  • PersonId : 836863
Michal Valko

Résumé

We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other. From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view. At the same time, we update the target network with a slow-moving average of the online network. While state-of-the art methods rely on negative pairs, BYOL achieves a new state of the art without them. BYOL reaches 74.3% top-1 classification accuracy on ImageNet using a linear evaluation with a ResNet-50 architecture and 79.6% with a larger ResNet. We show that BYOL performs on par or better than the current state of the art on both transfer and semi-supervised benchmarks. Our implementation and pretrained models are given on GitHub.
Fichier principal
Vignette du fichier
grill2020bootstrap.pdf (667.55 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02869787 , version 1 (16-06-2020)
hal-02869787 , version 2 (25-02-2021)

Identifiants

Citer

Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H Richemond, et al.. Bootstrap Your Own Latent: A new approach to self-supervised learning. Neural Information Processing Systems, 2020, Montréal, Canada. ⟨hal-02869787v1⟩
648 Consultations
891 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More