VICRegL: Self-Supervised Learning of Local Visual Features - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2022

VICRegL: Self-Supervised Learning of Local Visual Features

Abstract

Most recent self-supervised methods for learning image representations focus on either producing a global feature with invariance properties, or producing a set of local features. The former works best for classification tasks while the latter is best for detection and segmentation tasks. This paper explores the fundamental tradeoff between learning local and global features. A new method called VICRegL is proposed that learns good global and local features simultaneously, yielding excellent performance on detection and segmentation tasks while maintaining good performance on classification tasks. Concretely, two identical branches of a standard convolutional net architecture are fed two differently distorted versions of the same image. The VICReg criterion is applied to pairs of global feature vectors. Simultaneously, the VICReg criterion is applied to pairs of local feature vectors occurring before the last pooling layer. Two local feature vectors are attracted to each other if their l2-distance is below a threshold or if their relative locations are consistent with a known geometric transformation between the two input images. We demonstrate strong performance on linear classification and segmentation transfer tasks. Code and pretrained models are publicly available at: https://github.com/facebookresearch/VICRegL
Fichier principal
Vignette du fichier
vicregl_hal.pdf (12.11 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03893126 , version 1 (10-12-2022)

Identifiers

Cite

Adrien Bardes, Jean Ponce, Yann Lecun. VICRegL: Self-Supervised Learning of Local Visual Features. 36th Conference on Neural Information Processing Systems (NeurIPS 2022), Nov 2022, New Orleans, United States. ⟨hal-03893126⟩
44 View
49 Download

Altmetric

Share

Gmail Facebook X LinkedIn More