Granularity-Aware Adaptation for Image Retrieval Over Multiple Tasks - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Chapitre D'ouvrage Année : 2022

Granularity-Aware Adaptation for Image Retrieval Over Multiple Tasks

Résumé

Strong image search models can be learned for a specific domain, i.e. set of labels, provided that some labeled images of that domain are available. A practical visual search model, however, should be versatile enough to solve multiple retrieval tasks simultaneously, even if those cover very different specialized domains. Additionally, it should be able to benefit from even unlabeled images from these various retrieval tasks. This is the more practical scenario that we consider in this paper. We address it with the proposed , an approach that starts from a strong pretrained model, and adapts it to tackle multiple retrieval tasks concurrently, using only unlabeled images from the different task domains. We extend the pretrained model with multiple independently trained sets of adaptors that use pseudo-label sets of different sizes, effectively mimicking different pseudo-granularities. We reconcile all adaptor sets into a single unified model suited for all retrieval tasks by learning fusion layers that we guide by propagating pseudo-granularity attentions across neighbors in the feature space. Results on a benchmark composed of six heterogeneous retrieval tasks show that the unsupervised model improves the zero-shot performance of a state-of-the-art self-supervised learning model, and in some places reaches or improves over a task label-aware oracle that selects the most fitting pseudo-granularity per task.

Dates et versions

hal-04041564 , version 1 (22-03-2023)

Identifiants

Citer

Jon Almazán, Byungsoo Ko, Geonmo Gu, Diane Larlus, Yannis Kalantidis. Granularity-Aware Adaptation for Image Retrieval Over Multiple Tasks. Computer Vision – ECCV 2022. Lecture Notes in Computer Science, 13674, Springer Nature, pp.389-406, 2022, 978-3-031-19780-2. ⟨10.1007/978-3-031-19781-9_23⟩. ⟨hal-04041564⟩

Collections

MIAI ANR
4 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More