How does the degree of novelty impacts semi-supervised representation learning for novel class retrieval? - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

How does the degree of novelty impacts semi-supervised representation learning for novel class retrieval?

Quentin Leroy
  • Fonction : Auteur
  • PersonId : 1194255
Olivier Buisson
  • Fonction : Auteur
  • PersonId : 914124
Alexis Joly

Résumé

Supervised representation learning with deep networks tends to overfit the training classes and the generalization to novel classes is a challenging question. It is common to evaluate a learned embedding on held-out images of the same training classes. In real applications however, data comes from new sources and novel classes are likely to arise. We hypothesize that incorporating unlabelled images of novel classes in the training set in a semi-supervised fashion would be beneficial for the efficient retrieval of novel-class images compared to a vanilla supervised representation. To verify this hypothesis in a comprehensive way, we propose an original evaluation methodology that varies the degree of novelty of novel classes by partitioning the dataset category-wise either randomly, or semantically, i.e. by minimizing the shared semantics between base and novel classes. This evaluation procedure allows to train a representation blindly to any novel-class labels and evaluate the frozen representation on the retrieval of base or novel classes. We find that a vanilla supervised representation falls short on the retrieval of novel classes even more so when the semantics gap is higher. Semi-supervised algorithms allow to partially bridge this performance gap but there is still much room for improvement.
Fichier principal
Vignette du fichier
2208.08217.pdf (5.36 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03871584 , version 1 (17-10-2023)

Identifiants

Citer

Quentin Leroy, Olivier Buisson, Alexis Joly. How does the degree of novelty impacts semi-supervised representation learning for novel class retrieval?. 2023. ⟨hal-03871584⟩
33 Consultations
11 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More