Transformer-Based Zero-Shot Detection via Contrastive Learning - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Transformer-Based Zero-Shot Detection via Contrastive Learning

Résumé

Zero-Shot Detection (ZSD) is a challenging computer vision problem that enables simultaneous classification and localization of previously unseen objects via auxiliary information. Most of the existing methods learn a biased visual-semantic mapping function, which prefers predicting seen classes during testing, and they only focus on region of interest and ignore contextual information in an image. To tackle these problems, we propose a novel framework for ZSD named Transformer-based Zero-Shot Detection via Contrastive Learning (TZSDC). The proposed TZSDC contains four components: transformer-based backbone, Foreground-Background (FB) separation module, Instance-Instance Contrastive Learning (IICL) module, and Knowledge-Transfer (KT) module. The transformer backbone encodes long-range contextual information with less inductive bias. The FB module separates foreground and background by scoring objectness from images. The IICL module optimizes the visual structure in embedding space to make it more discriminative and the KT module transfers knowledge from seen classes to unseen classes via category similarity. Benefiting from these modules, the accurate alignment between the contextual visual features and semantic features can be achieved. Experiments on MSCOCO well validate the effectiveness of the proposed method for ZSD and generalized ZSD.
Fichier sous embargo
Fichier sous embargo
0 5 16
Année Mois Jours
Avant la publication
mercredi 1 janvier 2025
Fichier sous embargo
mercredi 1 janvier 2025
Connectez-vous pour demander l'accès au fichier

Dates et versions

hal-04317188 , version 1 (01-12-2023)

Licence

Identifiants

Citer

Wei Liu, Hui Chen, Yongqiang Ma, Jianji Wang, Nanning Zheng. Transformer-Based Zero-Shot Detection via Contrastive Learning. 18th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2022, Hersonissos, Greece. pp.316-327, ⟨10.1007/978-3-031-08333-4_26⟩. ⟨hal-04317188⟩
26 Consultations
1 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More