PØDA: Prompt-driven Zero-shot Domain Adaptation - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

PØDA: Prompt-driven Zero-shot Domain Adaptation

Abstract

Domain adaptation has been vastly investigated in computer vision but still requires access to target images at train time, which might be intractable in some conditions, especially for long-tail samples. In this paper, we propose the task of `Prompt-driven Zero-shot Domain Adaptation', where we adapt a model trained on a source domain using only a general textual description of the target domain, i.e., a prompt. First, we leverage a pretrained contrastive vision-language model (CLIP) to optimize affine transformations of source features, bringing them closer to target text embeddings, while preserving their content and semantics. Second, we show that augmented features can be used to perform zero-shot domain adaptation for semantic segmentation. Experiments demonstrate that our method significantly outperforms CLIP-based style transfer baselines on several datasets for the downstream task at hand. Our prompt-driven approach even outperforms one-shot unsupervised domain adaptation on some datasets, and gives comparable results on others. The code is available at https://github.com/astra-vision/PODA.

Dates and versions

hal-03945337 , version 1 (18-01-2023)

Licence

Identifiers

Cite

Mohammad Fahes, Tuan-Hung Vu, Andrei Bursuc, Patrick Pérez, Raoul de Charette. PØDA: Prompt-driven Zero-shot Domain Adaptation. International Conference on Computer Vision (ICCV), Oct 2023, Paris, France. ⟨hal-03945337⟩

Collections

INRIA INRIA2 ANR
124 View
0 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More