Have Foundational Models Seen Satellite Images? - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

Have Foundational Models Seen Satellite Images?

Abstract

This paper presents an investigation into the zero-shot performance of pre-trained foundation models on remote sensing tasks. Recent advances in self-supervised learning suggest that these models, when trained on vast amounts of unsupervised data, could potentially improve generalization across a number of downstream tasks. Our study offers an empirical evaluation of these models on standard remote-sensing benchmarks such as EuroSAT and BigEarthNet-S2, with the intent to confirm whether these models have encountered satellite imagery during their training phase. Moreover, we examine the impact of adding a geospatial domain-specific textual description of classes, contrasting it with the standard class-based prompts. Our findings indicate that the fine-tuned BLIP models exhibit superior zeroshot performance on these benchmarks compared to their standard counterparts, signifying that fine-tuning on standard benchmarks enhances performance. Furthermore, the addition of geospatial context variably influences performance depending on the specific model and dataset. This work provides crucial insights into the applicability of foundation models in remote sensing tasks and lays the groundwork for further research.
Fichier principal
Vignette du fichier
IGARSS2023_ZeroShot.pdf (220.05 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04112634 , version 1 (31-05-2023)

Licence

Attribution

Identifiers

  • HAL Id : hal-04112634 , version 1

Cite

Akash Panigrahi, Sagar Verma, Matthieu Terris, Maria Vakalopoulou. Have Foundational Models Seen Satellite Images?. IGARSS 2023 - International Geoscience and Remote Sensing Symposium, IEEE, Jul 2023, Pasadena, United States. ⟨hal-04112634⟩
254 View
417 Download

Share

Gmail Facebook X LinkedIn More