Masking Strategies for Background Bias Removal in Computer Vision Models - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

Masking Strategies for Background Bias Removal in Computer Vision Models


Models for fine-grained image classification tasks, where the difference between some classes can be extremely subtle and the number of samples per class tends to be low, are particularly prone to picking up background-related biases and demand robust methods to handle potential examples with out-of-distribution (OOD) backgrounds. To gain deeper insights into this critical problem, our research investigates the impact of background-induced bias on finegrained image classification, evaluating standard backbone models such as Convolutional Neural Network (CNN) and Vision Transformers (ViT). We explore two masking strategies to mitigate background-induced bias: Early masking, which removes background information at the (input) image level, and late masking, which selectively masks high-level spatial features corresponding to the background. Extensive experiments assess the behavior of CNN and ViT models under different masking strategies, with a focus on their generalization to OOD backgrounds. The obtained findings demonstrate that both proposed strategies enhance OOD performance compared to the baseline models, with early masking consistently exhibiting the best OOD performance. Notably, a ViT variant employing GAP-Pooled Patch tokenbased classification combined with early masking achieves the highest OOD robustness. Our code and models are available at:
Fichier principal
Vignette du fichier
ICCV_camera-ready.pdf (1.33 Mo) Télécharger le fichier
ICCVW 2023-00020-supp.pdf (131.01 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04184449 , version 1 (21-08-2023)





Ananthu Aniraj, Cassio F. Dantas, Dino Ienco, Diego Marcos. Masking Strategies for Background Bias Removal in Computer Vision Models. Out of Distribution Generalization in Computer Vision workshop (in conjunction with ICCV 2023), Oct 2023, Paris, France. ⟨10.1109/ICCVW60793.2023.00474⟩. ⟨hal-04184449⟩
123 View
92 Download



Gmail Facebook X LinkedIn More