[Re] Modeling habits as self-sustained patterns of sensorimotor behavior - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Article Dans Une Revue The ReScience journal Année : 2022

[Re] Modeling habits as self-sustained patterns of sensorimotor behavior

Résumé

Scope of Reproducibility In this work, we study the reproducibility of the paper Counterfactual Generative Networks (CGN) by Sauer and Geiger to verify their main claims, which state that (i) their proposed model can reliably generate high-quality counterfactual images by disentangling the shape, texture and background of the image into independent mechanisms, (ii) each independent mechanism has to be considered, and jointly optimizing all of them endto-end is needed for high-quality images, and (iii) despite being synthetic, these counterfactual images can improve out-of-distribution performance of classifiers by making them invariant to spurious signals. Methodology The authors of the paper provide the implementation of CGN training in PyTorch. However, they did not provide code for all experiments. Consequently, we re-implement the code for most experiments, and run each experiment on 1080 Ti GPUs. Our reproducibility study comes at a total computational cost of 112 GPU hours. Results We find that the main claims of the paper of (i) generating high-quality counterfactuals, (ii) utilizing appropriate inductive biases, and (iii) using them to instil invariance in classifiers, do largely hold. However, we found certain experiments that were not directly reproducible due to either inconsistency between the paper and code, or incomplete specification of the necessary hyperparameters. Further, we were unable to reproduce a subset of experiments on a large-scale dataset due to resource constraints, for which we compensate by performing those on a smaller version of the same dataset with our results supporting the general performance trend.

Domaines

Informatique
Fichier principal
Vignette du fichier
article-3.pdf (3.34 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03725504 , version 1 (17-07-2022)

Identifiants

Citer

Tristan Gillard, Jérémy Fix, Alain Dutech. [Re] Modeling habits as self-sustained patterns of sensorimotor behavior. The ReScience journal, 2022, ⟨10.5281/zenodo.6478462⟩. ⟨hal-03725504⟩
48 Consultations
22 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More