What Does It Take to Reproduce Experiments? Evidences from the Neuro-Symbolic Domain - Inria - Institut national de recherche en sciences et technologies du numérique
Preprints, Working Papers, ... Year : 2023

What Does It Take to Reproduce Experiments? Evidences from the Neuro-Symbolic Domain

Abstract

Reproducibility is a desirable property of scientific experiments, which is gaining relevance in computer science. Although guidelines exist, it is not immediately obvious what is necessary for making an experiment reproducible and what is necessary for reproducing it. In this work, we address the task of independently reproducing results published in the neuro-symbolic domain. We describe the methodology used for reproducing (using the available code) and replicating (using an alternative implementation) the experiments conducted with Knowledge Enhanced Neural Networks. We also extend them by applying the KENN to additional datasets. For each of these steps, we identify issues that may arise. This work shows that reproducibility should not be taken for granted. We discuss solutions that have proven effective in overcoming the encountered problems. This can be used as a guide for further reproducibility studies and generally improve reproducibility in machine learning.
Fichier principal
Vignette du fichier
nsrep.pdf (363.01 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04035305 , version 1 (09-06-2022)
hal-04035305 , version 2 (17-03-2023)
hal-04035305 , version 3 (20-03-2023)
hal-04035305 , version 4 (13-12-2023)

Identifiers

  • HAL Id : hal-04035305 , version 2

Cite

Luisa Sophie Werner, Nabil Layaïda, Pierre Genevès, Jérôme Euzenat. What Does It Take to Reproduce Experiments? Evidences from the Neuro-Symbolic Domain. 2023. ⟨hal-04035305v2⟩
639 View
511 Download

Share

More