Reproduce, Replicate, Reevaluate. The Long but Safe Way to Extend Machine Learning Methods - Inria - Institut national de recherche en sciences et technologies du numérique
Conference Papers Year : 2024

Reproduce, Replicate, Reevaluate. The Long but Safe Way to Extend Machine Learning Methods

Luisa Sophie Werner
  • Function : Author
  • PersonId : 1089704
Nabil Layaïda
Pierre Genevès
Jérôme Euzenat
Damien Graux
  • Function : Author
  • PersonId : 1122264

Abstract

Reproducibility is a desirable property of scientific research. On the one hand, it increases confidence in results. On the other hand, reproducible results can be extended on a solid basis. In rapidly developing fields such as machine learning, the latter is particularly important to ensure the reliability of research. In this paper, we present a systematic approach to reproducing (using the available implementation), replicating (using an alternative implementation) and reevaluating (using different datasets) state-of-the-art experiments. This approach enables the early detection and correction of deficiencies and thus the development of more robust and transparent machine learning methods. We detail the independent reproduction, replication, and reevaluation of the initially published experiments with a method that we want to extend. For each step, we identify issues and draw lessons learned. We further discuss solutions that have proven effective in overcoming the encountered problems. This work can serve as a guide for further reproducibility studies and generally improve reproducibility in machine learning.
Fichier principal
Vignette du fichier
paper.pdf (198.3 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04035305 , version 1 (09-06-2022)
hal-04035305 , version 2 (17-03-2023)
hal-04035305 , version 3 (20-03-2023)
hal-04035305 , version 4 (13-12-2023)

Licence

Identifiers

  • HAL Id : hal-04035305 , version 4

Cite

Luisa Sophie Werner, Nabil Layaïda, Pierre Genevès, Jérôme Euzenat, Damien Graux. Reproduce, Replicate, Reevaluate. The Long but Safe Way to Extend Machine Learning Methods. AAAI 2024 - 38th Annual AAAI Conference on Artificial Intelligence, Feb 2024, Vancouver, Canada. pp.1-9. ⟨hal-04035305v4⟩
639 View
511 Download

Share

More