Challenge to test reproducibility of old computer code - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Nature Year : 2019

Challenge to test reproducibility of old computer code

Nicolas P. Rougier
Konrad Hinsen


We question whether analytical tools such as Common Workflow Language, which aim to make computational methods “reproducible and shareable”, can stand the test of time (see Nature 573, 149–150; 2019). The long-term validity of computational results will not be testable if the original code cannot be run many years later. Considering the rapidity of transformations in operating systems and programming languages, it is hard to predict the lifetime reproducibility of a particular code. We have therefore organized the Ten Years Reproducibility Challenge (see Researchers are invited to test code reproducibility by trying to rerun a code created for a scientific paper they published more than ten years ago. The codes can address any scientific domain (statistical analysis, numerical simulation or data processing, for example) and be written in any language.The challenge closes in April 2020. Our hope is that the results will offer insights into long-term causes of non-reproducibility.
Fichier principal
Vignette du fichier
2019 - Putting reproducibility to the test.pdf (64.14 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-02340398 , version 1 (06-11-2019)



Nicolas P. Rougier, Konrad Hinsen. Challenge to test reproducibility of old computer code. Nature, 2019, 574 (7780), pp.634. ⟨10.1038/d41586-019-03296-8⟩. ⟨hal-02340398⟩
194 View
96 Download



Gmail Facebook X LinkedIn More