Thoughts and Recommendations from the ACM SIGCOMM 2017 Reproducibility Workshop
Résumé
Ensuring the reproducibility of results is an essential part of experimental
sciences, including computer networking. Unfortunately, as highlighted
recently, a large portion of research results are hardly, if not at all,
reproducible, raising reasonable lack of conviction on the research carried
out around the world.
Recent years have shown an increasing awareness about reproducibility of
results as an essential part of research carried out by members of the ACM
SIGCOMM community. To address this important issue, ACM has introduced a new
policy on results and artifacts review and badging. The policy defines the
terminology to be used to assess results and artifacts but does not specify the
review process or how to make research reproducible.
During SIGCOMM'17 a side workshop has been organized with the specific purpose
to tackle this issue. The objective being to trigger discussion and activity in
order to craft recommendations on how to introduce incentives for authors to
share their artifacts, and the details on how to use them, as well as defining
the process to be used.
This editorial overviews the workshop activity and summarizes the main
discussions and outcomes.