Automatic Repair of Real Bugs in Java: A Large-Scale Experiment on the Defects4J Dataset - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2017

Automatic Repair of Real Bugs in Java: A Large-Scale Experiment on the Defects4J Dataset

Abstract

Defects4J is a large, peer-reviewed, structured dataset of real-world Java bugs. Each bug in Defects4J comes with a test suite and at least one failing test case that triggers the bug. In this paper, we report on an experiment to explore the effectiveness of automatic test-suite based repair on Defects4J. The result of our experiment shows that the considered state-of-the-art repair methods can generate patches for 47 out of 224 bugs. However, those patches are only test-suite adequate, which means that they pass the test suite and may potentially be incorrect beyond the test-suite satisfaction correctness criterion. We have manually analyzed 84 different patches to assess their real correctness. In total, 9 real Java bugs can be correctly repaired with test-suite based repair. This analysis shows that test-suite based repair suffers from under-specified bugs, for which trivial or incorrect patches still pass the test suite. With respect to practical applicability, it takes on average 14.8 minutes to find a patch. The experiment was done on a scientific grid, totaling 17.6 days of computation time. All the repair systems and experimental results are publicly available on Github in order to facilitate future research on automatic repair.
automatic-repair-defects4j.pdf (854.21 Ko) Télécharger le fichier
framework.pdf (82.78 Ko) Télécharger le fichier
intersection.pdf (23 Ko) Télécharger le fichier
patch_correctness.pdf (78.47 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Origin : Files produced by the author(s)
Origin : Files produced by the author(s)
Origin : Files produced by the author(s)

Dates and versions

hal-01574671 , version 1 (02-11-2018)

Identifiers

Cite

Matias Martinez, Thomas Durieux, Romain Sommerard, Jifeng Xuan, Martin Monperrus. Automatic Repair of Real Bugs in Java: A Large-Scale Experiment on the Defects4J Dataset. ESEC/FSE 2017 - 11th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, Sep 2017, Paderborn, Germany. pp.1936-1964, ⟨10.1007/s10664-016-9470-4⟩. ⟨hal-01574671⟩

Relations

566 View
268 Download

Altmetric

Share

Gmail Facebook X LinkedIn More