Rotten Green Tests in Java, Pharo and Python - Inria - Institut national de recherche en sciences et technologies du numérique
Article Dans Une Revue Empirical Software Engineering Année : 2021

Rotten Green Tests in Java, Pharo and Python

Résumé

Rotten Green Tests are tests that pass, but not because the assertions they contain are true: a rotten test passes because some or all of its assertions are not actually executed. The presence of a rotten green test is a test smell, and a bad one, because the existence of a test gives us false confidence that the code under test is valid, when in fact that code may not have been tested at all. This article reports on an empirical evaluation of the tests in a corpus of projects found in the wild. We selected approximately one hundred mature projects written in each of Java, Pharo, and Python. We looked for rotten green tests in each project, taking into account test helper methods, inherited helpers, and trait composition. Previous work has shown the presence of rotten green tests in Pharo projects; the results reported here show that they are also present in Java and Python projects, and that they fall into similar categories. Furthermore, we found code bugs that were hidden by rotten tests in Pharo and Python. We also discuss two test smells —missed fail and missed skip —that arise from the misuse of testing frameworks, and which we observed in tests written in all three languages.
Fichier principal
Vignette du fichier
Aran21a-RottenGreenTest-EmpiricalSoftwareEngineering.pdf (563.12 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03281836 , version 1 (08-07-2021)
hal-03281836 , version 2 (04-10-2021)

Identifiants

Citer

Vincent Aranega, Julien Delplanque, Matias Martinez, Andrew P Black, Stéphane Ducasse, et al.. Rotten Green Tests in Java, Pharo and Python. Empirical Software Engineering, 2021, 26 (6), ⟨10.1007/s10664-021-10016-2⟩. ⟨hal-03281836v2⟩
300 Consultations
516 Téléchargements

Altmetric

Partager

More