Testing MapReduce-Based Systems - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2011

Testing MapReduce-Based Systems

Abstract

MapReduce (MR) is the most popular solution to build applications for large-scale data processing. These applications are often deployed on large clusters of commodity machines, where failures happen constantly due to bugs, hardware problems, and outages. Testing MR-based systems is hard, since it is needed a great eort of test harness to execute distributed test cases upon failures. In this paper, we present a novel testing solution to tackle this issue called HadoopTest. This solution is based on a scalable harness approach, where distributed tester components are hung around each map and reduce worker (i.e., node). Testers are allowed to stimulate each worker to inject failures on them, monitor their behavior, and validate testing results. HadoopTest was used to test two applications bundled into Hadoop, the Apache open source MapReduce implementation. Our initial implementation demonstrates promising results, with HadoopTest coordinating test cases across distributed MapReduce workers, and finding bugs.
No file

Dates and versions

hal-00728729 , version 1 (06-09-2012)

Identifiers

  • HAL Id : hal-00728729 , version 1

Cite

João Eugenio Marynowski, Michel Albonico, Eduardo Cunha de Almeida, Gerson Gerson Sunyé. Testing MapReduce-Based Systems. SBBD - XXVI Simpósio Brasileiro de Banco de Dados, Oct 2011, Florianopolis, SC, Brazil. ⟨hal-00728729⟩
139 View
1 Download

Share

Gmail Facebook X LinkedIn More