Performance and Energy Efficiency of Big Data Applications in Cloud Environments: A Hadoop Case Study
Résumé
The exponential growth of scientic and business data has resulted in the
evolution of the cloud computing environments and the MapReduce parallel
programming model. The focus of cloud computing is increased utilization
and power savings through consolidation while MapReduce enables large
scale data analysis. Hadoop, an open source implementation of MapReduce
has gained popularity in the last few years. In this paper, we evaluate Hadoop
performance in both the traditional model of collocated data and compute
services as well as consider the impact of separating out the services. The
separation of data and compute services provides more
exibility in environments where data locality might not have a considerable impact such as
virtualized environments and clusters with advanced networks. In this paper,
we also conduct an energy eciency evaluation of Hadoop on physical and
virtual clusters in dierent congurations. Our extensive evaluation shows
that: (1) coexisting virtual machines on servers decrease the disk throughput;
(2) performance on physical clusters is signicantly better than on virtual
clusters; (3) performance degradation due to separation of the services depends on the data to compute ratio; (4) application completion progress
correlates with the power consumption and power consumption is heavily
application specic. Finally, we present a discussion on the implications of
using cloud environments for big data analyses.