Governing Energy Consumption in Hadoop through CPU Frequency Scaling: an Analysis - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Future Generation Computer Systems Year : 2016

Governing Energy Consumption in Hadoop through CPU Frequency Scaling: an Analysis

Abstract

With increasingly inexpensive storage and growing processing power, the cloud has rapidly become the environment of choice to store and analyze data for a variety of applications. Most large-scale data computations in the cloud heavily rely on the MapReduce paradigm and on its Hadoop implementation. Nevertheless, this exponential growth in popularity has significantly impacted power consumption in cloud infrastructures. In this paper, we focus on MapReduce processing and we investigate the impact of dynamically scaling the frequency of compute nodes on the performance and energy consumption of a Hadoop cluster. To this end, a series of experiments are conducted to explore the implications of Dynamic Voltage and Frequency Scaling (DVFS) settings on power consumption in Hadoop clusters. By enabling various existing DVFS governors (i.e., performance, powersave, ondemand, conservative and userspace) in a Hadoop cluster, we observe significant variation in performance and power consumption across different applications: the different DVFS settings are only sub-optimal for several representative MapReduce applications. Furthermore, our results reveal that the current CPU governors do not exactly reflect their design goal and may even become ineffective to manage the power consumption in Hadoop clusters. This study aims at providing a clearer understanding of the interplay between performance and power management in Hadoop clusters and therefore offers useful insight into designing power-aware techniques for Hadoop systems.
Fichier principal
Vignette du fichier
fgcs-ibrahim-HAL.pdf (7.42 Mo) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01166252 , version 1 (26-06-2015)

Identifiers

Cite

Shadi Ibrahim, Tien-Dat Phan, Alexandra Carpen-Amarie, Houssem-Eddine Chihoub, Diana Moise, et al.. Governing Energy Consumption in Hadoop through CPU Frequency Scaling: an Analysis. Future Generation Computer Systems, 2016, pp.14. ⟨10.1016/j.future.2015.01.005⟩. ⟨hal-01166252⟩
774 View
617 Download

Altmetric

Share

Gmail Facebook X LinkedIn More