Evaluation of Data Locality Strategies for Hybrid Cloud Bursting of Iterative MapReduce - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year :

Evaluation of Data Locality Strategies for Hybrid Cloud Bursting of Iterative MapReduce

Abstract

Hybrid cloud bursting (i.e., leasing temporary off-premise cloud resources to boost the overall capacity during peak utilization) is a popular and cost-effective way to deal with the increasing complexity of big data analytics. It is particularly promising for iterative MapReduce applications that reuse massive amounts of input data at each iteration, which compensates for the high overhead and cost of concurrent data transfers from the on-premise to the off-premise VMs over a weak inter-site link that is of limited capacity. In this paper we study how to combine various MapReduce data locality techniques designed for hybrid cloud bursting in order to achieve scalability for iterative MapReduce applications in a cost-effective fashion. This is a non-trivial problem due to the complex interaction between the data movements over the weak link and the scheduling of computational tasks that have to adapt to the shifting data distribution. We show that using the right combination of techniques, iterative MapReduce applications can scale well in a hybrid cloud bursting scenario and come even close to the scalability observed in single sites.
Fichier principal
Vignette du fichier
paper.pdf (169.22 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01469991 , version 1 (17-02-2017)

Identifiers

  • HAL Id : hal-01469991 , version 1

Cite

Francisco Clemente-Castello, Bogdan Nicolae, M. Mustafa Rafique, Rafael Mayo, Juan Carlos Fernandez. Evaluation of Data Locality Strategies for Hybrid Cloud Bursting of Iterative MapReduce. CCGrid’17: 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, May 2017, Madrid, Spain. ⟨hal-01469991⟩
74 View
283 Download

Share

Gmail Facebook Twitter LinkedIn More