FLEX: A Slot Allocation Scheduling Optimizer for MapReduce Workloads - Inria - Institut national de recherche en sciences et technologies du numérique
Communication Dans Un Congrès Année : 2010

FLEX: A Slot Allocation Scheduling Optimizer for MapReduce Workloads

Résumé

Originally, MapReduce implementations such as Hadoop employed First In First Out (fifo) scheduling, but such simple schemes cause job starvation. The Hadoop Fair Scheduler (hfs) is a slot-based MapReduce scheme designed to ensure a degree of fairness among the jobs, by guaranteeing each job at least some minimum number of allocated slots. Our prime contribution in this paper is a different, flexible scheduling allocation scheme, known as flex. Our goal is to optimize any of a variety of standard scheduling theory metrics (response time, stretch, makespan and Service Level Agreements (slas), among others) while ensuring the same minimum job slot guarantees as in hfs, and maximum job slot guarantees as well. The flex allocation scheduler can be regarded as an add-on module that works synergistically with hfs. We describe the mathematical basis for flex, and compare it with fifo and hfs in a variety of experiments.
Fichier principal
Vignette du fichier
middleware10-cr.pdf (414.88 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01055274 , version 1 (12-08-2014)

Licence

Identifiants

Citer

Joel Wolf, Deepak Rajan, Kirsten Hildrum, Rohit Khandekar, Vibhore Kumar, et al.. FLEX: A Slot Allocation Scheduling Optimizer for MapReduce Workloads. ACM/IFIP/USENIX 11th International Middleware Conference (MIDDLEWARE), Nov 2010, Bangalore, India. pp.1-20, ⟨10.1007/978-3-642-16955-7_1⟩. ⟨hal-01055274⟩
182 Consultations
569 Téléchargements

Altmetric

Partager

More