FLEX: A Slot Allocation Scheduling Optimizer for MapReduce Workloads - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2010

FLEX: A Slot Allocation Scheduling Optimizer for MapReduce Workloads


Originally, MapReduce implementations such as Hadoop employed First In First Out (fifo) scheduling, but such simple schemes cause job starvation. The Hadoop Fair Scheduler (hfs) is a slot-based MapReduce scheme designed to ensure a degree of fairness among the jobs, by guaranteeing each job at least some minimum number of allocated slots. Our prime contribution in this paper is a different, flexible scheduling allocation scheme, known as flex. Our goal is to optimize any of a variety of standard scheduling theory metrics (response time, stretch, makespan and Service Level Agreements (slas), among others) while ensuring the same minimum job slot guarantees as in hfs, and maximum job slot guarantees as well. The flex allocation scheduler can be regarded as an add-on module that works synergistically with hfs. We describe the mathematical basis for flex, and compare it with fifo and hfs in a variety of experiments.
Fichier principal
Vignette du fichier
middleware10-cr.pdf (1.04 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-01055274 , version 1 (12-08-2014)





Joel Wolf, Deepak Rajan, Kirsten Hildrum, Rohit Khandekar, Vibhore Kumar, et al.. FLEX: A Slot Allocation Scheduling Optimizer for MapReduce Workloads. ACM/IFIP/USENIX 11th International Middleware Conference (MIDDLEWARE), Nov 2010, Bangalore, India. pp.1-20, ⟨10.1007/978-3-642-16955-7_1⟩. ⟨hal-01055274⟩
146 View
530 Download



Gmail Facebook Twitter LinkedIn More