Improving the Performance of Batch Schedulers Using Online Job Runtime Classification - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Journal of Parallel and Distributed Computing Year : 2022

Improving the Performance of Batch Schedulers Using Online Job Runtime Classification

Abstract

Job scheduling in high-performance computing platforms is a hard problem that involves uncertainties on both the job arrival process and their execution times. Users typically provide only loose upper bounds for job execution times, which are not so useful for scheduling heuristics based on processing times. Previous studies focused on applying regression techniques to obtain better execution time estimates, which worked reasonably well and improved scheduling metrics. However, these approaches require a long period of training data. In this work, we propose a simpler approach by classifying jobs as small or large and prioritizing the execution of small jobs over large ones. Indeed, small jobs are the most impacted by queuing delays, but they typically represent a light load and incur a small burden on the other jobs. The classifier operates online and learns by using data collected over the previous weeks, facilitating its deployment and enabling a fast adaptation to changes in the workload characteristics. We evaluate our approach using four scheduling policies on seven HPC platform workload traces. We show that: first, incorporating such classification reduces the average bounded slowdown of jobs in all scenarios, second, in most considered scenarios, the improvements are comparable to the ideal hypothetical situation where the scheduler would know in advance the exact running time of jobs.
Fichier principal
Vignette du fichier
main.pdf (4.22 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03023222 , version 1 (25-11-2020)
hal-03023222 , version 2 (28-02-2022)

Identifiers

Cite

Salah Zrigui, Raphael y de Camargo, Arnaud Legrand, Denis Trystram. Improving the Performance of Batch Schedulers Using Online Job Runtime Classification. Journal of Parallel and Distributed Computing, 2022, 164, pp.83-95. ⟨10.1016/j.jpdc.2022.01.003⟩. ⟨hal-03023222v2⟩
361 View
265 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More