Improving Batch Schedulers with Node Stealing for Failed Jobs - Inria - Institut national de recherche en sciences et technologies du numérique
Article Dans Une Revue Concurrency and Computation: Practice and Experience Année : 2024

Improving Batch Schedulers with Node Stealing for Failed Jobs

Résumé

After a machine failure, batch schedulers typically re-schedule the job that failed with a high priority. This is fair for the failed job but still requires that job to re-enter the submission queue and to wait for enough resources to become available. The waiting time can be very long when the job is large and the platform highly loaded, as is the case with typical HPC platforms. We propose another strategy: when a job J fails, if no platform node is available, we steal one node from another job J', and use it to continue the execution of J despite the failure. In this work, we give a detailed assessment of this node stealing strategy using traces from the Mira supercomputer at Argonne National Laboratory. The main conclusion is that node stealing improves the utilization of the platform and dramatically reduces the flow of large jobs, at the price of slightly increasing the flow of small jobs.
Fichier principal
Vignette du fichier
revised-ccpe.pdf (458.25 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03643403 , version 1 (15-04-2022)
hal-03643403 , version 2 (02-03-2024)

Licence

Identifiants

Citer

Yishu Du, Loris Marchal, Guillaume Pallez, Yves Robert. Improving Batch Schedulers with Node Stealing for Failed Jobs. Concurrency and Computation: Practice and Experience, 2024, 36 (12), pp.1-36. ⟨10.1002/cpe.8043⟩. ⟨hal-03643403v2⟩
214 Consultations
225 Téléchargements

Altmetric

Partager

More