Doing better for jobs that failed: node stealing from a batch scheduler's perspective - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Preprints, Working Papers, ... Year :

Doing better for jobs that failed: node stealing from a batch scheduler's perspective

Abstract

After a machine failure, batch schedulers typically reschedule the job that failed with a high priority. This is fair for the failed job but still requires that job to re-enter the submission queue and to wait for enough resources to become available. The waiting time can be very long when the job is large and the platform highly loaded, as is the case with typical HPC platforms. We propose another strategy: when a job J fails, if no platform node is available, we steal one node from another job J , and use it to continue the execution of J despite the failure. In this work, we give a detailed assessment of this node stealing strategy using traces from the Mira supercomputer at Argonne National Laboratory. The main conclusion is that node stealing improves the utilization of the platform and dramatically reduces the flow of large jobs, at the price of a slight increases the flow of small jobs.
Fichier principal
Vignette du fichier
hal-version.pdf (461.76 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03643403 , version 1 (15-04-2022)

Identifiers

  • HAL Id : hal-03643403 , version 1

Cite

Yishu Du, Loris Marchal, Guillaume Pallez, Yves Robert. Doing better for jobs that failed: node stealing from a batch scheduler's perspective. 2022. ⟨hal-03643403⟩
108 View
97 Download

Share

Gmail Facebook Twitter LinkedIn More