Checkpointing strategies to tolerate non-memoryless failures on HPC platforms - Inria - Institut national de recherche en sciences et technologies du numérique
Journal Articles ACM Transactions on Parallel Computing Year : 2024

Checkpointing strategies to tolerate non-memoryless failures on HPC platforms

Abstract

This paper studies checkpointing strategies for parallel applications subject to failures. The optimal strategy to minimize total execution time, or makespan, is well known when failure IATs obey an Exponential distribution, but it is unknown for non-memoryless failure distributions. We explain why the latter fact is misunderstood in recent literature. We propose a general strategy that maximizes the expected efficiency until the next failure, and we show that this strategy achieves an asymptotically optimal makespan, thereby establishing the first optimality result for arbitrary failure distributions. Through extensive simulations, we show that the new strategy is always at least as good as the Young/Daly strategy for various failure distributions. For distributions with a high infant mortality (such as LogNormal with shape parameter k = 2.51 or Weibull with shape parameter 0.5), the execution time is divided by a factor 1.9 on average, and up to a factor 4.2 for recently deployed platforms.
Fichier principal
Vignette du fichier
topc-with-supplemental.pdf (1.11 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04215283 , version 1 (22-09-2023)

Licence

Identifiers

Cite

Anne Benoit, Lucas Perotin, Yves Robert, Frédéric Vivien. Checkpointing strategies to tolerate non-memoryless failures on HPC platforms. ACM Transactions on Parallel Computing, 2024, 11 (1), pp.1-26. ⟨10.1145/3624560⟩. ⟨hal-04215283⟩
51 View
52 Download

Altmetric

Share

More