Speculative Scheduling for Stochastic HPC Applications - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2019

Speculative Scheduling for Stochastic HPC Applications

Résumé

New emerging fields are developing a growing number of large-scale applications with heterogeneous, dynamic and data-intensive requirements that put a high emphasis on productivity and thus are not tuned to run efficiently on today's high performance computing (HPC) systems. Some of these applications, such as neuroscience workloads and those that use adaptive numerical algorithms, develop modeling and simulation workflows with stochastic execution times and unpredictable resource requirements. When they are deployed on current HPC systems using existing resource management solutions, it can result in loss of efficiency for the users and decrease in effective system utilization for the platform providers. In this paper, we consider the current HPC scheduling model and describe the challenge it poses for stochastic applications due to the strict requirement in its job deployment policies. To address the challenge, we present speculative scheduling techniques that adapt the resource requirements of a stochastic application on-the-fly, based on its past execution behavior instead of relying on estimates given by the user. We focus on improving the overall system utilization and application response time without disrupting the current HPC scheduling model or the application development process. Our solution can operate alongside existing HPC batch schedulers without interfering with their usage modes. We show that speculative scheduling can improve the system utilization and average application response time by 25-30% compared to the classical HPC approach.
Fichier principal
Vignette du fichier
icpp19_speculative.pdf (1.25 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02158598 , version 1 (18-06-2019)

Licence

Paternité

Identifiants

  • HAL Id : hal-02158598 , version 1

Citer

Ana Gainaru, Guillaume Pallez, Hongyang Sun, Padma Raghavan. Speculative Scheduling for Stochastic HPC Applications. ICPP 2019 - 48th International Conference on Parallel Processing, Aug 2019, Kyoto, Japan. ⟨hal-02158598⟩

Collections

CNRS INRIA INRIA2
115 Consultations
238 Téléchargements

Partager

Gmail Facebook X LinkedIn More