Towards Smarter Schedulers: Molding Jobs into the Right Shape via Monitoring and Modeling - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Towards Smarter Schedulers: Molding Jobs into the Right Shape via Monitoring and Modeling

Résumé

High-performance computing is not only a race towards the fastest supercomputers but also the science of using such massive machines productively to acquire valuable results-outlining the importance of performance modelling and optimization. However, it appears that more than punctual optimization is required for current architectures, with users having to choose between multiple intertwined parallelism possibilities, dedicated accelerators, and I/O solutions. Witnessing this challenging context, our paper establishes an automatic feedback loop between how applications run and how they are launched, with a specific focus on I/O. One goal is to optimize how applications are launched through moldability (launch-time malleability). As a first step in this direction, we propose a new, always-on measurement infrastructure based on state-of-the-art cloud technologies adapted for HPC. In this paper, we present the measurement infrastructure and associated design choices. Moreover, we leverage an existing performance modelling tool to generate I/O performance models. We outline sample modelling capabilities, as derived from our measurement chain showing the critical importance of the measurement in future HPC systems, especially concerning resource configurations. Thanks to this precise performance model infrastructure, we can improve moldability and malleability on HPC systems.
Fichier sous embargo
Fichier sous embargo
0 0 6
Année Mois Jours
Avant la publication
vendredi 10 mai 2024
Fichier sous embargo
vendredi 10 mai 2024
Connectez-vous pour demander l'accès au fichier

Dates et versions

hal-04093528 , version 1 (10-05-2023)

Licence

Paternité

Identifiants

  • HAL Id : hal-04093528 , version 1

Citer

Jean-Baptiste Besnard, Ahmad Tarraf, Clément Barthélemy, Alberto Cascajo, Emmanuel Jeannot, et al.. Towards Smarter Schedulers: Molding Jobs into the Right Shape via Monitoring and Modeling. HPCMALL 2023 - 2nd International Workshop on Malleability Techniques Applications in High-Performance Computing, May 2023, Hamburg, Germany. ⟨hal-04093528⟩

Collections

CNRS INRIA INRIA2
98 Consultations
25 Téléchargements

Partager

Gmail Facebook X LinkedIn More