Towards Smarter Schedulers: Molding Jobs into the Right Shape via Monitoring and Modeling - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

Towards Smarter Schedulers: Molding Jobs into the Right Shape via Monitoring and Modeling

Abstract

High-performance computing is not only a race towards the fastest supercomputers but also the science of using such massive machines productively to acquire valuable results-outlining the importance of performance modelling and optimization. However, it appears that more than punctual optimization is required for current architectures, with users having to choose between multiple intertwined parallelism possibilities, dedicated accelerators, and I/O solutions. Witnessing this challenging context, our paper establishes an automatic feedback loop between how applications run and how they are launched, with a specific focus on I/O. One goal is to optimize how applications are launched through moldability (launch-time malleability). As a first step in this direction, we propose a new, always-on measurement infrastructure based on state-of-the-art cloud technologies adapted for HPC. In this paper, we present the measurement infrastructure and associated design choices. Moreover, we leverage an existing performance modelling tool to generate I/O performance models. We outline sample modelling capabilities, as derived from our measurement chain showing the critical importance of the measurement in future HPC systems, especially concerning resource configurations. Thanks to this precise performance model infrastructure, we can improve moldability and malleability on HPC systems.
Embargoed file
Embargoed file
0 0 27
Year Month Jours
Avant la publication
Friday, May 10, 2024
Embargoed file
Friday, May 10, 2024
Please log in to request access to the document

Dates and versions

hal-04093528 , version 1 (10-05-2023)

Licence

Attribution

Identifiers

  • HAL Id : hal-04093528 , version 1

Cite

Jean-Baptiste Besnard, Ahmad Tarraf, Clément Barthélemy, Alberto Cascajo, Emmanuel Jeannot, et al.. Towards Smarter Schedulers: Molding Jobs into the Right Shape via Monitoring and Modeling. HPCMALL 2023 - 2nd International Workshop on Malleability Techniques Applications in High-Performance Computing, May 2023, Hamburg, Germany. ⟨hal-04093528⟩

Collections

CNRS INRIA INRIA2
91 View
25 Download

Share

Gmail Facebook X LinkedIn More