Supporting Dynamic Allocation of Heterogeneous Storage Resources on HPC Systems - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Concurrency and Computation: Practice and Experience Year : 2023

Supporting Dynamic Allocation of Heterogeneous Storage Resources on HPC Systems

Abstract

Scaling up large-scale scientific applications on supercomputing facilities is largely dependent on the ability to scale up efficiently data storage and retrieval. However, there is an ever-widening gap between I/O and computing performance. To address this gap, an increasingly popular approach consists in introducing new intermediate storage tiers (node-local storage, burst-buffers, ...) between the compute nodes and the traditional global shared parallel file-system. Unfortunately, without advanced techniques to allocate and size these resources, they remain underutilized. In this paper, we investigate how heterogeneous storage resources can be allocated on an HPC platform, just like compute resources. To this purpose, we introduce StorAlloc, a simulator used as a testbed for assessing storage-aware job scheduling algorithms and evaluating various storage infrastructures. We illustrate its usefullness by showing through a large series of experiments how this tool can be used to size a burst-buffer partition on a top-tier supercomputer by using the job history of a production year.
Fichier principal
Vignette du fichier
SupportingDynamicAllocationOfHeterogeneousStorageResources.pdf (1.47 Mo) Télécharger le fichier
Origin : Files produced by the author(s)
Licence : CC BY - Attribution

Dates and versions

hal-03922866 , version 1 (05-01-2023)
hal-03922866 , version 2 (13-11-2023)

Licence

Attribution

Identifiers

Cite

Julien Monniot, François Tessier, Matthieu Robert, Gabriel Antoniu. Supporting Dynamic Allocation of Heterogeneous Storage Resources on HPC Systems. Concurrency and Computation: Practice and Experience, 2023, Special Issue:S2 World 2020. CUG 2021 & 2022. PN_HCP. HeteroPar 2022, 35 (28), pp.1-16. ⟨10.1002/cpe.7890⟩. ⟨hal-03922866v2⟩
118 View
44 Download

Altmetric

Share

Gmail Facebook X LinkedIn More