Modeling Allocation of Heterogeneous Storage Resources on HPC Systems - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Poster Communications Year : 2022

Modeling Allocation of Heterogeneous Storage Resources on HPC Systems

Modélisation d'allocation de ressources de stockage hétérogènes sur plateformes HPC

Julien Monniot
Gabriel Antoniu

Abstract

In recent years, and despite remarkable progress in computing and network performance, HPC platforms have struggled to maintain satisfactory I/O throughput. Various solutions have been proposed to mitigate the contention and variability experienced by more and more concurrent applications, particularly on heavily shared parallel file systems. As a result, many large scale platforms now offer complex hierarchies of storage resources using various architectures (node-local, burst buffers, network-attached storage and so on) based on diversified hardware technologies such as persistent memories or flash for instance. In that context, we propose to study how to efficiently allocate these heterogeneous storage resources for scientific applications and workflows. For that purpose, we developed StorAlloc, a modular and extensible simulator of a storage-aware job-scheduler. We ran a large set of experiments with StorAlloc to investigate storage system designs and resource scheduling algorithms, through the analysis of multiple storagerelated metrics. For example, we have been able to determine an estimate of the size of a burst buffers partition sufficient to capture the intensive I/O of a top-tier supercomputer.
Fichier principal
Vignette du fichier
rpost103s3.pdf (4.68 Mo) Télécharger le fichier
extended_abstract.pdf (79.49 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Origin : Files produced by the author(s)

Dates and versions

hal-03878252 , version 1 (29-11-2022)

Licence

Attribution

Identifiers

  • HAL Id : hal-03878252 , version 1

Cite

Julien Monniot, François Tessier, Gabriel Antoniu. Modeling Allocation of Heterogeneous Storage Resources on HPC Systems. SC 2022 - International Conference for High Performance Computing, Networking, Storage, and Analysis (Posters), Nov 2022, Dallas, United States. , pp.1-1. ⟨hal-03878252⟩
52 View
62 Download

Share

Gmail Facebook X LinkedIn More