Pufferscale: Rescaling HPC Data Services for High Energy Physics Applications - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2020

Pufferscale: Rescaling HPC Data Services for High Energy Physics Applications

Abstract

User-space HPC data services are emerging as an appealing alternative to traditional parallel file systems, because of their ability to be tailored to application needs while eliminating unnecessary overheads incurred by POSIX compliance. The High Energy Physics (HEP) community is progressively turning towards such services to enable high-throughput accesses under heavy concurrency to billions of event data produced by instruments and consumed by subsequent analysis workflows. Such services would benefit from the possibility to be rescaled up and down to adapt to changing workloads, as experimental campaigns progress , in order to optimize resource usage. This paper formalizes rescaling a distributed storage system as a multi objective optimization problem considering three criteria: load balance, data balance, and duration of the rescaling operation. We propose a heuristic for rapidly finding a good approximate solution, while allowing users to weight the criteria as needed. The heuristic is evaluated with Pufferscale, a new rescaling manager for microservice-based distributed storage systems. To validate our approach in a real-world ecosystem, we showcase the use of Pufferscale as a means to enable storage malleability in the HEPnOS storage system for HEP applications.
Fichier principal
Vignette du fichier
CCGrid_2020_paper_140.pdf (298.92 Ko) Télécharger le fichier
Origin Files produced by the author(s)
Loading...

Dates and versions

hal-02961907 , version 1 (08-10-2020)

Identifiers

Cite

Nathanael Cheriere, Matthieu Dorier, Gabriel Antoniu, Stefan M. Wild, Sven Leyffer, et al.. Pufferscale: Rescaling HPC Data Services for High Energy Physics Applications. CCGRID 2020 - 20th IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing, May 2020, Melbourne, Australia. pp.182-191, ⟨10.1109/CCGrid49817.2020.00-75⟩. ⟨hal-02961907⟩
98 View
219 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More