Caching for dataset-based workloads with heterogeneous file sizes - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2022

Caching for dataset-based workloads with heterogeneous file sizes

Olga Chuchuk
Giovanni Neglia
Markus Schulz
  • Function : Author
Dirk Duellmann
  • Function : Author


Caching can effectively reduce the cost of serving content and improve the user experience. In this paper, we explore the benefits of caching for existing scientific workloads, taking the Worldwide LHC (Large Hadron Collider) Computing Grid as an example. It is a globally distributed system that stores and processes multiple hundred petabytes of data and serves the needs of thousands of scientists around the globe. Scientific computation differs from other applications like video streaming as file sizes vary from a few bytes to terabytes and logical links between the files affect user access patterns. These factors profoundly influence caches' performance and, therefore, should be carefully analyzed to select which caching policy to deploy or to design new ones. In this work, we study how the hierarchical organization of the LHC physics data into files and groups of files called datasets affects the request patterns. We then propose new caching policies that exploit dataset-specific knowledge and compare them with file-based ones. Moreover, we show that limited connectivity between the computing and storage sites leads to the delayed hits phenomenon and estimate the consequent reduction in the potential benefits of caching.
Fichier principal
Vignette du fichier
5057007a965dcf58aeab033d23db2b695aa9.pdf (1.44 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03894955 , version 1 (12-12-2022)


  • HAL Id : hal-03894955 , version 1


Olga Chuchuk, Giovanni Neglia, Markus Schulz, Dirk Duellmann. Caching for dataset-based workloads with heterogeneous file sizes. ISGC 2022 - International Symposium on Grids & Clouds 2022, Mar 2022, Taipei, Taiwan. ⟨hal-03894955⟩
13 View
13 Download


Gmail Facebook Twitter LinkedIn More