Towards Efficient I/O Pipelines using Accumulated Compression - Inria - Institut national de recherche en sciences et technologies du numérique
Communication Dans Un Congrès Année : 2023

Towards Efficient I/O Pipelines using Accumulated Compression

M Mustafa Rafique
Franck Cappello
  • Fonction : Auteur
  • PersonId : 1102088
Bogdan Nicolae

Résumé

High-Performance Computing (HPC) workloads generate large volumes of data at high-frequency during their execution, which needs to be captured concurrently at scale. These workloads exploit accelerators such as GPU for faster performance. However, the limited onboard high-bandwidth memory (HBM) on the GPU, and slow device-to-host memory PCIe interconnects lead to I/O overheads during application execution, thereby exacerbating their overall runtime. To overcome the aforementioned limitations, techniques such as compression and asynchronous transfers have been used by data management runtimes. However, compressing small blocks of data leads to a significant runtime penalty on the application. In this paper, we design and develop strategies to optimize the tradeoff between compressing checkpoints instantly and enqueuing transfers immediately versus accumulating snapshots and delaying compression to achieve faster compression throughput. Our evaluations on synthetic and real-life workloads for different systems and workload configurations demonstrate 1.3× to 8.3× speedup compared to the existing checkpoint approaches.
Fichier principal
Vignette du fichier
HiPC_23__Accumulated_checkpoint_compression.pdf (841.9 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04343670 , version 1 (14-12-2023)

Licence

Identifiants

  • HAL Id : hal-04343670 , version 1

Citer

Avinash Maurya, M Mustafa Rafique, Franck Cappello, Bogdan Nicolae. Towards Efficient I/O Pipelines using Accumulated Compression. HIPC’23: 30th IEEE International Conference on High Performance Computing, Data, and Analytics, Dec 2023, Goa, India. ⟨hal-04343670⟩
109 Consultations
143 Téléchargements

Partager

More