ProvLight: Efficient Workflow Provenance Capture on the Edge-to-Cloud Continuum - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

ProvLight: Efficient Workflow Provenance Capture on the Edge-to-Cloud Continuum

Abstract

Modern scientific workflows require hybrid infrastructures combining numerous decentralized resources on the IoT/Edge interconnected to Cloud/HPC systems (aka the Computing Continuum) to enable their optimized execution. Understanding and optimizing the performance of such complex Edge-to-Cloud workflows is challenging. Capturing the provenance of key performance indicators, with their related data and processes, may assist in understanding and optimizing workflow executions. However, the capture overhead can be prohibitive, particularly in resource-constrained devices, such as the ones on the IoT/Edge. To address this challenge, based on a performance analysis of existing systems, we propose ProvLight, a tool to enable efficient provenance capture on the IoT/Edge. We leverage simplified data models, data compression and grouping, and lightweight transmission protocols to reduce overheads. We further integrate ProvLight into the E2Clab framework to enable workflow provenance capture across the Edge-to-Cloud Continuum. This integration makes E2Clab a promising platform for the performance optimization of applications through reproducible experiments. We validate ProvLight at a large scale with synthetic workloads on 64 real-life IoT/Edge devices in the FIT IoT LAB testbed. Evaluations show that ProvLight outperforms state-of-the-art systems like ProvLake and DfAnalyzer in resource-constrained devices. ProvLight is 26---37x faster to capture and transmit provenance data; uses 5---7x less CPU; 2x less memory; transmits 2x less data; and consumes 2---2.5x less energy. ProvLight and E2Clab are available as open-source tools.
Fichier principal
Vignette du fichier
0_main.pdf (1.5 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04161546 , version 1 (18-07-2023)

Licence

Attribution

Identifiers

  • HAL Id : hal-04161546 , version 1

Cite

Daniel Rosendo, Marta Mattoso, Alexandru Costan, Renan Souza, Débora Pina, et al.. ProvLight: Efficient Workflow Provenance Capture on the Edge-to-Cloud Continuum. IEEE Cluster 2023 - IEEE International Conference on Cluster Computing, Oct 2023, Santa Fe, New Mexico, United States. pp.1-13. ⟨hal-04161546⟩
90 View
43 Download

Share

Gmail Facebook X LinkedIn More