Melissa: Large Scale In Transit Sensitivity Analysis Avoiding Intermediate Files - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2017

Melissa: Large Scale In Transit Sensitivity Analysis Avoiding Intermediate Files

Abstract

Global sensitivity analysis is an important step for analyzing and validating numerical simulations. One classical approach consists in computing statistics on the outputs from well-chosen multiple simulation runs. Simulation results are stored to disk and statistics are computed postmortem. Even if supercomputers enable to run large studies, scientists are constrained to run low resolution simulations with a limited number of probes to keep the amount of intermediate storage manageable. In this paper we propose a file avoiding, adaptive, fault tolerant and elastic framework that enables high resolution global sensitivity analysis at large scale. Our approach combines iterative statistics and in transit processing to compute Sobol' indices without any intermediate storage. Statistics are updated on-the-fly as soon as the in transit parallel server receives results from one of the running simulations. For one experiment, we computed the Sobol' indices on 10M hexahedra and 100 timesteps, running 8000 parallel simulations executed in 1h27 on up to 28672 cores, avoiding 48TB of file storage.
Fichier principal
Vignette du fichier
main-Sobol-SC-2017-HALVERSION.pdf (2.39 Mo) Télécharger le fichier
Vignette du fichier
Variance.jpg (66.48 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Format : Figure, Image
Loading...

Dates and versions

hal-01607479 , version 1 (02-10-2017)

Identifiers

  • HAL Id : hal-01607479 , version 1

Cite

Théophile Terraz, Alejandro Ribes, Yvan Fournier, Bertrand Iooss, Bruno Raffin. Melissa: Large Scale In Transit Sensitivity Analysis Avoiding Intermediate Files. The International Conference for High Performance Computing, Networking, Storage and Analysis (Supercomputing), Nov 2017, Denver, United States. pp.1 - 14. ⟨hal-01607479⟩
1323 View
1227 Download

Share

Gmail Facebook X LinkedIn More