Melissa: Large Scale In Transit Sensitivity Analysis Avoiding Intermediate Files - Inria - Institut national de recherche en sciences et technologies du numérique
Communication Dans Un Congrès Année : 2017

Melissa: Large Scale In Transit Sensitivity Analysis Avoiding Intermediate Files

Résumé

Global sensitivity analysis is an important step for analyzing and validating numerical simulations. One classical approach consists in computing statistics on the outputs from well-chosen multiple simulation runs. Simulation results are stored to disk and statistics are computed postmortem. Even if supercomputers enable to run large studies, scientists are constrained to run low resolution simulations with a limited number of probes to keep the amount of intermediate storage manageable. In this paper we propose a file avoiding, adaptive, fault tolerant and elastic framework that enables high resolution global sensitivity analysis at large scale. Our approach combines iterative statistics and in transit processing to compute Sobol' indices without any intermediate storage. Statistics are updated on-the-fly as soon as the in transit parallel server receives results from one of the running simulations. For one experiment, we computed the Sobol' indices on 10M hexahedra and 100 timesteps, running 8000 parallel simulations executed in 1h27 on up to 28672 cores, avoiding 48TB of file storage.
Fichier principal
Vignette du fichier
main-Sobol-SC-2017-HALVERSION.pdf (2.39 Mo) Télécharger le fichier
Vignette du fichier
Variance.jpg (66.48 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Format Figure, Image
Loading...

Dates et versions

hal-01607479 , version 1 (02-10-2017)

Identifiants

  • HAL Id : hal-01607479 , version 1

Citer

Théophile Terraz, Alejandro Ribes, Yvan Fournier, Bertrand Iooss, Bruno Raffin. Melissa: Large Scale In Transit Sensitivity Analysis Avoiding Intermediate Files. The International Conference for High Performance Computing, Networking, Storage and Analysis (Supercomputing), Nov 2017, Denver, United States. pp.1 - 14. ⟨hal-01607479⟩
1368 Consultations
1275 Téléchargements

Partager

More