Exploiting system level heterogeneity to improve the performance of a GeoStatistics multi-phase task-based application
Abstract
Heterogeneity is part of HPC infrastructures, not only at the intranode but at the system level. Applications with multiple phases with distinct resource necessities can take advantage of this internode heterogeneity to improve performance and reduce resource idleness. Such an application is ExaGeoStat, a task-based machine learning framework specifically designed for geostatistics data. This work presents strategies to efficiently distribute multi-phase applications in system-level heterogeneous resources. We both (1) improve application phase overlap by optimizing runtime and scheduling decisions and (2) compute the optimal distribution for all the phases using a linear program leveraging node heterogeneity while limiting communication overhead. The performance gains of our phase overlap improvements are between 36% and 50% compared to the original base synchronous and homogeneous execution. We show that by adding some slow nodes to a homogeneous set of fast nodes, we can improve the performance by another 25% compared to a standard block-cyclic distribution, thereby harnessing any machine.
Origin : Files produced by the author(s)