Evaluation of two topology-aware heuristics on level-3 BLAS library for multi-GPU platforms - Inria - Institut national de recherche en sciences et technologies du numérique
Conference Papers Year : 2021

Evaluation of two topology-aware heuristics on level-3 BLAS library for multi-GPU platforms

Abstract

Nowadays GPUs have dominated the market considering the computing/power metric and numerous research works have provided Basic Linear Algebra Subprograms implementations accelerated on GPUs. Several software libraries have been developed for exploiting performance of systems with accelerators, but the real performance may be far from the platform peak performance with multiple GPUs. This paper presents two runtime heuristics to gain in performance when task based programs are performed on heterogeneous architecture such as multi-GPU systems. The first is a topology-aware policy to takes into account the heterogeneity of the high speed links that interconnect GPUs. The second is an optimistic heuristic that favor communication between devices. These have been implemented in the XKBLAS library BLAS-3 library. We made experiments on a NVIDIA DGX-1 with up to 8 GPUs V100 on a set of Basic Linear Algebra Subroutines. Experimental results on kernels showed that XKBlas outperformed most implementations including the overhead of creation and scheduling of dynamic tasks.
Fichier principal
Vignette du fichier
xkblas_pawatm106s1-file1.pdf (876.59 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03363275 , version 1 (03-10-2021)

Identifiers

  • HAL Id : hal-03363275 , version 1

Cite

Thierry Gautier, Joao Vicente Ferreira Lima. Evaluation of two topology-aware heuristics on level-3 BLAS library for multi-GPU platforms. PAW-ATM 2021 - 4th Annual Parallel Applications Workshop, Alternatives To MPI+X, Nov 2021, Saint Louis, United States. pp.1-11. ⟨hal-03363275⟩
126 View
217 Download

Share

More