Task-based parallel programming for scalable matrix product algorithms - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles ACM Transactions on Mathematical Software Year : 2023

Task-based parallel programming for scalable matrix product algorithms


Task-based programming models have succeeded in gaining the interest of the high-performance mathematical software community because they relieve part of the burden of developing and implementing distributed-memory parallel algorithms in an efficient and portable way.In increasingly larger, more heterogeneous clusters of computers, these models appear as a way to maintain and enhance more complex algorithms. However, task-based programming models lack the flexibility and the features that are necessary to express in an elegant and compact way scalable algorithms that rely on advanced communication patterns. We show that the Sequential Task Flow paradigm can be extended to write compact yet efficient and scalable routines for linear algebra computations. Although, this work focuses on dense General Matrix Multiplication, the proposed features enable the implementation of more complex algorithms. We describe the implementation of these features and of the resulting GEMM operation. Finally, we present an experimental analysis on two homogeneous supercomputers showing that our approach is competitive up to 32,768 CPU cores with state-of-the-art libraries and may outperform them for some problem dimensions. Although our code can use GPUs straightforwardly, we do not deal with this case because it implies other issues which are out of the scope of this work.
Fichier principal
Vignette du fichier
journal-nocopy.pdf (819.84 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03936659 , version 1 (12-01-2023)
hal-03936659 , version 2 (10-02-2023)



Emmanuel Agullo, Alfredo Buttari, Abdou Guermouche, Julien Herrmann, Antoine Jego. Task-based parallel programming for scalable matrix product algorithms. ACM Transactions on Mathematical Software, 2023, ⟨10.1145/3583560⟩. ⟨hal-03936659v2⟩
292 View
264 Download



Gmail Mastodon Facebook X LinkedIn More