Parallel Memory-Independent Communication Bounds for SYRK - Inria - Institut national de recherche en sciences et technologies du numérique
Conference Papers Year : 2023

Parallel Memory-Independent Communication Bounds for SYRK

Hussam Al Daas
  • Function : Author
  • PersonId : 1249073
Grey Ballard
  • Function : Author
  • PersonId : 1249074
Kathryn Rouse
  • Function : Author
  • PersonId : 1249075

Abstract

In this paper, we focus on the parallel communication cost of multiplying a matrix with its transpose, known as a symmetric rank-k update (SYRK). SYRK requires half the computation of general matrix multiplication because of the symmetry of the output matrix. Recent work (Beaumont et al., SPAA '22) has demonstrated that the sequential I/O complexity of SYRK is also a constant factor smaller than that of general matrix multiplication. Inspired by this progress, we establish memory-independent parallel communication lower bounds for SYRK with smaller constants than general matrix multiplication, and we show that these constants are tight by presenting communication-optimal algorithms. The crux of the lower bound proof relies on extending a key geometric inequality to symmetric computations and analytically solving a constrained nonlinear optimization problem. The optimal algorithms use a triangular blocking scheme for parallel distribution of the symmetric output matrix and corresponding computation.
Fichier principal
Vignette du fichier
SYRK-spaa2023.pdf (861.21 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04076513 , version 1 (20-04-2023)

Identifiers

Cite

Hussam Al Daas, Grey Ballard, Laura Grigori, Suraj Kumar, Kathryn Rouse. Parallel Memory-Independent Communication Bounds for SYRK. SPAA '23 - ACM Symposium on Parallelism in Algorithms and Architectures, Jun 2023, Orlando, United States. ⟨10.1145/3558481.3591072⟩. ⟨hal-04076513⟩
143 View
143 Download

Altmetric

Share

More