Characterizing the Performance of Modern Architectures Through Opaque Benchmarks: Pitfalls Learned the Hard Way - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2017

Characterizing the Performance of Modern Architectures Through Opaque Benchmarks: Pitfalls Learned the Hard Way

Résumé

Determining key characteristics of High Performance Computing machines that would allow to predict its performance is an old and recurrent dream. This was, for example, the rationale behind the design of the LogP model that later evolved into many variants (LogGP, LogGPS, LoGPS) to cope with network technology evolution and complexity. Although network has received a lot of attention, predicting the performance of computation kernels can be very challenging as well. In particular, the tremendous increase of internal parallelism and deep memory hierarchy in modern multi-core architectures often limit applications by the memory access rate. In this context, determining the key characteristics of a machine such as the peak bandwidth of each cache level as well as how an application uses such memory hierarchy can be the key to predict or to extrapolate the performance of applications. Based on such performance models, most high-level simulation-based frameworks separately characterize a machine and an application, later convolving both signatures to predict the overall performance. We evaluate the suitability of such approaches to modern architectures and applications by trying to reproduce the work of others. When trying to build our own framework, we realized that, regardless of the quality of the underlying models or software, most of these framework rely on " opaque " benchmarks to characterize the platform. In this article, we report the many pitfalls we encountered when trying to characterize both the network and the memory performance of modern machines. We claim that opaque benchmarks that do not clearly separate experiment design, measurements, and analysis should be avoided as much as possible. Likewise an a priori identification of experimental factors should be done to make sure the experimental conditions are adequate.
Fichier principal
Vignette du fichier
IPDPS_REPPAR_2017_camera_ready.pdf (2 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01470399 , version 1 (17-02-2017)
hal-01470399 , version 2 (22-03-2017)

Identifiants

  • HAL Id : hal-01470399 , version 2

Citer

Luka Stanisic, Lucas Mello Schnorr, Augustin Degomme, Franz Heinrich, Arnaud Legrand, et al.. Characterizing the Performance of Modern Architectures Through Opaque Benchmarks: Pitfalls Learned the Hard Way. IPDPS 2017 - 31st IEEE International Parallel & Distributed Processing Symposium (RepPar workshop), May 2017, Orlando, United States. ⟨hal-01470399v2⟩
694 Consultations
749 Téléchargements

Partager

Gmail Facebook X LinkedIn More