H-Revolve: A Framework for Adjoint Computation on Synchronous Hierarchical Platforms - Inria - Institut national de recherche en sciences et technologies du numérique
Article Dans Une Revue ACM Transactions on Mathematical Software Année : 2020

H-Revolve: A Framework for Adjoint Computation on Synchronous Hierarchical Platforms

Résumé

We study the problem of checkpointing strategies for adjoint computation on synchronous hierarchical platforms, specifically computational platforms with several levels of storage with different writing and reading costs. When reversing a large adjoint chain, choosing which data to checkpoint and where is a critical decision for the overall performance of the computation. We introduce H-Revolve, an optimal algorithm for this problem. We make it available in a public Python library along with the implementation of several state-of- the-art algorithms for the variant of the problem with two levels of storage. We provide a detailed description of how one can use this library in an adjoint computation software in the field of automatic differentiation or backpropagation. Finally, we evaluate the performance of H-Revolve and other checkpointing heuristics though an extensive campaign of simulation.
Fichier principal
Vignette du fichier
toms.pdf (1.32 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02080706 , version 1 (27-03-2019)
hal-02080706 , version 2 (28-02-2020)

Identifiants

Citer

Julien Herrmann, Guillaume Pallez. H-Revolve: A Framework for Adjoint Computation on Synchronous Hierarchical Platforms. ACM Transactions on Mathematical Software, 2020, ⟨10.1145/3378672⟩. ⟨hal-02080706v2⟩
208 Consultations
406 Téléchargements

Altmetric

Partager

More