H-Revolve: A Framework for Adjoint Computation on Synchronous Hierarchical Platforms
Résumé
We study the problem of checkpointing strategies for adjoint computation on synchronous hierarchical
platforms, specifically computational platforms with several levels of storage with different writing and
reading costs. When reversing a large adjoint chain, choosing which data to checkpoint and where is a critical
decision for the overall performance of the computation. We introduce H-Revolve, an optimal algorithm for
this problem. We make it available in a public Python library along with the implementation of several state-of-
the-art algorithms for the variant of the problem with two levels of storage. We provide a detailed description
of how one can use this library in an adjoint computation software in the field of automatic differentiation
or backpropagation. Finally, we evaluate the performance of H-Revolve and other checkpointing heuristics
though an extensive campaign of simulation.
Origine | Fichiers produits par l'(les) auteur(s) |
---|
Loading...