Leveraging augmented-Lagrangian techniques for differentiating over infeasible quadratic programs in machine learning - Inria - Institut national de recherche en sciences et technologies du numérique
Conference Papers Year : 2024

Leveraging augmented-Lagrangian techniques for differentiating over infeasible quadratic programs in machine learning

Abstract

Optimization layers within neural network architectures have become increasingly popular for their ability to solve a wide range of machine learning tasks and to model domain-specific knowledge. However, designing optimization layers requires careful consideration as the underlying optimization problems might be infeasible during training. Motivated by applications in learning, control and robotics, this work focuses on convex quadratic programming (QP) layers. The specific structure of this type of optimization layer can be efficiently exploited for faster computations while still allowing rich modeling capabilities. We leverage primal-dual augmented Lagrangian techniques for computing derivatives of both feasible and infeasible QP solutions. More precisely, we propose a unified approach which tackles the differentiability of the closest feasible QP solutions in a classical sense. We then harness this approach to enrich the expressive capabilities of existing QP layers. More precisely, we show how differentiating through infeasible QPs during training enables to drive towards feasibility at test time a new range of QP layers. These layers notably demonstrate superior predictive performance in some conventional learning tasks. Additionally, we present alternative formulations that enhance numerical robustness, speed, and accuracy for training such layers. Along with these contributions, we provide an open-source C++ software package called QPLayer for differentiating feasible and infeasible convex QPs and which can be interfaced with modern learning frameworks.
Fichier principal
Vignette du fichier
2515_Leveraging_augmented_Lagr.pdf (2.25 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04133055 , version 1 (19-06-2023)
hal-04133055 , version 2 (21-09-2024)

Licence

Identifiers

  • HAL Id : hal-04133055 , version 2

Cite

Antoine Bambade, Fabian Schramm, Adrien Taylor, Justin Carpentier. Leveraging augmented-Lagrangian techniques for differentiating over infeasible quadratic programs in machine learning. ICLR 2024 - The Twelfth International Conference on Learning Representations, May 2024, Vienne (Austria), France. ⟨hal-04133055v2⟩
801 View
1102 Download

Share

More