Enhanced Distribution Modelling via Augmented Architectures For Neural ODE Flows - Inria - Institut national de recherche en sciences et technologies du numérique
Communication Dans Un Congrès Année : 2022

Enhanced Distribution Modelling via Augmented Architectures For Neural ODE Flows

Résumé

While the neural ODE formulation of normalizing flows such as in FFJORD enables us to calculate the determinants of free form Jacobians in O(D) time, the flexibility of the transformation underlying neural ODEs has been shown to be suboptimal. In this paper, we present AFFJORD, a neural ODE-based normalizing flow which enhances the representation power of FFJORD by defining the neural ODE through special augmented transformation dynamics which preserve the topology of the space. Furthermore, we derive the Jacobian determinant of the general augmented form by generalizing the chain rule in the continuous sense into the cable rule, which expresses the forward sensitivity of ODEs with respect to their initial conditions. The cable rule gives an explicit expression for the Jacobian of a neural ODE transformation, and provides an elegant proof of the instantaneous change of variable. Our experimental results on density estimation in synthetic and high dimensional data, such as MNIST, CIFAR-10 and CelebA (32 × 32), show that AFFJORD outperforms the baseline FFJORD through the improved flexibility of the underlying vector field.
Fichier principal
Vignette du fichier
Enhanced_distribution_modelling_AFFJORD_HAL.pdf (1.13 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03911870 , version 1 (23-12-2022)
hal-03911870 , version 2 (23-12-2023)

Licence

Identifiants

  • HAL Id : hal-03911870 , version 2

Citer

Etrit Haxholli, Marco Lorenzi. Enhanced Distribution Modelling via Augmented Architectures For Neural ODE Flows. DLDE-III Workshop in the 37th Conference on Neural Information Processing Systems (NeurIPS 2023), Dec 2023, New Orleans, Louisiana, United States. ⟨hal-03911870v2⟩
235 Consultations
289 Téléchargements

Partager

More