Quasi-Symplectic Langevin Variational Au-Toencoder - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2020

Quasi-Symplectic Langevin Variational Au-Toencoder

Résumé

Variational autoencoder (VAE) as one of the well-investigated generative model is very popular in nowadays neural learning research works. To leverage VAE in practical tasks which have high dimensions and massive dataset often facing the problem of low variance evidence lower bounds construction. Markov chain Monte Carlo (MCMC) is an effective approach to tight the evidence lower bound (ELBO) for approximating the posterior distribution. Hamiltonian Variational Autoencoder (HVAE) is one of those effective MCMC inspired approaches for constructing the low-variance ELBO which is also amenable for reparameterization trick. The solution significantly improves the performance of the posterior estimation, yet, a main drawback of HVAE is the leapfrog method need to access the posterior gradient twice which leads to bad inference efficiency and the GPU memory requirement is fair large. This flaw limited the application of Hamiltonian based inference framework for large scale networks inference. To tackle this problem, we propose a Quasi-symplectic Langevin Variational autoencoder (Langevin-VAE), which can be a significant improvement over resource usage efficiency. We qualitatively and quantitatively demonstrate the effectiveness of the Langevin-VAE compared to the state-of-art gradients informed inference framework.
Fichier principal
Vignette du fichier
ICLR_Langevin_VAE.pdf (398.98 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03024748 , version 1 (26-11-2020)
hal-03024748 , version 2 (23-03-2021)
hal-03024748 , version 3 (19-04-2021)
hal-03024748 , version 4 (03-06-2021)

Identifiants

  • HAL Id : hal-03024748 , version 1

Citer

Zihao Wang, Hervé Delingette. Quasi-Symplectic Langevin Variational Au-Toencoder. 2020. ⟨hal-03024748v1⟩
63 Consultations
450 Téléchargements

Partager

Gmail Facebook X LinkedIn More