A Sparsity-promoting Dictionary Model for Variational Autoencoders - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2022

A Sparsity-promoting Dictionary Model for Variational Autoencoders

Résumé

Structuring the latent space in probabilistic deep generative models, e.g., variational autoencoders (VAEs), is important to yield more expressive models and interpretable representations, and to avoid overfitting. One way to achieve this objective is to impose a sparsity constraint on the latent variables, e.g., via a Laplace prior. However, such approaches usually complicate the training phase, and they sacrifice the reconstruction quality to promote sparsity. In this paper, we propose a simple yet effective methodology to structure the latent space via a sparsity-promoting dictionary model, which assumes that each latent code can be written as a sparse linear combination of a dictionary's columns. In particular, we leverage a computationally efficient and tuning-free method, which relies on a zeromean Gaussian latent prior with learnable variances. We derive a variational inference scheme to train the model. Experiments on speech generative modeling demonstrate the advantage of the proposed approach over competing techniques, since it promotes sparsity while not deteriorating the output speech quality.
Fichier principal
Vignette du fichier
main.pdf (247.14 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03623769 , version 1 (29-03-2022)
hal-03623769 , version 2 (17-06-2022)

Identifiants

  • HAL Id : hal-03623769 , version 1

Citer

Mostafa Sadeghi, Paul Magron. A Sparsity-promoting Dictionary Model for Variational Autoencoders. 2022. ⟨hal-03623769v1⟩
86 Consultations
123 Téléchargements

Partager

Gmail Facebook X LinkedIn More