MultiVae: A Python library for Multimodal Generative Autoencoders - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Preprints, Working Papers, ... Year : 2023

MultiVae: A Python library for Multimodal Generative Autoencoders

Abstract

In recent years, there has been a major boom in the development of multimodal machine learning models. Among open topics, representation (fusion) and generation of multimodal data are very active fields of research. Recently, multimodal variational autoencoders have been attracting growing interest for both tasks, thanks to their versatility, scalability, and interpretability as probabilistic latent variable models. They are also particularly interesting models in the partially observed setting, as some models can learn even with missing data. In this article, we present MultiVae, an open-source Python library designed to bring together unified implementations of multimodal generative autoencoders models. It has been designed for easy, customizable use of these models on partially or fully observed data. This library facilitates the development and benchmarking of algorithms by integrating several popular datasets, variety of evaluation metrics and tools for monitoring and sharing models. For each model included, a key result is reproduced to validate our implementation. We also propose a case study in the less studied partially observed setting, evaluating the robustness of the models as a function of the missing ratio in a five-modalities dataset.
Fichier principal
Vignette du fichier
MultiVae-8 (1).pdf (9.3 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04207151 , version 1 (14-09-2023)

Identifiers

  • HAL Id : hal-04207151 , version 1

Cite

Agathe Senellart, Clément Chadebec, Stéphanie Allassonnière. MultiVae: A Python library for Multimodal Generative Autoencoders. 2023. ⟨hal-04207151⟩
160 View
135 Download

Share

Gmail Mastodon Facebook X LinkedIn More