Designing losses for data-free training of normalizing flows on Boltzmann distributions - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Preprints, Working Papers, ... Year : 2023

Designing losses for data-free training of normalizing flows on Boltzmann distributions

Abstract

Generating a Boltzmann distribution in high dimension has recently been achieved with Normalizing Flows, which enable fast and exact computation of the generated density, and thus unbiased estimation of expectations. However, current implementations rely on accurate training data, which typically comes from computationally expensive simulations. There is therefore a clear incentive to train models with incomplete or no data by relying solely on the target density, which can be obtained from a physical energy model (up to a constant factor). For that purpose, we analyze the properties of standard losses based on Kullback-Leibler divergences. We showcase their limitations, in particular a strong propensity for mode collapse during optimization on high-dimensional distributions. We then propose strategies to alleviate these issues, most importantly a new loss function well-grounded in theory and with suitable optimization properties. Using as a benchmark the generation of 3D molecular configurations, we show on several tasks that, for the first time, imperfect pre-trained models can be further optimized in the absence of training data.
Fichier principal
Vignette du fichier
main.pdf (1.41 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03936982 , version 1 (12-01-2023)

Licence

Identifiers

Cite

Loris Felardos, Jérôme Hénin, Guillaume Charpiat. Designing losses for data-free training of normalizing flows on Boltzmann distributions. 2023. ⟨hal-03936982⟩
70 View
35 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More