Efficient Bayes Inference in Neural Networks through Adaptive Importance Sampling - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Journal of The Franklin Institute Year : 2023

Efficient Bayes Inference in Neural Networks through Adaptive Importance Sampling


Bayesian neural networks (BNNs) have received an increased interest in the last years. In BNNs, a complete posterior distribution of the unknown weight and bias parameters of the network is produced during the training stage. This probabilistic estimation offers several advantages with respect to point-wise estimates, in particular, the ability to provide uncertainty quantification when predicting new data. This feature inherent to the Bayesian paradigm, is useful in countless machine learning applications. It is particularly appealing in areas where decision-making has a crucial impact, such as medical healthcare or autonomous driving. The main challenge of BNNs is the computational cost of the training procedure since Bayesian techniques often face a severe curse of dimensionality. Adaptive importance sampling (AIS) is one of the most prominent Monte Carlo methodologies benefiting from sounded convergence guarantees and ease for adaptation. This work aims to show that AIS constitutes a successful approach for designing BNNs. More precisely, we propose a novel algorithm named PMCnet that includes an efficient adaptation mechanism, exploiting geometric information on the complex (often multimodal) posterior distribution. Numerical results illustrate the excellent performance and the improved exploration capabilities of the proposed method for both shallow and deep neural networks.
Fichier principal
Vignette du fichier
main.pdf (932.03 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04210696 , version 1 (19-09-2023)





Yunshi Huang, Emilie Chouzenoux, Víctor Elvira, Jean-Christophe Pesquet. Efficient Bayes Inference in Neural Networks through Adaptive Importance Sampling. Journal of The Franklin Institute, 2023, 360 (16), pp.12125-12149. ⟨10.1016/j.jfranklin.2023.08.044⟩. ⟨hal-04210696⟩
37 View
18 Download



Gmail Facebook X LinkedIn More