Equivalence Between Nested Gibbs Measures and Log-Linear Combinations of Gibbs Measures
Résumé
In this report, three operations on Gibbs probability measures are studied. The first operation, which takes as argument one Gibbs probability measure and is often referred to as renormalization, consists in generating a new Gibbs measure by normalizing a power of the density of the given measure. Such a normalization has a twofold effect: first, it changes the regularization factor; and second, it concentrates the support within a subset of the original support. Interestingly, it is shown that these effects can be independently controlled by different parameters. The second operation, which takes as argument two Gibbs probability measures, consists of changing the reference measure of the latter by the former. Hence, the former is said to be "nested" within the latter, yielding a new Gibbs probability measure. The third operation consists of a normalized log-linear combination of the densities of Gibbs probability measures. The resulting probability measures, from both second and third operations, which are also Gibbs probability measures, are shown, respectively, to be the solutions to optimization problems of the expectations of linear combinations of the objective functions of the given measures, subject to a relative entropy regularization. Such optimization problems differ exclusively in the coefficients of the linear combinations. This observation leads the conclusion that there exists a set of parameters for which nesting one Gibbs probability measure into another has the same effect as log-linearly combining them. These operations are shown to have relevant applications in statistical learning. As an example, a one-shot federated learning system in which clients send to the server their locally trained Gibbs algorithms for being log-linearly combined by the server, is shown to achieve the same performance as a Gibbs algorithm trained upon the aggregation of all local training datasets.
Mots clés
| Origine | Fichiers produits par l'(les) auteur(s) |
|---|---|
| Licence |