Learning Rate Adaptation by Line Search in Evolution Strategies with Recombination - Inria - Institut national de recherche en sciences et technologies du numérique
Communication Dans Un Congrès Année : 2022

Learning Rate Adaptation by Line Search in Evolution Strategies with Recombination

Résumé

In this paper, we investigate the effect of a learning rate for the mean in Evolution Strategies with recombination. We study the effect of a half-line search after the mean shift direction is established, hence the learning rate value is conditioned to the direction. We prove convergence and study convergence rates in different dimensions and for different population sizes on the sphere function with the step-size proportional to the distance to the optimum. We empirically find that a perfect half-line search increases the maximal convergence rate on the sphere function by up to about 70%, assuming the line search imposes no additional costs. The speedup becomes less pronounced with increasing dimension. The line search reduces-however does not eliminate-the dependency of the convergence rate on the step-size. The optimal step-size assumes considerably smaller values with line search, which is consistent with previous results for different learning rate settings. The step-size difference is more pronounced in larger dimension and with larger population size, thereby diminishing an important advantage of a large population. CCS CONCEPTS • Theory of computation → Bio-inspired optimization; Nonconvex optimization.
Fichier principal
Vignette du fichier
line-search-es.pdf (11.22 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03644404 , version 1 (19-04-2022)

Licence

Identifiants

Citer

Armand Gissler, Anne Auger, Nikolaus Hansen. Learning Rate Adaptation by Line Search in Evolution Strategies with Recombination. GECCO 2022, Jul 2022, Boston, United States. ⟨10.1145/3512290.3528760⟩. ⟨hal-03644404⟩
108 Consultations
142 Téléchargements

Altmetric

Partager

More