Learning Rate Adaptation by Line Search in Evolution Strategies with Recombination - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2022

Learning Rate Adaptation by Line Search in Evolution Strategies with Recombination

Abstract

In this paper, we investigate the effect of a learning rate for the mean in Evolution Strategies with recombination. We study the effect of a half-line search after the mean shift direction is established, hence the learning rate value is conditioned to the direction. We prove convergence and study convergence rates in different dimensions and for different population sizes on the sphere function with the step-size proportional to the distance to the optimum. We empirically find that a perfect half-line search increases the maximal convergence rate on the sphere function by up to about 70%, assuming the line search imposes no additional costs. The speedup becomes less pronounced with increasing dimension. The line search reduces-however does not eliminate-the dependency of the convergence rate on the step-size. The optimal step-size assumes considerably smaller values with line search, which is consistent with previous results for different learning rate settings. The step-size difference is more pronounced in larger dimension and with larger population size, thereby diminishing an important advantage of a large population. CCS CONCEPTS • Theory of computation → Bio-inspired optimization; Nonconvex optimization.
Fichier principal
Vignette du fichier
line-search-es.pdf (11.22 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03644404 , version 1 (19-04-2022)

Licence

Identifiers

Cite

Armand Gissler, Anne Auger, Nikolaus Hansen. Learning Rate Adaptation by Line Search in Evolution Strategies with Recombination. GECCO 2022, Jul 2022, Boston, United States. ⟨10.1145/3512290.3528760⟩. ⟨hal-03644404⟩
96 View
111 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More