Black-Box Optimization Revisited: Improving Algorithm Selection Wizards through Massive Benchmarking - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles IEEE Transactions on Evolutionary Computation Year : 2022

Black-Box Optimization Revisited: Improving Algorithm Selection Wizards through Massive Benchmarking

Abstract

Existing studies in black-box optimization for machine learning suffer from low generalizability, caused by a typically selective choice of problem instances used for training and testing different optimization algorithms. Among other issues, this practice promotes overfitting and poor-performing user guidelines. To address this shortcoming, we propose in this work a benchmark suite, OptimSuite, which covers a broad range of black-box optimization problems, ranging from academic benchmarks to real-world applications, from discrete over numerical to mixed-integer problems, from small to very large-scale problems, from noisy over dynamic to static problems, etc. We demonstrate the advantages of such a broad collection by deriving from it Automated Black Box Optimizer (ABBO), a general-purpose algorithm selection wizard. Using three different types of algorithm selection techniques, ABBO achieves competitive performance on all benchmark suites. It significantly outperforms previous state of the art on some of them, including YABBOB and LSGO. ABBO relies on many high-quality base components. Its excellent performance is obtained without any task-specific parametrization. The OptimSuite benchmark collection, the ABBO wizard and its base solvers have all been merged into the open-source Nevergrad platform, where they are available for reproducible research.
Fichier principal
Vignette du fichier
2010.04542.pdf (8.37 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03154019 , version 1 (26-02-2021)

Identifiers

Cite

Laurent Meunier, Herilalaina Rakotoarison, Pak Kan Wong, Baptiste Roziere, Jeremy Rapin, et al.. Black-Box Optimization Revisited: Improving Algorithm Selection Wizards through Massive Benchmarking. IEEE Transactions on Evolutionary Computation, 2022, 26 (3), ⟨10.1109/TEVC.2021.3108185⟩. ⟨hal-03154019⟩
140 View
361 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More