GECCO 2023 Tutorial on Benchmarking Multiobjective Optimizers 2.0 - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Documents Associated With Scientific Events Year : 2023

GECCO 2023 Tutorial on Benchmarking Multiobjective Optimizers 2.0

Dimo Brockhoff
Tea Tusar


Benchmarking is an important part of algorithm design, selection and recommendation---both in single- and multiobjective optimization. Benchmarking multiobjective solvers seems at first sight more complicated than benchmarking single-objective ones as there exists no natural total order on the objective space. In the past, comparisons of multiobjective solvers have therefore been done either entirely visually (at first) or via quality indicators and the attainment function. Only very recently did we realize that the quality indicator approach transforms a multiobjective problem into a single-objective (set-based) problem and thus all recent progress from the rapidly evolving single-objective benchmarking field can be transferred to the multiobjective case as well. Moreover, many multiobjective test functions have been proposed in the past but not much has changed in the last 15 or so years in terms of addressing the disadvantages of those problems (like Pareto sets on constraint boundaries, usage of distance variables, etc.). In this tutorial, we will discuss the past and future of benchmarking multiobjective optimizers. In particular, we will discuss the new view on benchmarking multiobjective algorithms by falling back on single-objective comparisons and thus being able to use all methodologies and tools from the single-objective domain such as empirical distributions of runtimes. We will also discuss the advantages and drawbacks of some widely used multiobjective test suites that we have all become familiar with over the years and explain how we can do better: by going back to the roots of what a multi-objective problem is in practice, namely the simultaneous optimization of multiple objective functions. Finally, we discuss recent advances in the visualization of (multiobjective) problem landscapes and compare the previous and newly proposed benchmark problems in the context of those landscape visualizations.
Fichier principal
Vignette du fichier
benchmarking2point0-brockhoff-tusar-2023.pdf (10.13 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04393953 , version 1 (15-01-2024)





Dimo Brockhoff, Tea Tusar. GECCO 2023 Tutorial on Benchmarking Multiobjective Optimizers 2.0. GECCO 2023 Companion - Companion Conference on Genetic and Evolutionary Computation, Jul 2023, Lisbon, Portugal. ACM, pp.1183-1212, GECCO '23 Companion: Companion Conference on Genetic and Evolutionary Computation. ⟨10.1145/2598394.2605339⟩. ⟨hal-04393953⟩
18 View
38 Download



Gmail Mastodon Facebook X LinkedIn More