Collaborative hyperparameter tuning - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2013

Collaborative hyperparameter tuning


Hyperparameter learning has traditionally been a manual task because of the limited number of trials. Today's computing infrastructures allow bigger evaluation budgets, thus opening the way for algorithmic approaches. Recently, surrogate-based optimization was successfully applied to hyperparameter learning for deep belief networks and to WEKA classifiers. The methods combined brute force computational power with model building about the behavior of the error function in the hyperparameter space, and they could significantly improve on manual hyperparameter tuning. What may make experienced practitioners even better at hyperparameter optimization is their ability to generalize across similar learning problems. In this paper, we propose a generic method to incorporate knowledge from previous experiments when simultaneously tuning a learning algorithm on new problems at hand. To this end, we combine surrogate-based ranking and optimization techniques for surrogate-based collaborative tuning (SCoT). We demonstrate SCoT in two experiments where it outperforms standard tuning techniques and single-problem surrogate-based optimization.
Fichier principal
Vignette du fichier
bardenet13.pdf (273.54 Ko) Télécharger le fichier
Origin Publisher files allowed on an open archive

Dates and versions

in2p3-00907381 , version 1 (21-11-2013)


  • HAL Id : in2p3-00907381 , version 1


R. Bardenet, M. Brendel, Balázs Kégl, Michèle Sebag. Collaborative hyperparameter tuning. 30th International Conference on Machine Learning (ICML 2013), Jun 2013, Atlanta, United States. pp.199-207. ⟨in2p3-00907381⟩
263 View
303 Download


Gmail Mastodon Facebook X LinkedIn More