Meta-learning from Learning Curves: Challenge Design and Baseline Results - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Meta-learning from Learning Curves: Challenge Design and Baseline Results

Résumé

Meta-learning has been widely studied and implemented in many Automated Machine Learning systems to improve the process of selecting and training Machine Learning models for new tasks, by leveraging expertise acquired on previously observed tasks. We design a novel meta-learning challenge aiming at learning-to-learn from one of the most essential model evaluation data, the learning curve. It consists of multiple model evaluations collected during the process of training. A meta-learner is expected to apply a learned policy to learning curves of partially trained models on the task at hand, to rapidly find the best task solution, without training all potential models to convergence. This implies learning the exploration-exploitation trade-off. Our challenge is split into two phases: a development phase and a final test phase. In each phase, a meta-learner is meta-trained and meta-tested on validation learning curves (development phase) or test learning curves (final test phase). During meta-training, the meta-learner is allowed to learn from the provided learning curves in any possible way. In meta-testing, we borrowed the common Reinforcement Learning setting in which an agent (a meta-learner) learns by interacting with an environment storing pre-computed learning curves. A meta-learner must pay a cost (corresponding to the actual training and testing time) to reveal learning curve information progressively. The meta-learner is evaluated and ranked based on the average area under its learning curves. This challenge was accepted as part of the official selection of WCCI 2022 competitions.
Fichier principal
Vignette du fichier
conference_101719.pdf (2.03 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03740118 , version 1 (16-09-2022)

Identifiants

Citer

Manh Hung Nguyen, Lisheng Sun-Hosoya, Nathan Grinsztajn, Isabelle Guyon. Meta-learning from Learning Curves: Challenge Design and Baseline Results. IJCNN 2022 - International Joint Conference on Neural Networks, Jul 2022, Padua, Italy. pp.1-8, ⟨10.1109/IJCNN55064.2022.9892534⟩. ⟨hal-03740118⟩
92 Consultations
139 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More