Online A-optimal design and active linear regression - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Online A-optimal design and active linear regression

Xavier Fontaine
  • Fonction : Auteur
  • PersonId : 1063788
Pierre Perrault
  • Fonction : Auteur
Michal Valko

Résumé

We consider in this paper the problem of optimal experiment design where a decision maker can choose which points to sample to obtain an estimateβ of the hidden parameter β of an underlying linear model. The key challenge of this work lies in the heteroscedasticity assumption that we make, meaning that each covariate has a different and unknown variance. The goal of the decision maker is then to figure out on the fly the optimal way to allocate the total budget of T samples between covariates, as sampling several times a specific one will reduce the variance of the estimated model around it (but at the cost of a possible higher variance elsewhere). By trying to minimize the 2-loss E[ β − β 2 ] the decision maker is actually minimizing the trace of the covariance matrix of the problem, which corresponds then to online A-optimal design. Combining techniques from bandit and convex optimization we propose a new active sampling algorithm and we compare it with existing ones. We provide theoretical guarantees of this algorithm in different settings, including a O(T −2) regret bound in the case where the covariates form a basis of the feature space, generalizing and improving existing results. Numerical experiments validate our theoretical findings.
Fichier principal
Vignette du fichier
fontaine2021online.pdf (2.42 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03289318 , version 1 (16-07-2021)

Identifiants

Citer

Xavier Fontaine, Pierre Perrault, Michal Valko, Vianney Perchet. Online A-optimal design and active linear regression. International Conference on Machine Learning, Jul 2021, Vienna / Virtual, Austria. ⟨10.48550/arXiv.1906.08509⟩. ⟨hal-03289318⟩
89 Consultations
158 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More