Convex relaxations of penalties for sparse correlated variables with bounded total variation - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Machine Learning Year : 2015

Convex relaxations of penalties for sparse correlated variables with bounded total variation

Abstract

We study the problem of statistical estimation with a signal known to be sparse, spatially contiguous, and containing many highly correlated variables. We take inspiration from the recently introduced k-support norm, which has been successfully applied to sparse prediction problems with correlated features, but lacks any explicit structural constraints commonly found in machine learning and image processing. We address this problem by incorporating a total variation penalty in the k-support framework. We introduce the (k, s) support total variation norm as the tightest convex relaxation of the intersection of a set of sparsity and total variation constraints. We show that this norm leads to an intractable combinatorial graph optimization problem, which we prove to be NP-hard. We then introduce a tractable relaxation with approximation guarantees that scale well for grid structured graphs. We devise several first-order optimization strategies for statistical parameter estimation with the described penalty. We demonstrate the effectiveness of this penalty on classification in the low-sample regime, classification with M/EEG neuroimaging data, and image recovery with synthetic and real data background subtracted image recovery tasks. We extensively analyse the application of our penalty on the complex task of identifying predictive regions from low-sample high-dimensional fMRI brain data, we show that our method is particularly useful compared to existing methods in terms of accuracy, interpretability, and stability.
Fichier principal
Vignette du fichier
ConvexECML2015.pdf (644.22 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01167861 , version 1 (15-07-2015)

Identifiers

Cite

Eugene Belilovsky, Andreas Argyriou, Gaël Varoquaux, Matthew B. Blaschko. Convex relaxations of penalties for sparse correlated variables with bounded total variation. Machine Learning, 2015, pp.1-21. ⟨10.1007/s10994-015-5511-2⟩. ⟨hal-01167861⟩
523 View
681 Download

Altmetric

Share

Gmail Facebook X LinkedIn More