Global Convergence of Frank Wolfe on One Hidden Layer Networks - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Preprints, Working Papers, ... Year : 2020

Global Convergence of Frank Wolfe on One Hidden Layer Networks

Abstract

We derive global convergence bounds for the Frank Wolfe algorithm when training one hidden layer neural networks. When using the ReLU activation function, and under tractable preconditioning assumptions on the sample data set, the linear minimization oracle used to incrementally form the solution can be solved explicitly as a second order cone program. The classical Frank Wolfe algorithm then converges with rate $O(1/T)$ where $T$ is both the number of neurons and the number of calls to the oracle.

Dates and versions

hal-02983259 , version 1 (29-10-2020)

Identifiers

Cite

Alexandre d'Aspremont, Mert Pilanci. Global Convergence of Frank Wolfe on One Hidden Layer Networks. 2020. ⟨hal-02983259⟩
2160 View
0 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More