Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2022

Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization

Abstract

Many supervised machine learning methods are naturally cast as optimization problems. For prediction models which are linear in their parameters, this often leads to convex problems for which many mathematical guarantees exist. Models which are non-linear in their parameters such as neural networks lead to non-convex optimization problems for which guarantees are harder to obtain. In this review paper, we consider two-layer neural networks with homogeneous activation functions where the number of hidden neurons tends to infinity, and show how qualitative convergence guarantees may be derived.
Fichier principal
Vignette du fichier
ICM-Bach-Chizat-HAL.pdf (1.72 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03379011 , version 1 (15-10-2021)

Identifiers

Cite

Francis Bach, Lénaïc Chizat. Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization. International Congress of Mathematicians, Jul 2022, Saint-Petersbourg, Russia. ⟨hal-03379011⟩
138 View
1151 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More