Finite-time Lyapunov exponents of deep neural networks - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

Finite-time Lyapunov exponents of deep neural networks

Résumé

We compute how small input perturbations affect the output of deep neural networks, exploring an analogy between deep networks and dynamical systems, where the growth or decay of local perturbations is characterised by finite-time Lyapunov exponents. We show that the maximal exponent forms geometrical structures in input space, akin to coherent structures in dynamical systems. Ridges of large positive exponents divide input space into different regions that the network associates with different classes. These ridges visualise the geometry that deep networks construct in input space, shedding light on the fundamental mechanisms underlying their learning capabilities.

Dates et versions

hal-04304090 , version 1 (24-11-2023)

Licence

Paternité

Identifiants

Citer

L. Storm, H. Linander, J. Bec, K. Gustavsson, B. Mehlig. Finite-time Lyapunov exponents of deep neural networks. 2023. ⟨hal-04304090⟩
20 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More