Feedback Control for Online Training of Neural Networks - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2019

Feedback Control for Online Training of Neural Networks

Résumé

Convolutional neural networks (CNNs) are commonly used for image classification tasks, raising the challenge of their application on data flows. During their training, adaptation is often performed by tuning the learning rate. Usual learning rate strategies are time-based i.e. monotonously decreasing. In this paper, we advocate switching to a performance-based adaptation, in order to improve the learning efficiency. We present E (Exponential)/PI (Proportional Integral)-Control, a conditional learning rate strategy that combines a feedback PI controller based on the CNN loss function, with an exponential control signal to smartly boost the learning and adapt the PI parameters. Stability proof is provided as well as an experimental evaluation using two state of the art image datasets (CIFAR-10 and Fashion-MNIST). Results show better performances than the related works (faster network accuracy growth reaching higher levels) and robustness of the E/PI-Control regarding its parametrization.
Fichier principal
Vignette du fichier
CCTA19.pdf (807.83 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02115916 , version 1 (04-09-2019)
hal-02115916 , version 2 (19-11-2019)

Identifiants

Citer

Zilong Zhao, Sophie Cerf, Bogdan Robu, Nicolas Marchand. Feedback Control for Online Training of Neural Networks. CCTA 2019 - 3rd IEEE Conference on Control Technology and Applications, Aug 2019, Hong Kong, China. ⟨10.1109/CCTA.2019.8920662⟩. ⟨hal-02115916v2⟩
301 Consultations
169 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More