Weight identification through global optimization in a new hysteretic neural network model - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

Weight identification through global optimization in a new hysteretic neural network model

Abstract

Unlike their biological counterparts, simple artificial neural networks are unable to retain information from their past state to influence their behavior. In this contribution, we propose to consider new nonlinear activation functions, whose outputs depend both from the current and past inputs through a hysteresis effect. This hysteresis model is developed in the framework of convolutional neural networks. We then show that, by choosing the nonlinearity in the vast class of rational functions, the identification of the weights amounts to solving a rational optimization problem. For the latter, recent methods are applicable that come with global optimality guarantee, contrary to most optimization methods used in the neural network community. Finally, simulations show that such hysteresis nonlinear activation functions cannot be approximated by traditional ones and illustrate the effectiveness of our weight identification method.
Fichier principal
Vignette du fichier
article_icassp2021.pdf (233 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03227078 , version 1 (16-05-2021)

Identifiers

Cite

Elie Leroy, Arthur Marmin, Marc Castella, Laurent Duval. Weight identification through global optimization in a new hysteretic neural network model. ICASSP 2021 - 46th International Conference on Acoustics, Speech and Signal Processing, Jun 2021, Toronto (online), Canada. pp.5315-5319, ⟨10.1109/ICASSP39728.2021.9413383⟩. ⟨hal-03227078⟩
116 View
175 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More