Normalization-Equivariant Neural Networks with Application to Image Denoising - Inria - Institut national de recherche en sciences et technologies du numérique
Conference Papers Year : 2023

Normalization-Equivariant Neural Networks with Application to Image Denoising

Abstract

In many information processing systems, it may be desirable to ensure that any change of the input, whether by shifting or scaling, results in a corresponding change in the system response. While deep neural networks are gradually replacing all traditional automatic processing methods, they surprisingly do not guarantee such normalization-equivariance (scale + shift) property, which can be detrimental in many applications. To address this issue, we propose a methodology for adapting existing neural networks so that normalization-equivariance holds by design. Our main claim is that not only ordinary convolutional layers, but also all activation functions, including the ReLU (rectified linear unit), which are applied element-wise to the pre-activated neurons, should be completely removed from neural networks and replaced by better conditioned alternatives. To this end, we introduce affine-constrained convolutions and channel-wise sort pooling layers as surrogates and show that these two architectural modifications do preserve normalization-equivariance without loss of performance. Experimental results in image denoising show that normalization-equivariant neural networks, in addition to their better conditioning, also provide much better generalization across noise levels.
Fichier principal
Vignette du fichier
NENN_arXiv-9.pdf (1.44 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04123516 , version 1 (09-06-2023)

Licence

Identifiers

Cite

Sébastien Herbreteau, Emmanuel Moebel, Charles Kervrann. Normalization-Equivariant Neural Networks with Application to Image Denoising. Conference on Neural Information Processing Systems (NeurIPS), Dec 2023, New-Orleans, United States. ⟨10.48550/arXiv.2306.05037⟩. ⟨hal-04123516⟩
48 View
31 Download

Altmetric

Share

More