Beyond $\ell$1 sparse coding in V1 - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles PLoS Computational Biology Year : 2023

Beyond $\ell$1 sparse coding in V1

Ilias Rentzeperis
Luca Calatroni
Dario Prandi

Abstract

Growing evidence indicates that only a sparse subset from a pool of sensory neurons is active for the encoding of visual stimuli at any instant in time. Traditionally, to replicate such biological sparsity, generative models have been using the $\ell$1 norm as a penalty due to its convexity, which makes it amenable to fast and simple algorithmic solvers. In this work, we use biological vision as a test-bed and show that the soft thresholding operation associated to the use of the $\ell$1 norm is highly suboptimal compared to other functions suited to approximating $\ell$p with 0 ≤ p < 1 (including recently proposed continuous exact relaxations), in terms of performance. We show that $\ell$1 sparsity employs a pool with more neurons, i.e. has a higher degree of overcompleteness, in order to maintain the same reconstruction error as the other methods considered. More specifically, at the same sparsity level, the thresholding algorithm using the $\ell$1 norm as a penalty requires a dictionary of ten times more units compared to the proposed approach, where a non-convex continuous relaxation of the $\ell$0 pseudo-norm is used, to reconstruct the external stimulus equally well. At a fixed sparsity level, both $\ell$0 - and $\ell$1 -based regularization develop units with receptive field (RF) shapes similar to biological neurons in V1 (and a subset of neurons in V2), but $\ell$0 -based regularization shows approximately five times better reconstruction of the stimulus. Our results in conjunction with recent metabolic findings indicate that for V1 to operate efficiently it should follow a coding regime which uses a regularization that is closer to the $\ell$0 pseudo-norm rather than the $\ell$1 one, and suggests a similar mode of operation for the sensory cortex in general.
Fichier principal
Vignette du fichier
journal.pcbi.1011459.pdf (1.7 Mo) Télécharger le fichier
Origin Publisher files allowed on an open archive
Licence

Dates and versions

hal-04227569 , version 1 (03-10-2023)

Licence

Identifiers

Cite

Ilias Rentzeperis, Luca Calatroni, Laurent Perrinet, Dario Prandi. Beyond $\ell$1 sparse coding in V1. PLoS Computational Biology, 2023, 19 (9), pp.e1011459. ⟨10.1371/journal.pcbi.1011459⟩. ⟨hal-04227569⟩
49 View
25 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More