Learning Sparse Features with an Auto-Associator - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Book Sections Year : 2014

Learning Sparse Features with an Auto-Associator


A major issue in statistical machine learning is the design of a representa-tion, or feature space, facilitating the resolution of the learning task at hand. Sparse representations in particular facilitate discriminant learning: On the one hand, they are robust to noise. On the other hand, they disentangle the factors of variation mixed up in dense representations, favoring the separa-bility and interpretation of data. This chapter focuses on auto-associators (AAs), i.e. multi-layer neural networks trained to encode/decode the data and thus de facto defining a feature space. AAs, first investigated in the 80s, were recently reconsidered as building blocks for deep neural networks. This chapter surveys related work about building sparse representations, and presents a new non-linear explicit sparse representation method referred to as Sparse Auto-Associator (SAA), integrating a sparsity objective within the standard auto-associator learning criterion. The comparative empirical val-idation of SAAs on state-of-art handwritten digit recognition benchmarks shows that SAAs outperform standard auto-associators in terms of classifi-cation performance and yield similar results as denoising auto-associators. Furthermore, SAAs enable to control the representation size to some extent, through a conservative pruning of the feature space.
Fichier principal
Vignette du fichier
chapterSAA-Paugam.pdf (710.72 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-01109773 , version 1 (26-01-2015)



Sébastien Rebecchi, Hélène Paugam-Moisy, Michèle Sebag. Learning Sparse Features with an Auto-Associator. Kowaliw, Taras and Bredeche, Nicolas and Doursat, René. Growing Adaptive Machines, 557, Springer Verlag, pp.139 - 158, 2014, Studies in Computational Intelligence, ⟨10.1007/978-3-642-55337-0_4⟩. ⟨hal-01109773⟩
293 View
365 Download



Gmail Facebook X LinkedIn More