Heuristics to reduce linear combinations of activation functions to improve image classification - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

Heuristics to reduce linear combinations of activation functions to improve image classification

Abstract

Image classification is one of the classical problems in computer vision, and CNNs (Convolutional Neural Networks) are widely used for this task. However, the choice of a CNN can vary depending on the chosen dataset. In this context, we have trainable activation functions that are crucial in CNNs and adapt to the data. One technique for constructing these functions is to write them as a linear combination of other activation functions, where the coefficients of this combination are learned during training. However, if we have a large number of activation functions to combine, the computational cost can be very high, and manually testing and choosing these functions may be impractical, depending on the number of available activation functions. To alleviate the difficulty of choosing which activation functions should be part of the linear combination, we propose two heuristics: Linear Combination Approximator by Coefficients (LCAC) and Major and Uniform Coefficient Extractor (MUCE). Our heuristics provide an efficient selection of a subset of activation functions so that their results are better or equivalent to the linear combination that uses all 34 available activation functions in our experiments (C34), considering the image classification problem. Compared to the C34 function, the LCAC function was better or equivalent in 62.5%, and the MUCE function in 87.5% of the conducted experiments.
No file

Dates and versions

hal-04396390 , version 1 (15-01-2024)

Licence

Attribution

Identifiers

Cite

Rogério Ferreira de Moraes, Raphael dos S. Evangelista, Andre Luiz da S. Pereira, Yanexis Pupo Toledo, Leandro Fernandes, et al.. Heuristics to reduce linear combinations of activation functions to improve image classification. 2023 36th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), Nov 2023, Rio Grande, Brazil. pp.169-174, ⟨10.1109/SIBGRAPI59091.2023.10347043⟩. ⟨hal-04396390⟩

Collections

INRIA
7 View
0 Download

Altmetric

Share

Gmail Facebook X LinkedIn More