Modularity in Deep Learning: A Survey - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

Modularity in Deep Learning: A Survey

Abstract

Modularity is a general principle present in many fields. It offers attractive advantages, including, among others, ease of conceptualization, interpretability, scalability, module combinability, and module reusability. The deep learning community has long sought to take inspiration from the modularity principle, either implicitly or explicitly. This interest has been increasing over recent years. We review the notion of modularity in deep learning around three axes: data, task, and model, which characterize the life cycle of deep learning. Data modularity refers to the observation or creation of data groups for various purposes. Task modularity refers to the decomposition of tasks into sub-tasks. Model modularity means that the architecture of a neural network system can be decomposed into identifiable modules. We describe different instantiations of the modularity principle, and we contextualize their advantages in different deep learning sub-fields. Finally, we conclude the paper with a discussion of the definition of modularity and directions for future research.
Fichier principal
Vignette du fichier
main.pdf (1.96 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04224278 , version 1 (01-10-2023)

Identifiers

Cite

Haozhe Sun, Isabelle Guyon. Modularity in Deep Learning: A Survey. SAI 2023 - Intelligent Computing Conference, Jun 2023, London, United Kingdom. pp.561-595, ⟨10.1007/978-3-031-37963-5_40⟩. ⟨hal-04224278⟩
43 View
29 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More