Efficient Identification of Butterfly Sparse Matrix Factorizations - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles SIAM Journal on Mathematics of Data Science Year : 2023

Efficient Identification of Butterfly Sparse Matrix Factorizations

Abstract

Fast transforms correspond to factorizations of the form $\mathbf{Z} = \mathbf{X}^{(1)} \ldots \mathbf{X}^{(J)}$, where each factor $ \mathbf{X}^{(\ell)}$ is sparse and possibly structured. This paper investigates essential uniqueness of such factorizations, i.e., uniqueness up to unavoidable scaling ambiguities. Our main contribution is to prove that any $N \times N$ matrix having the so-called butterfly structure admits an essentially unique factorization into $J$ butterfly factors (where $N = 2^{J}$), and that the factors can be recovered by a hierarchical factorization method, which consists in recursively factorizing the considered matrix into two factors. This hierarchical identifiability property relies on a simple identifiability condition in the two-layer and fixed-support setting. This approach contrasts with existing ones that fit the product of butterfly factors to a given matrix via gradient descent. The proposed method can be applied in particular to retrieve the factorization of the Hadamard or the discrete Fourier transform matrices of size $N=2^J$. Computing such factorizations costs $\mathcal{O}(N^{2})$, which is of the order of dense matrix-vector multiplication, while the obtained factorizations enable fast $\mathcal{O}(N \log N)$ matrix-vector multiplications and have the potential to be applied to compress deep neural networks.
Fichier principal
Vignette du fichier
main.pdf (730.61 Ko) Télécharger le fichier
Vignette du fichier
butterfly-vignette-bis.png (375.79 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Format : Figure, Image

Dates and versions

hal-03362626 , version 1 (01-10-2021)
hal-03362626 , version 2 (11-11-2021)
hal-03362626 , version 3 (15-02-2022)
hal-03362626 , version 4 (04-04-2022)
hal-03362626 , version 5 (02-08-2022)
hal-03362626 , version 6 (07-10-2022)

Identifiers

Cite

Léon Zheng, Elisa Riccietti, Rémi Gribonval. Efficient Identification of Butterfly Sparse Matrix Factorizations. SIAM Journal on Mathematics of Data Science, 2023, 5 (1), pp.22-49. ⟨10.1137/22M148872⟩. ⟨hal-03362626v6⟩

Relations

448 View
651 Download

Altmetric

Share

Gmail Facebook X LinkedIn More