Optimal quantization of rank-one matrices in floating-point arithmetic---with applications to butterfly factorizations - Inria - Institut national de recherche en sciences et technologies du numérique
Preprints, Working Papers, ... Year : 2023

Optimal quantization of rank-one matrices in floating-point arithmetic---with applications to butterfly factorizations

Abstract

We consider the problem of optimally quantizing rank-one matrices to low precision floating-point arithmetic. We first explain that the naive strategy of separately quantizing the two rank-one factors can be far from optimal, and we provide worst case error bounds to support this observation. We characterize the optimal solution as the quantization of suitably scaled factors of the rank-one matrix and we develop an algorithm of tractable complexity to find the optimal scaling parameters. Using random rank-one matrices, we show experimentally that our algorithm can significantly reduce the quantization error. We then apply this algorithm to the quantization of butterfly factorizations, a fundamental tool that appears in many fast linear transforms. We show how the properties of butterfly supports can be exploited to approach the problem via a series of rank-one quantization problems and we employ our algorithm as a building block in a heuristic procedure to quantize a product of butterfly factors. We show that, despite being only heuristic, this strategy can be much more accurate than quantizing each factor independently or, equivalently, can achieve storage reductions of up to 30% with no loss of accuracy.
Fichier principal
Vignette du fichier
rank1_quant.pdf (1.02 Mo) Télécharger le fichier
figs_def.zip (484.13 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04125381 , version 1 (12-06-2023)

Licence

Identifiers

  • HAL Id : hal-04125381 , version 1

Cite

Rémi Gribonval, Theo Mary, Elisa Riccietti. Optimal quantization of rank-one matrices in floating-point arithmetic---with applications to butterfly factorizations. 2023. ⟨hal-04125381⟩
171 View
167 Download

Share

More