In this article, we focus on the parameterization of non-rigid geometrical deformations with a small number of flexible degrees of freedom . In previous work, we proposed a general framework called polyaffine to parameterize deformations with a finite number of rigid or affine components, while guaranteeing the invertibility of global deformations. However, this framework lacks some important properties: the inverse of a polyaffine transformation is not polyaffine in general, and the polyaffine fusion of affine components is not invariant with respect to a change of coordinate system. We present here a novel general framework, called Log-Euclidean polyaffine, which overcomes these defects. We also detail a simple algorithm, the Fast Polyaffine Transform, which allows to compute very efficiently Log-Euclidean polyaffine transformations and their inverses on regular grids. The results presented here on real 3D locally affine registration suggest that our novel framework provides a general and efficient way of fusing local rigid or affine deformations into a global invertible transformation without introducing artifacts, independently of the way local deformations are first estimated. Last but not least, we show in this article that the Log-Euclidean polyaffine framework is implicitely based on a Log-Euclidean framework for rigid and affine transformations, which generalizes to linear transformations the Log-Euclidean framework recently proposed for tensors. We detail in the Appendix of this article the properties of this novel framework, which allows a straightforward and efficient generalization to linear transformations of classical vectorial tools, with excellent theoretical properties. In particular, we propose here a simple generalization to locally rigid or affine deformations of a visco-elastic regularization energy used for dense transformations.