Group-wise analysis of time series of images requires to compare longitudinal evolutions of images observed on different subjects. In medical imaging, longitudinal anatomical changes can be modeled thanks to non-rigid registration of follow-up images. The comparison of longitudinal trajectories requires the transport (or "normalization") of longitudinal deformations in a common reference frame. We previously proposed an effective computational scheme based on the Schild's ladder for the parallel transport of diffeomorphic deformations parameterized by tangent velocity fields, based on the construction of a geodesic parallelogram on a manifold. Schild's ladder may be however inefficient for transporting longitudinal deformations from image time series of multiple time points, in which the computation of the geodesic diagonals is required several times. We propose here a new algorithm, the pole ladder, in which one diagonal of the parallelogram is the baseline-to-reference frame geodesic. This drastically reduces the number of geodesics to compute. Moreover, differently from the Schild's ladder, the pole ladder is symmetric with respect to the baseline-to-reference frame geodesic. From the theoretical point of view, we show that the pole ladder is rigorously equivalent to the Schild's ladder when transporting along geodesics. From the practical point of view, we establish the computational advantages and demonstrate the effectiveness of this very simple method by comparing with standard methods of transport on simulated images with progressing brain atrophy. Finally, we illustrate its application to a clinical problem: the measurement of the longitudinal progression in Alzheimer's disease. Results suggest that an important gain in sensitivity could be expected in group-wise comparisons.