Computational anatomy aims at developing models to understand the anatomical variability of organs and tissues. A widely used and validated instrument for comparing the anatomy in medical images is non-linear diffeomorphic registration which is based on a rich mathematical background. For instance, the "large deformation diffeomorphic metric mapping" (LDDMM) framework defines a Riemannian setting by providing a right invariant metric on the tangent spaces, and solves the registration problem by computing geodesics parametrized by time-varying velocity fields. A simpler alternative based on stationary velocity fields (SVF) has been proposed, using the one-parameter subgroups from Lie groups theory. In spite of its better computational efficiency, the geometrical setting of the SVF is more vague, especially regarding the relationship between one-parameter subgroups and geodesics. In this work, we detail the properties of finite dimensional Lie groups that highlight the geometric foundations of one-parameter subgroups. We show that one can define a proper underlying geometric structure (an affine manifold) based on the canonical Cartan connections, for which tne-parameter subgroups and their translations are geodesics. This geometric structure is perfectly compatible with all the group operations (left, right composition and inversion), contrarily to left- (or right-) invariant Riemannian metrics. Moreover, we derive closed-form expressions for the parallel transport. Then, we investigate the generalization of such properties to infinite dimensional Lie groups. We suggest that some of the theoretical objections might actually be ruled out by the practical implementation of both the LDDMM and the SVF frameworks for image registration. This leads us to a more practical study comparing the parameterization (initial velocity field) of metric and Cartan geodesics in the specific optimization context of longitudinal and inter-subject image registration.Our experimental results suggests that stationarity is a good approximation for longitudinal deformations, while metric geodesics notably differ from stationary ones for inter-subject registration, which involves much larger and non-physical deformations. Then, we turn to the practical comparison of five parallel transport techniques along oneparameter subgroups. Our results point out the fundamental role played by the numerical implementation, which may hide the theoretical differences between the different schemes. Interestingly, even if the parallel transport generally depends on the path used, an experiment comparing the Cartan parallel transport along the one-parameter subgroup and the LDDMM (metric) geodesics from inter-subject registration suggests that our parallel transport methods are not so sensitive to the path.