In this report, we propose a novel similarity-invariant approach to model shapes. The method assumes a control points representation of the model and an arbitrary interpolation strategy. First, we construct the prior manifold using the distributions of the relative normalized distances between pairs of control points within the training set. The considered shape model refers to an incomplete graph that consists of intra and inter-cluster connections representing the inter-dependencies of control points. The clusters are determined according to the co-dependencies of the deformations of the control points within the training set. Then, we introduce a geometric partition of the space using a Voronoi decomposition that aims to determine relationships between the control points and the image domain. The same prior model is extended to the temporal domain to encode dynamic dependencies between the control points in the case of image sequences. We apply our model to both segmentation and tracking. Our knowledge-based approach to solve these problems is expressed using a Markov Random Field, where the unknowns are the positions of the control points. The pairwise potentials encode the variations of the shape, while the singleton potentials refer to the data term through the Voronoi decomposition of the space, and to the dynamic constraints. State-of-the art techniques from linear programming are considered both for the clustering and the optimization of the designed function. We present our results for the segmentation of the hand and the left ventricle in CT images, and the tracking of walking people.