This paper introduces an original method to perform augmented or mixed reality on deformable objects. Compared to state-of-the-art techniques, our method is able to track deformations of volumetric objects and not only surfacic objects. A flexible framework that relies on the combination of a 3D motion estimation and a physics-based deformable model used as a regularization and interpolation step allows to perform a non-rigid and robust registration. Results are exposed, based on computer-generated datasets and video sequences of real environments in order to assess the relevance of our approach.