Online atlasing, i.e., incrementing an atlas with new images as they are acquired, is key when performing studies on very large, or still being gathered, databases. Regular approaches to atlasing however do not focus on this aspect and impose a complete reconstruction of the atlas when adding images. We propose instead a diffeomorphic online atlasing method that allows gradual updates to an atlas. In this iterative centroid approach, we integrate new subjects in the atlas in an iterative manner, gradually moving the centroid of the images towards its final position. This leads to a computationally cheap approach since it only necessitates one additional registration per new subject added. We validate our approach on several experiments with three main goals: 1-to evaluate atlas image quality of the obtained atlases with sharpness and overlap measures, 2-to assess the deviation in terms of transformations with respect to a conventional atlasing method and 3-to compare its computational time with regular approaches of the literature. We demonstrate that the transformations divergence with respect to a stateof-the-art atlas construction method is small and reaches a plateau, that the two construction methods have the same ability to map subject homologous regions onto a common space and produce images of equivalent quality. The computational time of our approach is also drastically reduced for regular updates. Finally, we also present a direct extension of our method to update spatio-temporal atlases, especially useful for developmental studies.