This paper proposes a technique for simplifying a given Gaussian mixture model, i.e. reformulating the density in a more parcimonious manner, if possible (less Gaussian components in the mixture). Numerous applications requiring aggregation of models from various sources, or index structures over sets of mixture models for fast access, may benefit from the technique. Variational Bayesian estimation of mixtures is known to be a powerful technique on punctual data. We derive herein a new version of the Variational-Bayes EM algorithm that operates on Gaussian components of a given mixture and suppresses redundancy, if any, while preserving structure of the underlying generative process. A main feature of the present scheme is that it merely resorts to the parameters of the original mixture, ensuring low computational cost. Experimental results are reported on real data.