The aim of this paper is to provide an overview of pharmacometric models that involve some latent process with Markovian dynamics. Such models include hidden Markov models which may be useful for describing the dynamics of a disease state that jumps from one state to another at discrete times. On the contrary, diffusion models are continuous-time and continuous-state Markov models that are relevant for modelling non observed phenomena that fluctuate continuously and randomly over time. We show that an extension of these models to mixed effects models is straightforward in a population context. We then show how the Forward-Backward algorithm used for inference in hidden Markov models and the extended Kalman filter used for inference in diffusion models can be combined with standard inference algorithms in mixed effects models for estimating the parameters of the model. The use of these models is illustrated with two applications: a hidden Markov model for describing the epileptic activity of a large number of patients and a stochastic differential equation based model for describing the pharmacokinetics of theophyllin.