Abstract Bernard O Koopman proposed an alternative view of dynamical systems based on linear operator theory, in which the time evolution of a dynamical system is analogous to the linear propagation of an infinite-dimensional vector of observables. In the last few years, several works have shown that finite-dimensional approximations of this operator can be extremely useful for several applications, such as prediction, control, and data assimilation. In particular, a Koopman representation of a dynamical system with a finite number of dimensions will avoid all the problems caused by nonlinearity in classical state-space models. In this work, the identification of finite-dimensional approximations of the Koopman operator and its associated observables is expressed through the inversion of an unknown augmented linear dynamical system. The proposed framework can be regarded as an extended dynamical mode decomposition that uses a collection of latent observables. The use of a latent dictionary applies to a large class of dynamical regimes, and it provides new means for deriving appropriate finite-dimensional linear approximations to high-dimensional nonlinear systems.