Reducing simulation time is critical for applications such as closed loop control or iterative design op- timisation. In this context, model reduction techniques have become a growing area of research in the last decades. While research efforts have mainly been centered around feature based approaches like POD, BPOD or DMD, direct approaches leveraging Deep Neural Networks have been proposed in recent years with great success. Despite these promising results, neural network architectures provide little to no physical guarantees, and have limited interpretability. On the other hand, feature based methods often reconstruct the final solution through a linear combination of modes embedded with physical constraints. However, this often comes at the cost of loss of information and increased error rates. POD-Galerkin models are a perfect example of this trade-off between physical guarantees and performance loss. These models have been shown to be very efficient for the reduction of linear systems, but they are extremely limited when applied to nonlinear systems such as the Navier-Stokes equations. For example, Noack et al. have shown that a simple 3 equations model was able to capture the oscillatory dynamics of a flow over a cylinder, but failed to correctly predict the transition time and trajectory from a steady point of the system to its oscillatory regime. To address these shortcomings, we propose to add a closure term to POD-Galerkin models to correct their dynamics. Observing that the information lost during the projection on the POD basis can be retrieved by considering the past states of the system, we use simple neural networks in combination with delay differential equations to reconstruct the required correction. We show that a satisfactory model can be trained through the Neural ODE framework to learn a memory based correction from simulation data. The final architecture can be compared to a time-continuous recurrent neural network. With this approach, we preserve the simple structure and low computational cost of Galerkin models while improving their performance. Using the 3 modes model example from [4], we show that the corrected ROM reproduces perfectly the original transition trajectory, and generalises well to unseen initial conditions. On-going work is concerned with validating and improving the proposed approach by applying it on the more challenging chaotic pinball case.