In the context of detecting changes in structural systems, multiple vibration-based damage detection methods have been proposed and successfully applied to both mechanical and civil structures over the past years. One of the popular schemes is based on a robust subspace-based residual and enjoys favorable statistical and computational properties, like invariance to changes in the excitation covariance and numerical stability. This paper presents an alternative Gaussian residual that is based on the difference of normalized Hankel matrices between reference and damaged states, which can be easily computed. The statistical properties of the residual are reported and used for efficient hypothesis testing. Its robustness to excitation changes is shown. The proposed scheme is evaluated in numerical simulations, validating its robustness, and tested on real data sets from a full scale bridge.