This paper addresses the problem of aligning two unsynchronized video sequences. We present a novel approach that allows for temporal and spa- tial alignment of similar videos captured from independently moving cameras. The goal is to synchronize two videos of a scene such that changes between the videos can be detected automatically. This aims at applications in driver assis- tance or surveillance systems but we also envision applications in map building. Our approach is novel in that it adapts an efficient information retrieval frame- work to a computer vision problem. In addition, we extend the recent ECC image- alignment algorithm to the temporal dimension in order to improve spatial regis- tration and enable synchro refinement. Experiments with traffic videos recorded by in-vehicle cameras demonstrate the efficiency of the proposed method and verify its effectiveness with respect to spatio-temporal alignment accuracy.