We address the problem of large scale image search, for which many recent methods use a bag-of-features image representation. We shows the sub-optimality of such a representation for matching descriptors and derive a more precise representation based on 1) Hamming embedding (HE) and 2) weak geometric consistency constraints (WGC). HE provides binary signatures that refine the matching based on visual words. WGC filters matching descriptors that are not consistent in terms of angle and scale. HE and WGC are integrated within an inverted file system and are efficiently exploited even in the case of very large datasets. Experiments performed on a dataset of one million images show a significant improvement due to the binary signatures and the weak geometric consistency constraints, as well as their efficiency. Estimation of the full geometric transformation, i.e., a re-ranking step on a short list of images, is complementary to our weak geometric consistency constraints and allows to further improve the accuracy.

This is joint work with H. Jegou and M. Douze