Comparing images is essential to several computer vision problems, like image retrieval or object identification. The comparison of two images heavily relies on the definition of a good distance function. Standard functions (e.g. the euclidean distance in the original feature space) are too generic and fail to encode the domain specific information. In this paper, we propose to learn a similarity measure specific to a given category (e.g. cars). This distance is learned from a training set of pairs of images labeled “same” or “different”, indicating if the two images represent the same object (e.g. same car model) or not. After learning, this measure is used to predict how similar two images of never seen objects are (see figure 1).