We propose a new supervised object retrieval method based on the selection of local visual features learned with the BLasso algorithm. BLasso is a boosting-like procedure that efficiently approximates the Lasso path through backward regularization steps. The advantage compared to a classical boosting strategy is that it produces a sparser selection of visual features. This allows us to improve the efficiency of the retrieval and, as discussed in the paper, it facilitates human visual interpretation of the models generated. We carried out our experiments on the Caltech-256 dataset with state-of-the-art local visual features. We show that our method outperforms AdaBoost in effectiveness while significantly reducing the model complexity and the prediction time. We discuss the evaluation of the visual models obtained in terms of human interpretability.