Diseases and pests in fruit trees is a key factor influencing the quality and quantity of fruit. Image retrieval technology into diseases and pests detection is an important way to improve the fruit tree management level and the quality and quantity of the fruit. This paper reviews the content-based image retrieval technology and the application of technology in the aspect of plant diseases and insect pests and proposed pest detection methods, steps and future research directions.