Disease in crops is a growing concern for the food and farming industry. In the last couple of decades, lack of immunity in plants and extreme climate conditions due to drastic climate changes have caused a significant increase in the growth of diseases among crops. On a large scale, these diseases cause eventual financial loss to the farmers due to a decrease in farming. Fast and early detection of the disease remains a challenge in most parts of the world because of the lack of robust research infrastructure. Automated techniques for disease detection are high in demand with a worldwide increase in image capturing and video recording devices, together with the continuous evolution in computer vision and machine learning. In this paper, a neural network model, trained on publicly available data which comprise both healthy and leaves with the disease, is proposed. The model adopts a new approach combining the ResNet and the InceptionNet architectures. Skip connections and the $$1 \times 1$$1×1 convolutions in both the architectures are put to good use here. We achieve an accuracy of 99.08% in the PlantVillage dataset [17] and reasonable accuracy of 66.06% on the PlantDoc dataset [16] which is an increment of more than 25% from the approaches in the previous works. This paper also suggests a method to improve the detection of diseases in crops in the real world by augmenting the number of data points. We have discussed the use of deformable convolution, which is capable of learning various geometric transformations and can improve the performance of the architecture.