Along with the aging of the Japanese agricultural population in recent years, the nation’s food self-sufficiency rate has declined. The vitality of agriculture has also declined. This study specifically examines the food supply chain to integrate farm production, manufacturing, and sales. For the food supply chain, it is necessary to ascertain the harvest time of field crops for establishing proper sales strategies. A method to predict harvest time from field crop images using convolutional neural networks (CNN) is proposed. Then the effectiveness of the proposed method is verified using computer experiments. Results show that the discrimination rate is high in the early stage of the growth, even if misclassification occurs. Labels close to the correct answer are predicted. However, the discrimination rate is not so high and the deviation from the correct answer becomes greater in later stages of growth.