Research and Application of Spark Platform on Big Data Processing in Intelligent Agriculture of Jilin Province - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2019

Research and Application of Spark Platform on Big Data Processing in Intelligent Agriculture of Jilin Province

Siwei Fu
  • Fonction : Auteur
  • PersonId : 1046181
Guifen Chen
  • Fonction : Auteur
  • PersonId : 972131

Résumé

Aiming at the demand of real-time massive data processing of Intelligent Agriculture in Jilin Province, this paper studies the big data processing of Intelligent Agriculture in Jilin Province based on Spark platform by acquiring real-time data through monitoring platform. This study first conducted the performance comparison experiment of Hadoop and Spark data processing platform, then used the Spark distributed cluster computing platform, real-time processing the big data of monitoring area. The experimental results show that the Spark platform speeds up 11.4 times faster than the Hadoop platform in the case of 100 million data sizes; and based on the Spark platform for real-time processing of big data intelligent agricultural monitoring network, not only provides memory calculations to reduce IO overhead, but also the results are faster and more accurate. The research results provide strong support for the implementation of precision agriculture technology in intelligent agriculture.
Fichier principal
Vignette du fichier
478293_1_En_1_Chapter.pdf (604.74 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02111532 , version 1 (26-04-2019)

Licence

Paternité

Identifiants

Citer

Siwei Fu, Guifen Chen, Shan Zhao, Enze Xiao. Research and Application of Spark Platform on Big Data Processing in Intelligent Agriculture of Jilin Province. 11th International Conference on Computer and Computing Technologies in Agriculture (CCTA), Aug 2017, Jilin, China. pp.1-12, ⟨10.1007/978-3-030-06179-1_1⟩. ⟨hal-02111532⟩
49 Consultations
40 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More