FP-Hadoop - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Logiciel Année : 2019

FP-Hadoop

Miguel Liroz-Gistau
  • Fonction : Auteur
  • PersonId : 901689
Reza Akbarinia
Patrick Valduriez

Résumé

FP-Hadoop makes the reduce side of Hadoop MapReduce more parallel and efficiently deals with the problem of data skew in the reduce side. In FP-Hadoop, there is a new phase, called intermediate reduce (IR), in which blocks of intermediate values, constructed dynamically, are processed by intermediate reduce workers in parallel. Our experiments using FP-Hadoop using synthetic and real benchmarks have shown excellent performance gains compared to native Hadoop, e.g. more than 10 times in reduce time and 5 times in total execution time.

Mots clés

Fichier non déposé

Dates et versions

hal-02093002 , version 1 (08-04-2019)

Identifiants

  • HAL Id : hal-02093002 , version 1

Citer

Miguel Liroz-Gistau, Reza Akbarinia, Patrick Valduriez. FP-Hadoop. 2019. ⟨hal-02093002⟩
97 Consultations
0 Téléchargements

Partager

Gmail Facebook X LinkedIn More