On the (In)Accuracy of Stragglers' Detection in Big Data Analytic Systems - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2024

On the (In)Accuracy of Stragglers' Detection in Big Data Analytic Systems

Résumé

Speculative execution can significantly improve the performance of Big Data applications by launching other copies of stragglers (slow tasks). Stragglers detection plays an important role in the effectiveness of speculative execution. The methods employed to detect stragglers use the information extracted from the last received heartbeats which may be outdated when triggering detection. This, in turn, can mislead Big Data analytic systems to make wrong detection with high inaccuracy. To shed light on this issue, we argue that heartbeat time-stamps should be considered when detecting stragglers. Accordingly, in this paper, we carry out extensive simulations over a large number of scenarios to identify how heartbeat arrival, task starting times, and detection methods impact the accuracy of stragglers detection in Big Data analytic systems. We reveal that the asynchrony in heartbeat arrivals not only result in overlooking real stragglers (false negatives) but can also lead to marking normal tasks as stragglers (false positives). For some state-of-the-art techniques, false positive rates can reach more than 50% on some of our experiments. We also show that time-stamped based detection methods can reduce the average false positive rate from 61 and 64% to 100%.
Fichier non déposé

Dates et versions

hal-04545304 , version 1 (13-04-2024)

Licence

Paternité

Identifiants

  • HAL Id : hal-04545304 , version 1

Citer

Thomas Lambert, Twinkle Jain, Shadi Ibrahim. On the (In)Accuracy of Stragglers' Detection in Big Data Analytic Systems. 2024. ⟨hal-04545304⟩
7 Consultations
0 Téléchargements

Partager

Gmail Facebook X LinkedIn More