A Fully Decentralized Autoscaling Algorithm for Stream Processing Applications - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2019

A Fully Decentralized Autoscaling Algorithm for Stream Processing Applications

Résumé

Stream Processing deals with the efficient, real-time processing of continuous streams of data. Stream Processing engines ease the development and deployment of such applications which are commonly pipelines of operators to be traversed by each data item. Due to the varying velocity of the streams, autoscaling is needed to dynamically adapt the number of instances of each operator. With the advent of geographically-dispersed computing platforms such as Fog platforms, operators are dispersed accordingly, and autoscaling needs to be decentralized as well. In this paper, we propose an algorithm allowing for scaling decisions to be taken and enforced in a fully-decentralized way. In particular, in spite of scaling actions being triggered concurrently, each operator maintains a view of its neighbours in the graph so as no data message is lost. The protocol is detailed and its correctness discussed. Its performance is captured through early simulation experiments.
Fichier principal
Vignette du fichier
autodasp2019.pdf (540.87 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02171172 , version 1 (02-07-2019)

Identifiants

  • HAL Id : hal-02171172 , version 1

Citer

Mehdi Belkhiria, Cédric Tedeschi. A Fully Decentralized Autoscaling Algorithm for Stream Processing Applications. Auto-DaSP 2019 - Third International Workshop on Autonomic Solutions for Parallel and Distributed Data Stream Processing, Aug 2019, Göttingen, Germany. pp.1-12. ⟨hal-02171172⟩
155 Consultations
326 Téléchargements

Partager

Gmail Facebook X LinkedIn More