An Elastic and Scalable Topic-Based Pub/Sub System Using Deep Reinforcement Learning - Inria - Institut national de recherche en sciences et technologies du numérique
Communication Dans Un Congrès Année : 2022

An Elastic and Scalable Topic-Based Pub/Sub System Using Deep Reinforcement Learning

Thanos Giannakopoulos
  • Fonction : Auteur
  • PersonId : 1475951
Vana Kalogeraki
  • Fonction : Auteur
  • PersonId : 998192

Résumé

The ability to handle large volumes of event data and react to unexpected spikes, in real-time, remains an important challenge in stream processing systems, such as Apache Kafka, due to the amount of custom coding and technical expertise required to configure these systems. In this paper we investigate the use of reinforcement learning as a promising approach to address these issues. By feeding the machine learning technique with system performance metrics under a wide variety of configurations, we can effectively address any changes in the pub/sub system or overload situations while maintaining the desired performance goals. We implement our methodology on the Kafka pub/sub system without any changes in the application logic. Our experimental results illustrate the performance and benefits of our approach.
Fichier principal
Vignette du fichier
523617_1_En_11_Chapter.pdf (1.73 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04827161 , version 1 (09-12-2024)

Licence

Identifiants

Citer

Thanos Giannakopoulos, Vana Kalogeraki. An Elastic and Scalable Topic-Based Pub/Sub System Using Deep Reinforcement Learning. 17th IFIP International Conference on Distributed Applications and Interoperable Systems (DAIS), Jun 2022, Lucca, Italy. pp.167-183, ⟨10.1007/978-3-031-16092-9_11⟩. ⟨hal-04827161⟩
0 Consultations
0 Téléchargements

Altmetric

Partager

More