Learning Distributed and Fair Policies for Network Load Balancing as Markov Potential Game - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Learning Distributed and Fair Policies for Network Load Balancing as Markov Potential Game

Résumé

This paper investigates the network load balancing problem in data centers (DCs) where multiple load balancers (LBs) are deployed, using the multi-agent reinforcement learning (MARL) framework. The challenges of this problem consist of the heterogeneous processing architecture and dynamic environments, as well as limited and partial observability of each LB agent in distributed networking systems, which can largely degrade the performance of in-production load balancing algorithms in real-world setups. Centralised-training-decentralised-execution (CTDE) RL scheme has been proposed to improve MARL performance, yet it incurs -- especially in distributed networking systems, which prefer distributed and plug-and-play design scheme -- additional communication and management overhead among agents. We formulate the multi-agent load balancing problem as a Markov potential game, with a carefully and properly designed workload distribution fairness as the potential function. A fully distributed MARL algorithm is proposed to approximate the Nash equilibrium of the game. Experimental evaluations involve both an event-driven simulator and real-world system, where the proposed MARL load balancing algorithm shows close-to-optimal performance in simulations, and superior results over in-production LBs in the real-world system.
Fichier principal
Vignette du fichier
main.pdf (2.39 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04116297 , version 1 (03-06-2023)

Licence

Paternité - Pas d'utilisation commerciale

Identifiants

Citer

Zhiyuan Yao, Zihan Ding. Learning Distributed and Fair Policies for Network Load Balancing as Markov Potential Game. 36th Conference on Neural Information Processing Systems (NeurIPS 2022)., Nov 2022, New Orleans (LA), United States. ⟨hal-04116297⟩

Collections

X IP_PARIS
7 Consultations
153 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More