A robust control-theory-based exploration strategy in deep reinforcement learning for virtual network embedding - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Computer Networks Year : 2022

A robust control-theory-based exploration strategy in deep reinforcement learning for virtual network embedding

Abstract

Network slice management and, more generally, resource orchestration should be fully automated in 6G networks, as envisioned by the ETSI ENI. In this context, artificial intelligence (AI) and context-aware policies are certainly major options to move in this direction and to adapt service delivery to changing user needs, environmental conditions and business objectives. In this paper, we step towards this objective by addressing the problem of optimal placement of dynamic virtual networks through a self-adaptive learning-based strategy. These constantly evolving networks present, however, several challenges, mainly due to their stochastic nature, and the high dimensionality of the state and the action spaces. This curse of dimensionality requires, indeed, a broader exploration, which is not always compatible with a real-time execution in an operational network. Thus, we propose DQMC, a new strategy for virtual network embedding in mobile networks combining a Deep Reinforcement Learning (DRL) strategy, namely a Deep Q-Network (DQN), and Monte Carlo (MC). As learning is costly in time and computing resources, and sensitive to changes in the network, we suggest a control-theory-based techniques to dynamically leverage exploration in DQMC. This leads to fast, efficient, and sober learning compared to a Monte Carlo-based strategy. This also ensures a reliable solution even in the case of a change in the requests' sizes or a node's failure, showing promising perspectives for solutions combining control-theory and machine learning.
Fichier principal
Vignette du fichier
Control_Learning_SDN-1.pdf (1.06 Mo) Télécharger le fichier
Published 2022 Computer Networks - A robust control-theory-based exploration strategy in deep reinforcementlearning for virtual network embedding.pdf (1.69 Mo) Télécharger le fichier
Origin Files produced by the author(s)
Origin Files produced by the author(s)

Dates and versions

hal-03792078 , version 1 (30-09-2022)

Identifiers

Cite

Ghina Dandachi, Sophie Cerf, Yassine Hadjadj-Aoul, Abdelkader Outtagarts, Eric Rutten. A robust control-theory-based exploration strategy in deep reinforcement learning for virtual network embedding. Computer Networks, 2022, 218, pp.1-27. ⟨10.1016/j.comnet.2022.109366⟩. ⟨hal-03792078⟩
123 View
132 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More