Feature Model-Guided Online Reinforcement Learning for Self-Adaptive Services - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2020

Feature Model-Guided Online Reinforcement Learning for Self-Adaptive Services

Abstract

A self-adaptive service can maintain its QoS requirements in the presence of dynamic environment changes. To develop a self-adaptive service, service engineers have to create self-adaptation logic encoding when the service should execute which adaptation actions. However, developing self-adaptation logic may be difficult due to design time uncertainty ; e.g., anticipating all potential environment changes at design time is in most cases infeasible. Online reinforcement learning addresses design time uncertainty by learning suitable adaptation actions through interactions with the environment at runtime. To learn more about its environment, reinforcement learning has to select actions that were not selected before, which is known as exploration. How exploration happens has an impact on the performance of the learning process. We focus on two problems related to how a service's adaptation actions are explored: (1) Existing solutions randomly explore adaptation actions and thus may exhibit slow learning if there are many possible adaptation actions to choose from. (2) Existing solutions are unaware of service evolution, and thus may explore new adaptation actions introduced during such evolution rather late. We propose novel exploration strategies that use feature models (from software product line engineering) to guide exploration in the presence of many adaptation actions and in the presence of service evolution. Experimental results for a self-adaptive cloud management service indicate an average speed-up of the learning process of 58.8% in the presence of many adaptation actions, and of 61.3% in the presence of service evolution. The improved learning performance in turn led to an average QoS improvement of 7.8% and 23.7% respectively.
Fichier principal
Vignette du fichier
ICSOC2020.pdf (548.39 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-02982029 , version 1 (28-10-2020)
hal-02982029 , version 2 (09-01-2021)

Identifiers

  • HAL Id : hal-02982029 , version 2

Cite

Andreas Metzger, Clément Quinton, Zoltan Adam-Mann, Luciano Baresi, Klaus Pohl. Feature Model-Guided Online Reinforcement Learning for Self-Adaptive Services. International Conference on Service Oriented Computing, Dec 2020, Dubai, United Arab Emirates. ⟨hal-02982029v2⟩
293 View
244 Download

Share

Gmail Mastodon Facebook X LinkedIn More