Input Sensitivity on the Performance of Configurable Systems: An Empirical Study - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Journal of Systems and Software Year : 2023

Input Sensitivity on the Performance of Configurable Systems: An Empirical Study

Abstract

Widely used software systems such as video encoders are by necessity highly configurable, with hundreds or even thousands of options to choose from. Their users often have a hard time finding suitable values for these options (i.e., finding a proper configuration of the software system) to meet their goals for the tasks at hand, e.g., compress a video down to a certain size. One dimension of the problem is of course that performance depends on the input data: e.g., a video as input to an encoder like x264 or a file fed to a tool like xz . To achieve good performance, users should therefore take into account both dimensions of (1) software variability and (2) input data. This paper details a large study over 8 configurable systems that quantifies the existing interactions between input data and configurations of software systems. The results exhibit that (1) inputs fed to software systems can interact with their configuration options in non-monotonous ways, significantly impacting their performance properties (2) input sensitivity can challenge our knowledge of software variability and question the relevance of performance predictive models for a field deployment. Given the results of our study, we call researchers to address the problem of input sensitivity when tuning, predicting, understanding, and benchmarking configurable systems.
Fichier principal
Vignette du fichier
main.pdf (2.24 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03476464 , version 1 (13-12-2021)
hal-03476464 , version 2 (16-02-2023)

Identifiers

Cite

Luc Lesoil, Mathieu Acher, Arnaud Blouin, Jean-Marc Jézéquel. Input Sensitivity on the Performance of Configurable Systems: An Empirical Study. Journal of Systems and Software, 2023, 201, pp.111671. ⟨10.1016/j.jss.2023.111671⟩. ⟨hal-03476464v2⟩
160 View
266 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More