OpenBenchmark: Repeatable and Reproducible Internet of Things Experimentation on Testbeds - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2019

OpenBenchmark: Repeatable and Reproducible Internet of Things Experimentation on Testbeds

Abstract

Experimentation on testbeds with Internet of Things (IoT) devices is hard. The tedious firmware development, the lack of user interfaces, the stochastic nature of the radio channel, the testbed learning curve, are some of the factors that make the evaluation process error prone. The impact of such errors on published results can be quite unfortunate, leading to misconclusions and false common wisdom. The selection of experiment conditions or performance metrics to evaluate one's own proposal may not lead to perfectly fair comparisons with state-of-the-art, either. Our research community is well aware of these problems and is actively working on solutions. We present OpenBenchmark, a cloud-based, reproducible, repeatable and comparable IoT benchmarking service. OpenBenchmark facilitates and improves the IoT experimentation workflow: it runs the experiments on supported testbeds, instruments the supported firmware according to the industry-relevant test scenarios, and collects and processes the experiment data to produce Key Performance Indicators (KPIs). This paper introduces the OpenBenchmark platform, discusses its applicability, design and implementation.
Fichier principal
Vignette du fichier
vucinic19openbenchmark.pdf (351.35 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-02266556 , version 1 (14-08-2019)

Identifiers

  • HAL Id : hal-02266556 , version 1

Cite

Malisa Vucinic, Bozidar Skrbic, Enis Kocan, Milica Pejanovic-Djurisic, Thomas Watteyne. OpenBenchmark: Repeatable and Reproducible Internet of Things Experimentation on Testbeds. CNERT - IEEE INFOCOM Workshop on Computer and Networking Experimental Research using Testbeds, Apr 2019, Paris, France. ⟨hal-02266556⟩
135 View
242 Download

Share

Gmail Facebook X LinkedIn More