FLEX: an Adaptive Exploration Algorithm for Nonlinear Systems - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

FLEX: an Adaptive Exploration Algorithm for Nonlinear Systems

Abstract

Model-based reinforcement learning is a powerful tool, but collecting data to fit an accurate model of the system can be costly. Exploring an unknown environment in a sample-efficient manner is hence of great importance. However, the complexity of dynamics and the computational limitations of real systems make this task challenging. In this work, we introduce FLEX, an exploration algorithm for nonlinear dynamics based on optimal experimental design. Our policy maximizes the information of the next step and results in an adaptive exploration algorithm, compatible with generic parametric learning models and requiring minimal resources. We test our method on a number of nonlinear environments covering different settings, including time-varying dynamics. Keeping in mind that exploration is intended to serve an exploitation objective, we also test our algorithm on downstream model-based classical control tasks and compare it to other state-of-theart model-based and model-free approaches. The performance achieved by FLEX is competitive and its computational cost is low.
Fichier principal
Vignette du fichier
FLEX.pdf (1008.1 Ko) Télécharger le fichier
Origin Publisher files allowed on an open archive

Dates and versions

hal-04176336 , version 1 (02-08-2023)

Identifiers

Cite

Matthieu Blanke, Marc Lelarge. FLEX: an Adaptive Exploration Algorithm for Nonlinear Systems. ICML 2023 - International Conference on Machine Learning, Jul 2023, Honololu, Hawaii, United States. pp.2577-2591. ⟨hal-04176336⟩
41 View
17 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More