Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Nonlinear Analysis: Hybrid Systems Year : 2024

Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems

Abstract

Optimal Control (OC) algorithms such as Differential Dynamic Programming (DDP) take advantage of the derivatives of the dynamics to efficiently control physical systems. Yet, in the presence of nonsmooth dynamical systems, such class of algorithms are likely to fail due, for instance, to the presence of discontinuities in the dynamics derivatives or because of non-informative gradient during the solving. On the contrary, Reinforcement Learning (RL) algorithms have shown better empirical results in scenarios exhibiting nonsmooth effects (contacts, frictions, etc). Our approach leverages recent works on Randomized Smoothing (RS) to tackle nonsmoothness issues commonly encountered in Optimal Control, and provides key insights on the interplay between RL and OC through the prism of RS methods. This naturally leads us to introduce the Randomized Differential Dynamic Programming (R-DDP) algorithm accounting for deterministic but non-smooth dynamics in a very sample-efficient way. The experiments demonstrate that our method is able to solve classic robotic problems with dry friction and frictional contacts, where classical OC algorithms are likely to fail and RL algorithms require in practice a prohibitive number of samples to find an optimal solution.
Fichier principal
Vignette du fichier
lelidec2023randomized.pdf (3.59 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03480419 , version 1 (14-12-2021)
hal-03480419 , version 2 (08-03-2022)
hal-03480419 , version 3 (11-03-2022)
hal-03480419 , version 4 (22-01-2024)

Identifiers

Cite

Quentin Le Lidec, Fabian Schramm, Louis Montaut, Cordelia Schmid, Ivan Laptev, et al.. Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems. Nonlinear Analysis: Hybrid Systems, 2024, 52, pp.101468. ⟨10.1016/j.nahs.2024.101468⟩. ⟨hal-03480419v4⟩
224 View
73 Download

Altmetric

Share

Gmail Facebook X LinkedIn More