Robust Visual Sim-to-Real Transfer for Robotic Manipulation - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

Robust Visual Sim-to-Real Transfer for Robotic Manipulation


Learning visuomotor policies in simulation is much safer and cheaper than in the real world. However, due to discrepancies between the simulated and real data, simulator-trained policies often fail when transferred to real robots. One common approach to bridge the visual sim-to-real domain gap is domain randomization (DR). While previous work mainly evaluates DR for disembodied tasks, such as pose estimation and object detection, here we systematically explore visual domain randomization methods and benchmark them on a rich set of challenging robotic manipulation tasks. In particular, we propose an off-line proxy task of cube localization to select DR parameters for texture randomization, lighting randomization, variations of object colors and camera parameters. Notably, we demonstrate that DR parameters have similar impact on our off-line proxy task and on-line policies. We, hence, use off-line optimized DR parameters to train visuomotor policies in simulation and directly apply such policies to a real robot. Our approach achieves 93% success rate on average when tested on a diverse set of challenging manipulation tasks. Moreover, we evaluate the robustness of policies to visual variations in real scenes and show that our simulator-trained policies outperform policies learned using real but limited data. Code, simulation environment, real robot datasets and trained models are available at
Fichier principal
Vignette du fichier
IROS2023_Sim2Real-36.pdf (4.21 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04192660 , version 1 (31-08-2023)
hal-04192660 , version 2 (01-09-2023)



Ricardo Garcia, Robin Strudel, Shizhe Chen, Etienne Arlaud, Ivan Laptev, et al.. Robust Visual Sim-to-Real Transfer for Robotic Manipulation. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct 2023, Detroit, United States. ⟨hal-04192660v2⟩
41 View
27 Download



Gmail Mastodon Facebook X LinkedIn More