Occlusion resistant learning of intuitive physics from videos - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Preprints, Working Papers, ... Year : 2021

Occlusion resistant learning of intuitive physics from videos


To reach human performance on complex tasks, akey ability for artificial systems is to understandphysical interactions between objects, and predictfuture outcomes of a situation. This ability, of-ten referred to asintuitive physics, has recentlyreceived attention and several methods were pro-posed to learn these physical rules from video se-quences. Yet, most of these methods are restrictedto the case where no, or only limited, occlusionsoccur. In this work we propose a probabilisticformulation of learning intuitive physics in 3Dscenes with significant inter-object occlusions. Inour formulation, object positions are modelledas latent variables enabling the reconstruction ofthe scene. We then propose a series of approx-imations that make this problem tractable. Ob-ject proposals are linked across frames using acombination of a recurrent interaction network,modeling the physics in object space, and a com-positional renderer, modeling the way in whichobjects project onto pixel space. We demonstratesignificant improvements over state-of-the-art inthe intuitive physics benchmark of Riochet et al.(2018). We apply our method to a second datasetwith increasing levels of occlusions, showing itrealistically predicts segmentation masks up to 30frames in the future. Finally, we also show resultson predicting motion of objects in real videos
Fichier principal
Vignette du fichier
2005.00069.pdf (3.17 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03139755 , version 1 (12-02-2021)


  • HAL Id : hal-03139755 , version 1


Ronan Riochet, Josef Sivic, Ivan Laptev, Emmanuel Dupoux. Occlusion resistant learning of intuitive physics from videos. 2021. ⟨hal-03139755⟩
55 View
42 Download


Gmail Mastodon Facebook X LinkedIn More