Learning Actionness via Long-range Temporal Order Verification - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2020

Learning Actionness via Long-range Temporal Order Verification


Current methods for action recognition typically rely on supervision provided by manual labeling. Such methods, however, do not scale well given the high burden of manual video annotation and a very large number of possible actions. The annotation is particularly difficult for temporal action localization where large parts of the video present no action, or background. To address these challenges, we here propose a self-supervised and generic method to isolate actions from their background. We build on the observation that actions often follow a particular temporal order and, hence, can be predicted by other actions in the same video. As consecutive actions might be separated by minutes, differently to prior work on the arrow of time, we here exploit long-range temporal relations in 10-20 minutes long videos. To this end, we propose a new model that learns actionness via a self-supervised proxy task of order verification. The model assigns high actionness scores to clips which order is easy to predict from other clips in the video. To obtain a powerful and action-agnostic model, we train it on the large-scale unlabeled HowTo100M dataset with highly diverse actions from instructional videos. We validate our method on the task of action localization and demonstrate consistent improvements when combined with other recent weakly-supervised methods.
Fichier principal
Vignette du fichier
paper.pdf (4.25 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03048753 , version 1 (09-12-2020)


  • HAL Id : hal-03048753 , version 1


Dimitri Zhukov, Jean-Baptiste Alayrac, Ivan Laptev, Josef Sivic. Learning Actionness via Long-range Temporal Order Verification. ECCV 2020 - European Conference on Computer Vision, Aug 2020, Glasgow / Virtual, United Kingdom. ⟨hal-03048753⟩
112 View
260 Download


Gmail Facebook X LinkedIn More