Hierarchical Multimodal Attention for Deep Video Summarization - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

Hierarchical Multimodal Attention for Deep Video Summarization


The way people consume sports on TV has drastically evolved in the last years, particularly under the combined effects of the legalization of sport betting and the huge increase of sport analytics. Several companies are nowadays sending observers in the stadiums to collect live data of all the events happening on the field during the match. Those data contain meaningful information providing a very detailed description of all the actions occurring during the match to feed the coaches and staff, the fans, the viewers, and the gamblers. Exploiting all these data, sport broadcasters want to generate extra content such as match highlights, match summaries, players and teams analytics, etc., to appeal subscribers. This paper explores the problem of summarizing professional soccer matches as automatically as possible using both the aforementioned event-stream data collected from the field and the content broadcasted on TV. We have designed an architecture, introducing first (1) a Multiple Instance Learning method that takes into account the sequential dependency among events and then (2) a hierarchical multimodal attention layer that grasps the importance of each event in an action. We evaluate our approach on matches from two professional European soccer leagues, showing its capability to identify the best actions for automatic summarization by comparing with real summaries made by human operators.
Fichier principal
Vignette du fichier
ICPR2020_Hierarchical_Multimodal_Attention_for_Deep_Video_Summarization.pdf (390.72 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-02964209 , version 1 (12-10-2020)



Melissa Sanabria, Frédéric Precioso, Thomas Menguy. Hierarchical Multimodal Attention for Deep Video Summarization. ICPR 2020 - 25th International Conference on Pattern Recognition, Jan 2021, Milan / Virtual, Italy. ⟨10.1109/ICPR48806.2021.9413097⟩. ⟨hal-02964209⟩
201 View
481 Download



Gmail Mastodon Facebook X LinkedIn More