SocialInteractionGAN: Multi-person Interaction Sequence Generation - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Preprints, Working Papers, ... Year : 2021

SocialInteractionGAN: Multi-person Interaction Sequence Generation


Prediction of human actions in social interactions has important applications in the design of social robots or artificial avatars. In this paper, we model human interaction generation as a discrete multi-sequence generation problem and present SocialInteractionGAN, a novel adversarial architecture for conditional interaction generation. Our model builds on a recurrent encoder-decoder generator network and a dual-stream discriminator. This architecture allows the discriminator to jointly assess the realism of interactions and that of individual action sequences. Within each stream a recurrent network operating on short subsequences endows the output signal with local assessments, better guiding the forthcoming generation. Crucially, contextual information on interacting participants is shared among agents and reinjected in both the generation and the discriminator evaluation processes. We show that the proposed SocialInteractionGAN succeeds in producing high realism action sequences of interacting people, comparing favorably to a diversity of recurrent and convolutional discriminator baselines. Evaluations are conducted using modified Inception Score and Fréchet Inception Distance metrics, that we specifically design for discrete sequential generated data. The distribution of generated sequences is shown to approach closely that of real data. In particular our model properly learns the dynamics of interaction sequences, while exploiting the full range of actions.
Fichier principal
Vignette du fichier
SocialInteractionGAN.pdf (1.87 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03163467 , version 1 (09-03-2021)
hal-03163467 , version 2 (09-09-2022)



Louis Airale, Dominique Vaufreydaz, Xavier Alameda-Pineda. SocialInteractionGAN: Multi-person Interaction Sequence Generation. 2021. ⟨hal-03163467v1⟩
261 View
279 Download



Gmail Mastodon Facebook X LinkedIn More