SocialInteractionGAN: Multi-person Interaction Sequence Generation - Inria - Institut national de recherche en sciences et technologies du numérique
Article Dans Une Revue IEEE Transactions on Affective Computing Année : 2022

SocialInteractionGAN: Multi-person Interaction Sequence Generation

Résumé

Prediction of human actions in social interactions has important applications in the design of social robots or artificial avatars. In this paper, we focus on a unimodal representation of interactions and propose to tackle interaction generation in a data-driven fashion. In particular, we model human interaction generation as a discrete multi-sequence generation problem and present SocialInteractionGAN, a novel adversarial architecture for conditional interaction generation. Our model builds on a recurrent encoder-decoder generator network and a dual-stream discriminator, that jointly evaluates the realism of interactions and individual action sequences and operates at different time scales. Crucially, contextual information on interacting participants is shared among agents and reinjected in both the generation and the discriminator evaluation processes. Experiments show that albeit dealing with low dimensional data, SocialInteractionGAN succeeds in producing high realism action sequences of interacting people, comparing favorably to a diversity of recurrent and convolutional discriminator baselines, and we argue that this work will constitute a first stone towards higher dimensional and multimodal interaction generation. Evaluations are conducted using classical GAN metrics, that we specifically adapt for discrete sequential data. Our model is shown to properly learn the dynamics of interaction sequences, while exploiting the full range of available actions.
Fichier principal
Vignette du fichier
SocialInteractionGAN.pdf (4.17 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03163467 , version 1 (09-03-2021)
hal-03163467 , version 2 (09-09-2022)

Identifiants

Citer

Louis Airale, Dominique Vaufreydaz, Xavier Alameda-Pineda. SocialInteractionGAN: Multi-person Interaction Sequence Generation. IEEE Transactions on Affective Computing, 2022, ⟨10.1109/TAFFC.2022.3171719⟩. ⟨hal-03163467v2⟩
279 Consultations
306 Téléchargements

Altmetric

Partager

More