Semi-supervised Emotion Recognition using Inconsistently Annotated Data - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2020

Semi-supervised Emotion Recognition using Inconsistently Annotated Data


Expression recognition remains challenging, predominantly due to (a) lack of sufficient data, (b) subtle emotion intensity, (c) subjective and inconsistent annotation, as well as due to (d) in-the-wild data containing variations in pose, intensity, and occlusion. To address such challenges in a unified framework, we propose a self-training based semi-supervised convolutional neural network (CNN) framework, which directly addresses the problem of (a) limited data by leveraging information from unannotated samples. Our method uses 'successive label smoothing' to adapt to the subtle expressions and improve the model performance for (b) low-intensity expression samples. Further, we address (c) inconsistent annotations by assigning sample weights during loss computation, thereby ignoring the effect of incorrect ground-truth. We observe significant performance improvement in in-the-wild datasets by leveraging the information from the in-the-lab datasets, related to challenge (d). Associated to that, experiments on four publicly available datasets demonstrate large performance gains in cross-database performance, as well as show that the proposed method achieves to learn different expression intensities, even when trained with categorical samples.
Fichier principal
Vignette du fichier
happy-FG20.pdf (584.19 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-02969840 , version 1 (16-10-2020)


  • HAL Id : hal-02969840 , version 1


S L Happy, Antitza Dantcheva, Francois F Bremond. Semi-supervised Emotion Recognition using Inconsistently Annotated Data. FG 2020 - 15th IEEE International Conference on Automatic Face and Gesture Recognition, Nov 2020, Buenos Aires / Virtual, Argentina. ⟨hal-02969840⟩
52 View
279 Download


Gmail Facebook X LinkedIn More