Communication Dans Un Congrès Année : 2022

Fine-tuning and Sampling Strategies for Multimodal Role Labeling of Entities under Class Imbalance

Résumé

We propose our solution to the multimodal semantic role labeling task from the CON-STRAINT’22 workshop. The task aims at clas-sifying entities in memes into classes such as “hero” and “villain”. We use several pre-trained multi-modal models to jointly encode the text and image of the memes, and implement three systems to classify the role of the entities. We propose dynamic sampling strategies to tackle the issue of class imbalance. Finally, we per-form qualitative analysis on the representations of the entities.

Fichier principal
Vignette du fichier
System_description_paper___MMSRL___ACL_Constraint_workshop.pdf (487.4 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-03840060 , version 1 (14-01-2026)

Licence

Identifiants

Citer

Syrielle Montariol, Étienne Simon, Arij Riabi, Djamé Seddah. Fine-tuning and Sampling Strategies for Multimodal Role Labeling of Entities under Class Imbalance. CONSTRAINT 2022 - Workshop on Combating Online Hostile Posts in Regional Languages during Emergency Situations, May 2022, Dublin, Ireland. pp.55-65, ⟨10.18653/v1/2022.constraint-1.7⟩. ⟨hal-03840060⟩
358 Consultations
92 Téléchargements

Altmetric

Partager

  • More