Confronting Active Learning for Relation Extraction to a Real-life Scenario on French Newspaper Data - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Confronting Active Learning for Relation Extraction to a Real-life Scenario on French Newspaper Data

Résumé

With recent deep learning advances in natural language processing, tasks such as relation extraction have been solved on benchmark data with near-perfect accuracy. However, in a realistic scenario, such as in a French newspaper company mostly dedicated to local information, relations are of varied, highly specific types, with virtually no data annotated for relations, and many entities co-occur in a sentence without being related. We question the use of supervised state-of-the-art models in such a context, where resources such as time, computing power and human annotators are limited. To adapt to these constraints, we experiment with an active-learning based relation extraction pipeline, consisting of a binary LSTM-based model for detecting the relations that do exist, and a state-of-the-art model for relation classification. We compare several classification models of different depths, from simplistic word embedding averaging, to graph neural networks and Bert-based models, as well as several active learning query strategies, including a proposal for a balanced uncertainty-based strategy, in order to find the most cost-efficient yet accurate approach in our newspaper company’s use case. Our findings highlight the unsuitability of deep models in this data-scarce scenario, as well as the need to further develop data-driven active learning strategies.
Fichier principal
Vignette du fichier
confronting_active_learning_fo.pdf (187.73 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03839438 , version 1 (09-11-2022)

Identifiants

  • HAL Id : hal-03839438 , version 1

Citer

Cyrielle Mallart, Michel Le Nouy, Guillaume Gravier, Pascale Sébillot. Confronting Active Learning for Relation Extraction to a Real-life Scenario on French Newspaper Data. InterNLP 2022 - 2nd Workshop on Interactive Learning for Natural Language Processing, Dec 2022, New Orleans, United States. pp.1-7. ⟨hal-03839438⟩
46 Consultations
75 Téléchargements

Partager

Gmail Facebook X LinkedIn More