An analogy based approach for solving target sense verification - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

An analogy based approach for solving target sense verification

Résumé

Contextualized language models have emerged as a de facto standard in natural language processing due to the vast amount of knowledge they acquire during pretraining. Nonetheless, their ability to solve tasks that require reasoning over this knowledge is limited. Certain tasks can be improved by analogical reasoning over concepts, e.g., understanding the underlying relations in "Man is to Woman as King is to Queen". In this work, we propose a way to formulate target sense verification as an analogy detection task, by transforming the input data into quadruples. We present AB4TSV (Analogy and BERT for TSV), a model that uses BERT to represent the objects in these quadruples combined with a convolutional neural network to decide whether they constitute valid analogies. We test our system on the WiC-TSV evaluation benchmark, and show that it can outperform existing approaches. Our empirical study shows the importance of the input encoding for BERT. This dependence gets alleviated by integrating the axiomatic properties of analogies during training, while preserving performance and improving interpretability.
Fichier principal
Vignette du fichier
zervakis_NLPIR22.pdf (757.01 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03792071 , version 1 (29-09-2022)

Identifiants

  • HAL Id : hal-03792071 , version 1

Citer

Georgios Zervakis, Emmanuel Vincent, Miguel Couceiro, Marc Schoenauer, Esteban Marquer. An analogy based approach for solving target sense verification. NLPIR 2022 - 6th International Conference on Natural Language Processing and Information Retrieval, Dec 2022, Bangkok, Thailand. ⟨hal-03792071⟩
349 Consultations
176 Téléchargements

Partager

Gmail Facebook X LinkedIn More