Exploring Category Structure with Contextual Language Models and Lexical Semantic Networks - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Exploring Category Structure with Contextual Language Models and Lexical Semantic Networks

Résumé

Recent work on predicting category structure with distributional models, using either static word embeddings (Heyman and Heyman, 2019) or contextualized language models (CLMs) (Misra et al., 2021), report low correlations with human ratings, thus calling into question their plausibility as models of human semantic memory. In this work, we revisit this question testing a wider array of methods for probing CLMs for predicting typicality scores. Our experiments, using BERT (Devlin et al., 2018), show the importance of using the right type of CLM probes, as our best BERT-based typicality prediction methods substantially improve over previous works. Second, our results highlight the importance of polysemy in this task: our best results are obtained when using a disambiguation mechanism. Finally, additional experiments reveal that Information Contentbased WordNet (Miller, 1995), also endowed with disambiguation, match the performance of the best BERT-based method, and in fact capture complementary information, which can be combined with BERT to achieve enhanced typicality predictions.
Fichier principal
Vignette du fichier
Typicality_EACL__Final_-2.pdf (800.08 Ko) Télécharger le fichier
eacl2023.pdf (108.79 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03986142 , version 1 (13-02-2023)

Identifiants

Citer

Joseph Renner, Pascal Denis, Rémi Gilleron, Angèle Brunellière. Exploring Category Structure with Contextual Language Models and Lexical Semantic Networks. EACL 2023 - 17th Conference of the European Chapter of the Association for Computational Linguistics, May 2023, Dubrovnik, Croatia. ⟨hal-03986142⟩
38 Consultations
143 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More