Analogies minus analogy test: measuring regularities in word embeddings - Inria - Institut national de recherche en sciences et technologies du numérique
Conference Papers Year : 2020

Analogies minus analogy test: measuring regularities in word embeddings

Abstract

Vector space models of words have long been claimed to capture linguistic regularities as simple vector translations, but problems have been raised with this claim. We decompose and empirically analyze the classic arithmetic word analogy test, to motivate two new metrics that address the issues with the standard test, and which distinguish between class-wise offset concentration (similar directions between pairs of words drawn from different broad classes, such as France-London, China-Ottawa,. . .) and pairing consistency (the existence of a regular transformation between correctly-matched pairs such as France:Paris::China:Beijing). We show that, while the standard analogy test is flawed, several popular word embeddings do nevertheless encode linguistic regularities.
Fichier principal
Vignette du fichier
2010.03446.pdf (825.01 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03070260 , version 1 (15-12-2020)

Identifiers

Cite

Louis Fournier, Emmanuel Dupoux, Ewan Dunbar. Analogies minus analogy test: measuring regularities in word embeddings. CoNLL 2020 - 24th Conference on Computational Natural Language Learning, Nov 2020, Virtual, France. ⟨hal-03070260⟩
50 View
78 Download

Altmetric

Share

More