Gamification for Word Sense Labeling - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2013

Gamification for Word Sense Labeling

Abstract

Obtaining gold standard data for word sense disambiguation is important but costly. We show how it can be done using a " Game with a Purpose " (GWAP) called Wordrobe. This game consists of a large set of multiple-choice questions on word senses generated from the Groningen Meaning Bank. The players need to answer these questions, scoring points depending on the agreement with fellow players. The working assumption is that the right sense for a word can be determined by the answers given by the players. To evaluate our method, we gold-standard tagged a portion of the data that was also used in the GWAP. A comparison yielded promising results, ranging from a precision of 0.88 and recall of 0.83 for relative majority agreement, to a precision of 0.98 and recall of 0.35 for questions that were answered unanimously.
Fichier principal
Vignette du fichier
W13-0215.pdf (164.09 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01342431 , version 1 (06-07-2016)

Identifiers

  • HAL Id : hal-01342431 , version 1

Cite

Noortje Venhuizen, Kilian Evang, Valerio Basile, Johan Bos. Gamification for Word Sense Labeling. Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013), Mar 2013, Potsdam, Germany. ⟨hal-01342431⟩
280 View
340 Download

Share

Gmail Facebook X LinkedIn More