Quality Assessment of Crowdwork via Eye Gaze: Towards Adaptive Personalized Crowdsourcing - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2021

Quality Assessment of Crowdwork via Eye Gaze: Towards Adaptive Personalized Crowdsourcing

Md. Rabiul Islam
  • Fonction : Auteur
  • PersonId : 1280655
Shun Nawa
  • Fonction : Auteur
  • PersonId : 1280656
Andrew Vargo
  • Fonction : Auteur
  • PersonId : 1280657
Motoi Iwata
  • Fonction : Auteur
  • PersonId : 1280658
Koichi Kise
  • Fonction : Auteur
  • PersonId : 1280660

Résumé

A significant challenge for creating efficient and fair crowdsourcing platforms is in rapid assessment of the quality of crowdwork. If a crowdworker lacks the skill, motivation, or understanding to provide adequate quality task completion, this reduces the efficacy of a platform. While this would seem like only a problem for task providers, the reality is that the burden of this problem is increasingly leveraged on crowdworkers. For example, task providers may not pay crowdworkers for their work after the evaluation of the task results has been completed. In this paper, we propose methods for quickly evaluating the quality of crowdwork using eye gaze information by estimating the correct answer rate. We find that the method with features generated by self-supervised learning (SSL) provides the most efficient result with a mean absolute error of 0.09. The results exhibit the potential of using eye gaze information to facilitate adaptive personalized crowdsourcing platforms.
Fichier principal
Vignette du fichier
520516_1_En_8_Chapter.pdf (1.79 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04196869 , version 1 (05-09-2023)

Licence

Paternité

Identifiants

Citer

Md. Rabiul Islam, Shun Nawa, Andrew Vargo, Motoi Iwata, Masaki Matsubara, et al.. Quality Assessment of Crowdwork via Eye Gaze: Towards Adaptive Personalized Crowdsourcing. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.104-113, ⟨10.1007/978-3-030-85616-8_8⟩. ⟨hal-04196869⟩
13 Consultations
5 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More