Reducing the Cost of Aggregation in Crowdsourcing - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

Reducing the Cost of Aggregation in Crowdsourcing

Résumé

Crowdsourcing is a way to solve problems that need human contribution. Crowdsourcing platforms distribute replicated tasks to workers, pay them for their contribution, and aggregate answers to produce a reliable conclusion. A fundamental problem is to infer a correct answer from the set of returned results. Another challenge is to obtain a reliable answer at a reasonable cost: unlimited budget allows hiring experts or large pools of workers for each task but a limited budget forces to use resources at best. This paper considers crowdsourcing of simple boolean tasks. We first define a probabilistic inference technique, that considers difficulty of tasks and expertise of workers when aggregating answers. We then propose CrowdInc, a greedy algorithm that reduce the cost needed to reach a consensual answer. CrowdInc distributes resources dynamically to tasks according to their difficulty. We show on several benchmarks that CrowdInc achieves good accuracy, reduces costs, and we compare its performance to existing solutions.
Fichier principal
Vignette du fichier
ICWS2020.pdf (824.32 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02964718 , version 1 (12-10-2020)

Identifiants

Citer

Rituraj Singh, Loïc Hélouët, Zoltan Miklos. Reducing the Cost of Aggregation in Crowdsourcing. Web Services - ICWS 2020 - 27th International Conference, Held as Part of the Services Conference Federation, SCF 2020, Sep 2020, Honolulu, United States. pp.77-95, ⟨10.1007/978-3-030-59618-7⟩. ⟨hal-02964718⟩
65 Consultations
138 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More