Identifying Redundancy and Exposing Provenance in Crowdsourced Data Analysis - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles IEEE Transactions on Visualization and Computer Graphics Year : 2013

Identifying Redundancy and Exposing Provenance in Crowdsourced Data Analysis

Wesley Willett
  • Function : Correspondent author
  • PersonId : 944334

Connectez-vous pour contacter l'auteur
Shiry Ginosar
  • Function : Author
  • PersonId : 945895
Avital Steinitz
  • Function : Author
  • PersonId : 945896
Björn Hartmann
  • Function : Author
  • PersonId : 945897
Maneesh Agrawala
  • Function : Author
  • PersonId : 945898


We present a system that lets analysts use paid crowd workers to explore data sets and helps analysts interactively examine and build upon workers' insights. We take advantage of the fact that, for many types of data, independent crowd workers can readily perform basic analysis tasks like examining views and generating explanations for trends and patterns. However, workers operating in parallel can often generate redundant explanations. Moreover, because workers have different competencies and domain knowledge, some responses are likely to be more plausible than others. To efficiently utilize the crowd's work, analysts must be able to quickly identify and consolidate redundant responses and determine which explanations are the most plausible. In this paper, we demonstrate several crowd-assisted techniques to help analysts make better use of crowdsourced explanations: (1) We explore crowd-assisted strategies that utilize multiple workers to detect redundant explanations. We introduce color clustering with representative selection--a strategy in which multiple workers cluster explanations and we automatically select the most-representative result--and show that it generates clusterings that are as good as those produced by experts. (2) We capture explanation provenance by introducing highlighting tasks and capturing workers' browsing behavior via an embedded web browser, and refine that provenance information via source-review tasks. We expose this information in an explanation-management interface that allows analysts to interactively filter and sort responses, select the most plausible explanations, and decide which to explore further.
Vignette du fichier
CrowdAnalytics-VAST-Thumb.png (18.25 Ko) Télécharger le fichier Fichier principal
Vignette du fichier
CrowdAnalytics-VAST2013-FinalCameraReady.pdf (1.78 Mo) Télécharger le fichier
Format : Figure, Image
Origin : Files produced by the author(s)

Dates and versions

hal-00850827 , version 1 (11-10-2013)



Wesley Willett, Shiry Ginosar, Avital Steinitz, Björn Hartmann, Maneesh Agrawala. Identifying Redundancy and Exposing Provenance in Crowdsourced Data Analysis. IEEE Transactions on Visualization and Computer Graphics, 2013, 19 (12), pp.2198 - 2206. ⟨10.1109/TVCG.2013.164⟩. ⟨hal-00850827⟩
534 View
333 Download



Gmail Facebook Twitter LinkedIn More