An Extensible Framework for Data Cleaning - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Reports (Research Report) Year : 1999

An Extensible Framework for Data Cleaning

Abstract

Data integration solutions dealing with large amounts of data have been strongly required in the last few years. Besides the traditional data integration problems (e.g. schema integration, local to global schema mappings), three additional data problems have to be dealt with: (1) the absence of universal keys across different databases that is known as the object identity problem, (2) the existence of keyborad errors in the data, and (3) the presence of inconsistencies in data coming from multiple sources. Dealing with these problems is globally called the data cleaning process. In this work, we propose a framework which offers the fundamental services required by this process: data transformation, duplicate elimination and multi-table matching. These services are implemented using a set of purposely designed macro-operators. Moreover, we propose an SQL extension for specifying each of the macro-operators. One important feature of the framework is the ability of explicitly including the human interaction in the process. The main novelty of the work is that the framework permits the following performance optimizations which are tailored for data cleaning applications: mixed evaluation, neighborhood hash join, decision push-down and short-circuited computation. We measure the benefits of each.
Fichier principal
Vignette du fichier
RR-3742.pdf (563.56 Ko) Télécharger le fichier

Dates and versions

inria-00072922 , version 1 (24-05-2006)

Identifiers

  • HAL Id : inria-00072922 , version 1

Cite

Helena Galhardas, Daniela Florescu, Dennis Shasha, Eric Simon. An Extensible Framework for Data Cleaning. [Research Report] RR-3742, INRIA. 1999. ⟨inria-00072922⟩
670 View
986 Download

Share

Gmail Facebook X LinkedIn More