Efficient Data and Program Integration Using Binding Patterns
Abstract
In this work, we investigate data and program integration in a fully distribut- ed peer-to-peer mediation architecture. The challenge in making such a system succeed at a large scale is twofold. First, sharing a resource should be easy; therefore, we need a simple concept for modeling resources. Second, we need an efficient architecture for distributed query execution, capable of handling well costly computations and large data transfers. To model heterogeneous resources, we propose using the unified abstraction of table with binding patterns, simple yet powerful enough to capture data and programs. To exploit a resource with restricted binding patterns, we propose an efficient BindJoin operator, following the classical iterator model, in which we build optimization techniques for minimizing large data transfers and costly computations, and maximizing parallelism. Furthermor- e, our BindJoin operator can be tuned to deliver most of its output in the early stages of the execution, which is an important asset in a system meant for human interaction. Our preliminary experimental evaluation validates the proposed BindJoin algorithms, and shows they can provide good performance in queries involving distributed data and expensive programs.