Dynamic Load Balancing in Hierarchical Parallel Database Systems
Abstract
We consider the execution of multi-join queries in a hierarchical parallel system, i.e., a shared-nothing system whose nodes are shared-memory multiprocessors. In this context, load balancing must be addressed at two levels, locally among the processors of each shared-memory node and globally among all nodes. In this paper, we propose a dynamic execution model that maximizes local load balancing within shared-memory nodes and minimizes the need for load sharing across nodes. This is obtained by allowing each processor to execute any operator that can be processed locally, thereby taking full advantage of inter- and intra-operator parallelism. We conducted a performance evaluation using an implementation on a 72-processo- r KSR1 computer. The experiments with many queries and large relations show very good speedup results, even with highly skewed data. We show that, in shared-memory, our execution model performs as well as a dedicated model and can scale up very well to deal with multiple nodes.
Domains
Other [cs.OH]
Loading...