The Architecture of BPFS: a Basic Parallel File System Version 1.0
Abstract
BPFS is a distributed, modular parallel file system designed to be used on networks of workstations. It is specified as a set of active components, functions utilized by the components, and protocols for communication between the components. The components can be implemented in many different ways, depending on the hardware and software support systems available, the performance desired, etc. A key idea is to be able to experiment with different implementations and configurations under different operating conditions to achieve a completely general, flexible system that is also capable of delivering good performance. In particular, it should be possible to implement this system on «commodity, off-the-shelf» (COTS) hardware and software. However, it should also be possible to implement specialized versions of some or all components to take advantage of unique hardware or software features. BPFS is intended to support a wide range of possible applications, including real-time video on demand, medical and satellite image processing, out-of-core array manipulations, and general parallel computations that need a high performance file system. The parallel file system is intended to be «always available» for simultaneous use by any number of different applications. It is also capable of efficiently handling huge files containing many terabytes of data. This report describes the architecture of BPFS in terms of the set of components, their organization, the functions they utilize, and the protocol specifications for communication between them.