Implementation and performance of a parallel file system for high performance distributed applications
Dedicated Cluster Parallel Computers (DCPCs) are emerging as low-cost high performance environments for many important applications in science and engineering. A significant class of applications that perform well on a DCPC are coarse-grain applications that involve large amounts of file I/O. Current research in parallel file systems for distributed systems is providing a mechanism for adapting these applications to the DCPC environment, We present the Parallel Virtual File System (PVFS), a system that provides disk striping across multiple nodes in a distributed parallel computer and file partitioning among tasks in a parallel program. PUS is unique among similar systems in that it uses a streams-based approach that represents each file access with a single set of request parameters and decouples the number of network messages from details of the files striping and partitioning. PVFS also provides support for efficient collective file accesses and allows overlapping file partitions. We present result so early performance experiments that show PUS achieves excellent speedups in accessing moderately sized file segments.
- OSTI ID:
- 421389
- Report Number(s):
- CONF-960835--; CNN: Grant 5555-20
- Country of Publication:
- United States
- Language:
- English
Similar Records
Bridging the gap between parallel file systems and local file systems : a case study with PVFS.
Experience in implementing a parallel file system