Implementation and performance of a parallel file system for high performance distributed applications
- 1 January 1996
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10828907,p. 471-480
- https://doi.org/10.1109/hpdc.1996.546218
Abstract
Dedicated cluster parallel computers (DCPCs) are emerging as low-cost high performance environments for many important applications in science and engineering. A significant class of applications that perform well on a DCPC are coarse-grain applications that involve large amounts of file I/O. Current research in parallel file systems for distributed systems is providing a mechanism for adapting these applications to the DCPC environment. We present the Parallel Virtual File System (PVFS), a system that provides disk striping across multiple nodes in a distributed parallel computer and file partitioning among tasks in a parallel program. PVFS is unique among similar systems in that it uses a stream-based approach that represents each file access with a single set of request parameters and decouples the number of network messages from details of the file striping and partitioning. PVFS also provides support for efficient collective file accesses and allows overlapping file partitions. We present results of early performance experiments that show PVFS achieves excellent speedups in accessing moderately sized file segments.Keywords
This publication has 5 references indexed in Scilit:
- Design and implementation of the Vesta parallel file systemPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- PIOUS: a scalable parallel I/O system for distributed computing environmentsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Low-Level Interfaces for High-Level Parallel I/OPublished by Springer Nature ,1996
- Performance Measurement of the Concurrent File System of the Intel iPSC/2 HypercubeJournal of Parallel and Distributed Computing, 1993
- Disk stripingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1986