Clusterfile: a flexible physical layout parallel file system
- 1 January 2001
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
Abstract
This paper presents Clusterfile, a parallel file system that provides parallel file access on a cluster of computers. Existing parallel file systems offer little control over matching the I/O access patterns and file data layout. Without this matching the applications may face the following problems: contention at I/O nodes, fragmentation of file data, false sharing, small network messages, high overhead of scattering/ gathering the data. Clusterfile addresses some of these inefficiencies. Parallel applications can physically partition a file in arbitrary patterns. They can also set arbitrary views on a file. Views hide the parallel structure of the file and ease the programmer’s burden of computing complex access indices. The intersections between views and layouts are computed by a memory redistribution algorithm. Read and write operations are optimized by pre-computing the direct mapping between access patterns and disks. Clusterfile uses the same data representation for file layouts, access patterns, and the mappings between each other.Keywords
This publication has 7 references indexed in Scilit:
- A Comparison of Logical and Physical Parallel I/o pAtternsThe International Journal of High Performance Computing Applications, 1998
- The Galley parallel file systemParallel Computing, 1997
- File-access characteristics of parallel scientific workloadsIEEE Transactions on Parallel and Distributed Systems, 1996
- PPFSPublished by Association for Computing Machinery (ACM) ,1995
- Parallel file systems for the IBM SP computersIBM Systems Journal, 1995
- High performance FortranIEEE Parallel & Distributed Technology: Systems & Applications, 1993
- nCUBE parallel I/O softwarePublished by Institute of Electrical and Electronics Engineers (IEEE) ,1992