SPIFFI-a scalable parallel file system for the Intel Paragon
- 1 January 1996
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 7 (11) , 1185-1200
- https://doi.org/10.1109/71.544358
Abstract
This paper presents the design and performance of SPIFFI, a scalable high-performance parallel file system intended for use by extremely I/O intensive applications including "Grand Challenge" scientific applications and multimedia systems. This paper contains experimental results from a SPIFFI prototype on a 64 node/64 disk Intel Paragon. The results show that SPIFFI provides high performance and linear scaleup on real hardware. The paper also explains how shared file pointers (i.e., file pointers that are shared by multiple processes) can simplify the design of a parallel application. By sequentializing I/O accesses and by providing dynamic I/O load balancing, a shared file pointer may even improve an application's performance.This paper also presents the predictions of a SPIFFI simulator that we validated using the prototype. The simulator results show that SPIFFI continues to provide high performance even when it is scaled to configurations with as many as 128 disks or 256 compute nodes.Keywords
This publication has 35 references indexed in Scilit:
- Concurrent file operations in a high performance FORTRANPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- Performance characteristics of the iPSC/860 and CM-2 I/O systemsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- RAMA: a file system for massively-parallel computersPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Multiprocessor file system interfacesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Dynamic file-access characteristics of a production parallel scientific workloadPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Maximizing performance in a striped disk arrayPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Disk-directed I/O for MIMD multiprocessorsACM Transactions on Computer Systems, 1997
- Practical prefetching techniques for multiprocessor file systemsDistributed and Parallel Databases, 1993
- Input/output behavior of supercomputing applicationsPublished by Association for Computing Machinery (ACM) ,1991
- Design and implementation of the wisconsin storage systemSoftware: Practice and Experience, 1985