Noncontiguous I/O through PVFS
- 26 June 2003
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
Abstract
With the tremendous advances in processor and memory technology, I/O has risen to become the bottleneck in high-performance computing for many applications. The development of parallel file systems has helped to ease the performance gap, but I/O still remains an area needing significant performance improvement. Research has found that noncontiguous I/O access patterns in scientific applications combined with current file system methods, to perform these accesses lead to unacceptable performance for large data sets. To enhance performance of noncontiguous I/O, we have created list I/O, a native version of noncontiguous I/O. We have used the Parallel Virtual File System (PVFS) to implement our ideas. Our research and experimentation shows that list I/O outperforms current noncontiguous I/O access methods in most I/O situations and can substantially enhance the performance of real-world scientific applications.Keywords
All Related Versions
This publication has 7 references indexed in Scilit:
- FLASH: An Adaptive Mesh Hydrodynamics Code for Modeling Astrophysical Thermonuclear FlashesThe Astrophysical Journal Supplement Series, 2000
- On implementing MPI-IO portably and with high performancePublished by Association for Computing Machinery (ACM) ,1999
- Data sieving and collective I/O in ROMIOPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1999
- File-access characteristics of parallel scientific workloadsIEEE Transactions on Parallel and Distributed Systems, 1996
- Parallel I/O Workload Characteristics Using VestaPublished by Springer Nature ,1996
- Implementation and performance of a parallel file system for high performance distributed applicationsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1996
- I/O requirements of scientific applications: an evolutionary viewPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1996