View I/O: improving the performance of non-contiguous I/O
- 1 January 2003
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
Abstract
This paper presents view I/O, a non-contiguous parallel I/O technique. We show that the linear file model may be an unsuitable abstraction for non-contiguous I/O optimizations. Additionally, the poor cooperation between a file system and an I/O library like MPI-IO may drastically affect the performance. View I/O has detailed knowledge about parallel structure of a file and about the potential access pattern and exploits it in order to improve performance. The access overhead is reduced by using a strategy "declare once, use several times" and by file off-set compaction. We compare and contrast view I/O with other non-contiguous I/O methods. Our measurements on a cluster of computers indicate a significant performance improvement over other approaches.Keywords
This publication has 7 references indexed in Scilit:
- Clusterfile: a flexible physical layout parallel file systemPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2001
- Data sieving and collective I/O in ROMIOPublished by Institute of Electrical and Electronics Engineers (IEEE) ,1999
- A Comparison of Logical and Physical Parallel I/o pAtternsThe International Journal of High Performance Computing Applications, 1998
- Workload characterization of input/output intensive parallel applicationsPublished by Springer Nature ,1997
- File-access characteristics of parallel scientific workloadsIEEE Transactions on Parallel and Distributed Systems, 1996
- The Vesta parallel file systemACM Transactions on Computer Systems, 1996
- High performance FortranIEEE Parallel & Distributed Technology: Systems & Applications, 1993