A case study in application I/O on Linux clusters
- 10 November 2001
- proceedings article
- Published by Association for Computing Machinery (ACM)
Abstract
A critical but often ignored component of system performance is the I/O system. Today's applications demand a great deal from underlying storage systems and software, and both high-performance distributed storage and high level interfaces have been developed to fill these needs.In this paper we discuss the I/O performance of a parallel scientific application on a Linux cluster, the FLASH astrophysics code. This application relies on three I/O software components to provide high-performance parallel I/O on Linux clusters: the Parallel Virtual File System, the ROMIO MPI-IO implementation, and the Hierarchical Data Format library. Through instrumentation of both the application and underlying system software code we discover the location of major software bottlenecks. We work around the most inhibiting of these bottlenecks, showing substantial performance improvement. We point out similarities between the inefficiencies found here and those found in message passing systems, indicating that research in the message passing field could be leveraged to solve similar problems in high-level I/O interfaces.Keywords
This publication has 5 references indexed in Scilit:
- Support for parallel out of core applications on Beowulf workstationsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- FLASH: An Adaptive Mesh Hydrodynamics Code for Modeling Astrophysical Thermonuclear FlashesThe Astrophysical Journal Supplement Series, 2000
- Toward Scalable Performance Visualization with JumpshotThe International Journal of High Performance Computing Applications, 1999
- On implementing MPI-IO portably and with high performancePublished by Association for Computing Machinery (ACM) ,1999
- A high-performance, portable implementation of the MPI message passing interface standardParallel Computing, 1996