Dynamic file-access characteristics of a production parallel scientific workload
- 17 December 2002
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
Abstract
Multiprocessors have permitted astounding increases in computational performance, but many cannot meet the intense I/O requirements of some scientific applications. An important component of any solution to this I/O bottleneck is a parallel file system that can provide high-bandwidth access to tremendous amounts of data in parallel to hundreds or thousands of processors. Most successful systems are based on a solid understanding of the expected workload, but thus far there have been no comprehensive workload characterizations of multiprocessor file systems. This paper presents the results of a three week tracing study in which all file-related activity on a massively parallel computer was recorded. Our instrumentation differs from previous efforts in that it collects information about every I/O request and about the mix of jobs running an a production environment. We also present the results of a trace-driven caching simulation and recommendations for designers of multiprocessor file systems.Keywords
This publication has 18 references indexed in Scilit:
- Multiprocessor file system interfacesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Dynamic file-access characteristics of a production parallel scientific workloadPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- A study of I/O behavior of Perfect benchmarks on a multiprocessorPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- High-performance I/O for massively parallel computers: problems and prospectsComputer, 1994
- An experimental performance evaluation of Touchstone Delta Concurrent File SystemPublished by Association for Computing Machinery (ACM) ,1993
- Performance Measurement of the Concurrent File System of the Intel iPSC/2 HypercubeJournal of Parallel and Distributed Computing, 1993
- Caching and Writeback Policies in Parallel File SystemsJournal of Parallel and Distributed Computing, 1993
- Directory reference patterns in hierarchical file systemsIEEE Transactions on Knowledge and Data Engineering, 1989
- A trace-driven analysis of the UNIX 4.2 BSD file systemPublished by Association for Computing Machinery (ACM) ,1985
- The DEMOS file systemPublished by Association for Computing Machinery (ACM) ,1977