Xen-Based HPC: A Parallel I/O Perspective
- 1 May 2008
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 154-161
- https://doi.org/10.1109/ccgrid.2008.119
Abstract
Virtualization using Xen-based virtual machine environment has yet to permeate the field of high performance computing (HPC). One major requirement for HPC is the availability of scalable and high performance I/O. Conventional wisdom suggests that virtualization of system services must lead to degraded performance. In this presentation, we take on a parallel I/O perspective to study the viability of Xen-based HPC for data-intensive programs. We have analyzed the overheads and migration costs for parallel I/O programs in a Xen-based virtual machine cluster. Our analysis covers PVFS-based parallel I/O over two different networking protocols: TCP-based Gigabit Ethernet and VMM-bypass InfiniBand. Our experimental results suggest that network processing in Xen-based virtualization can significantly impact the performance of Parallel I/O. By carefully tuning the networking layers, we have demonstrated the following for Xen-based HPC I/O: (I) TCP offloading can help achieve low overhead parallel I/O; (2) parallel reads and writes require different network tuning to achieve good I/O bandwidth; and (3) Xen-based HPC environment can support high performance parallel I/O with both negligible overhead and little migration cost.Keywords
This publication has 11 references indexed in Scilit:
- Proactive fault tolerance for HPC with Xen virtualizationPublished by Association for Computing Machinery (ACM) ,2007
- A case for high performance computing with virtual machinesPublished by Association for Computing Machinery (ACM) ,2006
- Transparent, Incremental Checkpointing at Kernel Level: a Foundation for Fault Tolerance for Parallel ComputersPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Diagnosing performance overheads in the xen virtual machine environmentPublished by Association for Computing Machinery (ACM) ,2005
- Architectural Characterization of TCP/IP Packet Processing on the Pentium® M MicroprocessorPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2005
- Parallel netCDFPublished by Association for Computing Machinery (ACM) ,2003
- Xen and the art of virtualizationPublished by Association for Computing Machinery (ACM) ,2003
- An abstract-device interface for implementing portable parallel-I/O interfacesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Size Scaling of Turbulent Transport in Magnetically Confined PlasmasPhysical Review Letters, 2002
- Scale and performance in the Denali isolation kernelPublished by Association for Computing Machinery (ACM) ,2002